Test execution: the driver and the interpreter

PreviousNext

Overview

AutoTest uses a 2-process model when executing tests:

This separation allows for increased robustness of the testing process. If executing one of the routines under test produces an error from which the process performing the call can't recover, in our model only the interpreter will be affected. The driver will wait for a message from the interpreter for a fixed time, and then kill the interpreter process and restart it, resuming testing from where it left off. If several calls to the same routine cause the interpreter process to become unstable, the driver can decide to abandon testing that routine altogether and go on to testing the next one.

The interpreter

The interpreter is a component that can be used on its own and is also delivered with the Eiffel Reflection Library Generator (ERL-G). It can execute simple commands and returns status messages. For more information on these commands and messages see the documentation available with the ERL-G tool.

The interpreter uses ERL-G to be able to dynamically call routines. The system under test is given as input to ERL-G, which generates corresponding reflectable types for all alive types in the system. Types are considered to be alive according to the following algorithm:

  1. Types corresponding to the classes under test (given on the command-line) are alive. When the user wants to test a generic class, the generic parameter(s) will be instantiated with objects of the constraining type (or one of its closest effective descendants if the contraining type is deferred).
  2. Types in the transitive closure of the above-mentioned types are alive.

The interpreter then uses these generated reflectable types to call the features under test.

Log files

Both the driver and the interpreter write the messages that they exchange during test execution to log files. These files are very important because they allow you to reconstruct the call sequences that lead to failed test cases - in other words, to extract bug-reproducing examples. The logs can be found in directory auto_test_gen/log under the names proxy_log.txt and interpreter_log.txt.

The proxy is the part of the driver process whose only job is to interact with the interpreter. In the proxy log you can see the commands that the driver issued to the interpreter, and the responses that it got back. The latter are listed as comments.

In the interpreter log you can see a list of the requests that the interpreter received, and information about any exceptions thrown during execution of these requests.


HomeTocPreviousNext