How to use AutoTest |
![]() ![]() |
AutoTest has a command-line interface that is easy and intuitive to use.
Command-line arguments
usage: auto_test [--help] [--version] [--verbose] [--just-test] [--define=<...>] [--output-dir=<...>] [--time-out=<min>] ace_filename class_name+
--help
Displays the above line about possible command-line options
--version
Displays currently running version of AutoTest
--verbose
Displays information about testing progress
--just_test
It allows you to re-use a previously generated interpreter. If your system under test and the reflectable classes have not changed, you can set this option and it will save you some (significant) time when running AutoTest.
--define
For xace variables (see gobo documentation)
--output_dir
Everything that AutoTest generates (the interpreter, log file, test results) is created in this directory
--time-out
How long AutoTest runs in minutes (10 minutes by default)
ace_filename
Ace/xace file containing the system under test
class_name+
List of classes (separated by spaces) that will be tested.
Running the example
The AutoTest distribution comes with an example (under ${AUTO_TEST}/example/hello_world) that should give you a kick-start on using the tool. The root class (HELLO_WORLD) has a routine that will always violate its postcondition.
To test this class:
- cd to the hello_world directory under the example directory
- On the command line type:
auto_test --verbose --time-out=15 ise.ace HELLO_WORLD
This assumes that you have updated your PATH environment variable to include the bin directory from your AutoTest installation (as indicated in the installation instructions) and that you want to test the class for 15 minutes.
- You will then see several output lines of the form:
0:4:29.824: selected HELLO_WORLD.conforms_to
The numbers in front represent the time that still remains in the format <hours>:<minutes>:<seconds>.<milliseconds>. Next you can see the routines that get called (to be tested).
- When testing is finished, AutoTest displays some information about the location of the results:
Testing finished. To view the results direct your HTML viewer to file 'auto_test_gen\result\index.html'.
- HTML output.When you open the indicated file, in the frame on the right you can see what has been tested. The classes are grouped under 2 categories: "Classes selected for testing" contains the classes that you provided on the command line; "Other classes" contains the classes that were tested on the side (routines that were called directly or indirectly by the routines under test, creation procedures called to create targets and arguments for the tested routines, etc.). Click on the name of the whole system, of a class, or of a feature to get its test result statistics. The numbers in the "Test Case Summary" represent numbers of feature calls. Additionally, for features you can see results for individual test cases. For valid test cases (the ones that satisfied the precondition of the routine under test), you can see the output of the interpreter. This is meant to help you especially for failed test cases to see the reason for the failure. If you click on test case results or overall results for a class or a feature (pass, fail, or bad response), you will get a popup explaining what that particular result type means.
![]() ![]() ![]() ![]() |