Short Tour Testing

This is a integration or system-level testing technique that scales well and works at the unit testing level as well.

I originally discovered it in an article by Tom Cargill in C++ Report many moons ago (see below). I have not found any electronic descriptions of the technique so I figured I would revive it for those who would find it useful.

Cargill appears to have originally derived the process from a text on validating computer protocols by Holzmann (see below). Having read the book thoroughly, I can see where he derived it from, though would not have occurred to me to do so.

The basic design and concept is very simple: for any mixture of states and transitions that can be walked through in some complex sequence to produce a bug there is a short tour (three to five steps) through the same set of states and transitions that will give you the same bug.

I found this to be invaluable for pounding on APIs in order to validate that they have the correct mixture of correct error handling and correct functionality.

I typically implement this is in Java in the pseudocode looks something like this:

int tour = 1001;

int numberOfMethods = 13;

int numberOfParamSetsPerMethod = 20;

for ( int step = tour; step--; step > 0)

{

int methodIndex = step % numberOfMethods;

int paramSetIndex = step % numberOfParamSetsPerMethod;

invoke(methodIndex , paramSetIndex );

}

The end result of this is a predictable "drunken walk" through the combinations of methods and parameters.

Of course, the code can be made even simpler using the reflection API in Java.

Once the tour code has been designed, the test is invoked and the results of the tour are validated by eye. Typically at that point I save the results of the log of the tour so that it can be programmatically compared against the test results each time.

If the parameter sets are chosen well, this form of testing will go a long way towards discovering interaction issues in the system. I have had great success using scripting languages such as Lua to call C language APIs to do this kind of testing. I have also used it to test service architectures to expose session managementand exception handling issues.

1. Cargill, Tom, "Short Tour Testing", C++ Report, vol 7, no. 2, February 1995, pp 60-62.

2. Holzmann, Gerard, "Design and Validation of Computer Protocols", Prentice Hall (c) 1991