New packaging equals new tools

Sometimes a new package gives new life to existing functionality.

I am working to track down a resource leak in our project at work, that only occurs during rare instances. Of course, I am using NeoLoad to stress the system, but since it appears to be a resource leak. I was planning to use the built-in JVM monitoring tools for doing what I need to do.

Those tools are generally commandline tools and I always have to refresh my memory about how they work.

But lo and behold, Sun had a brilliant idea in the package a front end interface called "Java Visual VM" that combines the profiling, monitoring, and heap dump capabilities all in one.

Go ahead and try it out. It should probably become a habit to routinely monitor applications you're working on when running unit tests and such. And since it is built into the 6.0 JDK , Sun has just lowered the bar to doing that kind of monitoring.

Very useful and built into your JVM.

Short Tour Testing

This is a integration or system-level testing technique that scales well and works at the unit testing level as well.

I originally discovered it in an article by Tom Cargill in C++ Report many moons ago (see below). I have not found any electronic descriptions of the technique so I figured I would revive it for those who would find it useful.

Cargill appears to have originally derived the process from a text on validating computer protocols by Holzmann (see below). Having read the book thoroughly, I can see where he derived it from, though would not have occurred to me to do so.

The basic design and concept is very simple: for any mixture of states and transitions that can be walked through in some complex sequence to produce a bug there is a short tour (three to five steps) through the same set of states and transitions that will give you the same bug.

I found this to be invaluable for pounding on APIs in order to validate that they have the correct mixture of correct error handling and correct functionality.

I typically implement this is in Java in the pseudocode looks something like this:

int tour = 1001;

int numberOfMethods = 13;

int numberOfParamSetsPerMethod = 20;

for ( int step = tour; step--; step > 0)

{

int methodIndex = step % numberOfMethods;

int paramSetIndex = step % numberOfParamSetsPerMethod;

invoke(methodIndex , paramSetIndex );

}

The end result of this is a predictable "drunken walk" through the combinations of methods and parameters.

Of course, the code can be made even simpler using the reflection API in Java.

Once the tour code has been designed, the test is invoked and the results of the tour are validated by eye. Typically at that point I save the results of the log of the tour so that it can be programmatically compared against the test results each time.

If the parameter sets are chosen well, this form of testing will go a long way towards discovering interaction issues in the system. I have had great success using scripting languages such as Lua to call C language APIs to do this kind of testing. I have also used it to test service architectures to expose session managementand exception handling issues.

1. Cargill, Tom, "Short Tour Testing", C++ Report, vol 7, no. 2, February 1995, pp 60-62.

2. Holzmann, Gerard, "Design and Validation of Computer Protocols", Prentice Hall (c) 1991

Text processing without the pain

Don't get me wrong, I love sed and awk. I have whole libraries of sed and awk scripts for doing all sorts of things. But some of them took a lot longer to write than they should've.

Last night I was faced with the task of translating a whole bunch of documentation from Tex and Latex to Docbook 5.0 XML. That meant doing multiline matches with sed and some preprocessing with awk, and my spirit rebelled.

I went beserk with online searches for "alternatives to sed awk" "text processing commandline utilities" and so on. The problem is that the standard text processing utilities need so much explanation that tutorials on how to do things with sed and awk are churned out so that they outnumber the alternatives by at least an order of magnitude difference.

I finally did what I should have done in the first place. I went to sourceforge.net and searched for "text processing". And I found Gema (pause for heavenly choir music).

It is not perfect by any means. the documentation in particular is just as cryptic as the original sed man pages. But in less than a half hour I had a script up and running that cleanly handled the multiline matches I needed to do.

As an example:

\\item\[*\]#\\=($1)#\n

will match the following

\\item[First Name]

The first name of the individual

This should not include any periods or commas.

And print out the following:

(First Name) The first name of the individual

This should not include any periods or commas

It's clean, it appears to be quick though none of the files I used it on with that large. It is well worth looking at. You also might want to take a look at this article.

Load Testing Tools

Apologies for the delay is in posting. My company has been dealing with a customer problem that required some significant load testing. The previous set of load tests had appeared to miss something and we had to go back to the drawing board. The person who owned and managed the previous set of load tests was no longer here, and the load test framework, while documented, was not easy to extend.

So we went looking for a load test tool to give us a leg up quickly. Unfortunately, even though open source is my default choice I was not able to get any of the open choice solutions up and running quickly while giving me scope for quickly adapting to different load test scenarios. If I had had a few more days I probably could've put an open-source solution in place that would give me the same responsiveness as the proprietary solutions.

It very quickly came down to the proprietary solutions of NeoLoad and PureLoad. Almost all of the other proprietary solutions were too expensive or took too long to set up and configure.

Of the two, NeoLoad passed the five-minute test with flying colors. After downloading the install. It took me less than four minutes to start running load tests. If you are unfamiliar with the language of load testing, it may take you as long as a half hour to understand the documentation. I recommend looking at the Wikipedia entries for load testing first before attempting to use any load testing tool. The terminology can be misleading when you're first exposed to it.

PureLoad almost passed that test. I was able to get the free (PureTest) component that allows you to record and run the tests themselves up and running in five minutes.They have a separate tool called PureLoad tool that allows you to run the same tests as load tests. I was not able to get that up and running in the types of scenarios I wanted and my call for pre-sales support went 48 hours before I got a response. By that time, I had already committed to going with NeoLoad.

With NeoLoad, we have the license for 60 simultaneous virtual users within an hour after they received a purchase order. I was up and running load tests within the next half hour. The fact that they had an eval license that allowed me to prepare all of the tasks ahead of time and run them with three simultaneous users, allowed me to ramp up before I even got the full license. That is what I like!

Sidenote: What was truly frustrating was the fact that when I did get a presales support call back from PureLoad, the individual in question tried to convince me to change my mind and use the PureLoad product on the basis of price even when I highlighted to him that the reason for going with NeoLoad was speed of setup and speed of response. I pointed this out to him two times. After the second time, when he again brought up the subject of price, I had to make it clear that no means no. This is why I consider communication and listening skills to be critical for any kind of presales support. If, for some reason, NeoLoad does not work out I will be reluctant to go back to the PureLoad people simply because I don't want to deal with people that don't listen.

The build lifecycle

One of the things that is critical to any build system ( and this is something that the Maven guys nailed) is that it have a clearly defined lifecycle.

For myself over the last 20 years I have developed a build lifecycle that appears to answer my needs for every build.

• Clean - Remove all build artifacts from the file system.
• Init - Initialize the build system. All properties should be set here.
• Prep - Prepare the file system for the build. Create folders as necessary.
  
• GetDependencies - Resolve all dependencies. Locate and pull down the necessary dependencies and make them available to the build
• Gen - Generate or Preprocess source code and resources of any type.
• Build - Build anything that can be compiled or assembled.
• UnitTest - Execute any tests that can be performed without deployment
• Pkg - Package anything that can be packaged
• Verify - Establish the internal integrity of the package
• Deploy - Deploy any packages that need to be deployed.
• SmokeTest - Execute any smoke tests against the deployed application(s)
  
• Stage - Publish the build artifacts for local (i.e. machine local) usage.
  
• Share - Publish the build artifacts for team wide usage.
• Release - Publish the build artifacts for general (i.e. network) usage.
• IntegrationTest - Execute any smoke tests

Most of that is not new. You have probably seen build systems that use subsets or supersets of these. The key is in making it very easy to plug in to a lifecycle event so that you can perform additional actions as necessary. Doing that in Ant takes some very careful thought but produces some very clean results.

It does require some conventions on the project and source code structure side of things. Another thing that the Maven guys nailed. :-)

The directory structure conventions I am using with my build system are the following:

I have a top level [project] Folder with several folders underneath.

• [project]/root - This is the top level location from which you do builds that execute against everything. Commands like "CleanAll", "BuildAll", etc...
  
• [project]/common-build - This contains the common build system I use across all projects. It is a separate project in my revision control system and is typically pulled down into a location under the project using Subversion's externals command or using Git's submodules.
  
• [project]/components - This folder contains all of the sub projects that make up the components of this project.
  
• [project]/apps - This folder contains all of the subprojects that make up the applications of this project.
  
• [project]/installers - This folder contains all of the subprojects used to generate the installers of this project.
  

Under each components or apps or installers folder is a named subproject with a standard structure.

For example: The project "calendar-server" with the "calendar-utils" component would look like this:

calendar-server/components/calendar-utils

The component has a series of files. The build.xml file is of course the ant file for the component. The ivy.xml is a very simple file that describes what artifacts the component is dependent on. And the .project and .classpath files are those files needed by Eclipse.

calendar-utils/build.xml
calendar-utils/ivy.xml
calendar-utils/.project
calendar-utils/.classpath

And the calendar-utils component would have a checked-in folder structure that looks like this:

calendar-utils/src/main/java
calendar-utils/src/main/conf
calendar-utils/src/test/java
calendar-utils/src/test/conf

The source trees are separated for several reasons. Often the Java test code is compiled , managed, preprocessed, executed or referenced separately from the main code. The conf folders are also separate for some of the same reasons as well as allowing separate configurations to be set up for test and actual deployment.

There are also a series of common folders that are typically generated during the build. the first set of those is for generated code and those follow the same conventions as the checked source for much the same reasons.

calendar-utils/gen-src/main/java
calendar-utils/gen-src/main/conf
calendar-utils/gen-src/test/java
calendar-utils/gen-src/test/conf

The other set of folders is for the targets or results of the build. They all lie under the target folder. The classes and test classes are kept separate for the same reasons the sources are kept separate. The distrib folder contains all of the final output artifacts of the build ( jars, documents, zip files and executables etc.). The reports folder contains reports on the tests, profiling, etc.

calendar-utils/target/classes
calendar-utils/target/test-classes
calendar-utils/target/distrib
calendar-utils/target/reports

So to clean up this directory structure for a new pristine build only requires us to delete the gen-src and target folders.

In the next post I will talk about the design of the Ant build files found in the components and the common-build folder.

Build Systems : Ant versus Maven

Ever since I discovered Make (25+ years ago) I have been searching for a good build system. I have used everything from Configure and Make (talk about icing on a mud pie) to JAM and now Ant and Maven.

I keep going back and trying Maven again when they do a new release. It is such a good idea that I keep going back in the hopes that the implementation and documentation will finally live up to that promise. And I think a good number of people stay with Maven because it is such a good idea that they persevere and endure the slings and arrows of outrageous documentation and implementation. Alas, each time I come away frustrated.

The Maven repository concept is pure genius. And in fact, the implementation works well enough that I use it in connection with Apache's Ivyto do dependency management.

What is Ivy? A set of dependency management tasks used by Ant to pull down and access the appropriate jar or other dependencies needed by your project. I won't go into a tutorial about Ivy since there are more than a few out there. But the project itself is available at http://ant.apache.org/ivy/. it does suffer from some of the same documentation issues that Maven does but between the online forums and other peoples blog posts you can usually figure something out pretty quickly.

In the following few posts I will be discussing how I use Ivy in connection with Ant to produce a fairly clean build system with minimal bootstrap requirements.

Revision Control: GIT

I love Subversion for revision control. It has a lot of power. And every once in awhile I need more power.

For those moments, there is GIT. It is blazing fast and industrial-strength. Unfortunately, reading the user documentation has my head ready to explode. There is this vague haunting of concepts just out of reach with explanations that almost, but don't quite give you the clue I need.

For those people who want a clear explanation of what it does, how it does it, and the concepts behind it so that you can manipulate it well; here is the book you need: "Pragmatic Version Control using Git" by Travis Swicegood.

Go to the Pragmatic Programming website and download the PDF for $22. It is clear, it is straightforward, it tells you what's going on in the background as well as having simple straightforward examples.