Technology choices revisited

In one of my earlier posts Technology Choices I had looked at what user interface toolkit I should be using. What I've discovered in the intervening time is that this application is going to put a premium on flexibility in presentation. As a result, I am seriously reconsidering the user interface toolkit. The Eclipse RCP is a very powerful platform upon which to build an application. Unfortunately it does have a very strong set of metaphors upon which it is built (Views and Workspaces) which seem too rigid for what I am attempting.

So I am seriously looking at using QT with the Java bindings.

Any thoughts?

Going to and fro

The other part of the last few months has been looking at importing and exporting and synchronizing.

Spent a great deal of time looking at the iCalendar specification and all of the recurrence rules. And then I read all of the use cases for recurrence rules interoperability that have been put together by some incredibly diligent groups of people. The people that work on those specs definitely come under my heading of unsung heroes.

As often appears to be the case in the area of calendaring, there are really no clean answers. And, going back and forth between the icalender format and a more flexible internal model is going to require a great deal of detailed work. But, it is very clear that iCalendar is the de facto standard for now and I see nothing on the horizon yet that will replace it.

And personally, I think it gives the biggest bang for the buck. If I can support CalDAV and ICalendar to any significant degree I will be able to publish calendars to and from Google Calendar , as well as many others applications.

So my immediate attention is to support import/export for ICalendar , and then support using CalDAV to publish.

- Jim

When things don't fit into a neatly modeled the world

I am back after a long hiatus. A combination of work plus a course I'm teaching has kept me very busy.

But always in the background I am thinking about the PIM. That's how we referred to it in my household "The PIM". I would wonder if I'm a little obsessed except I'm having so much fun.

I've been coding some commandline utilities to test the data model and, as I expected, I have run into the limitations of my original model. I had originally started with calendars, and agendas and so on being first class objects. What I mean by that is that they have some distinct existence in the real world , and that is why I am modeling them. What I have discovered is that calendars, address books, task lists, journals and so on are all simply semantically themed collections of stuff we care about.

There are few assertions I can make about what a calendar is that are universal enough to be easily recognizable to everyone.

So I stepped back and thought about it for a bit and this is what I have come up with: we have an ocean of first-class objects that we would like to track (incense, tasks, promises, contact information, little notes) and so on. And then we spend our time trying to organize them into multiple collections they give us easy access to what we want to do when we want to do.

So I am experimenting with having any of these collections simply be taxonomies. Just like described in an earlier post. This would mean that many calendars have a relationship to each other. An example would be the calendar I maintain for my children's schedule all together which could have two related calendars ( one for Aidan and one for Trent ) . both Trent and Aidan's calendar would refer to events in the Boy Scout calendar.

The same would be true of address books. I could have an address book that references my friends and a related address book that only has the subset of my college friends.

The same appears to work well for journals, agendas, sets of conversations, and the collection of resources and notes that I refer to as Data Mines or just Mines.

New packaging equals new tools

Sometimes a new package gives new life to existing functionality.

I am working to track down a resource leak in our project at work, that only occurs during rare instances. Of course, I am using NeoLoad to stress the system, but since it appears to be a resource leak. I was planning to use the built-in JVM monitoring tools for doing what I need to do.

Those tools are generally commandline tools and I always have to refresh my memory about how they work.

But lo and behold, Sun had a brilliant idea in the package a front end interface called "Java Visual VM" that combines the profiling, monitoring, and heap dump capabilities all in one.

Go ahead and try it out. It should probably become a habit to routinely monitor applications you're working on when running unit tests and such. And since it is built into the 6.0 JDK , Sun has just lowered the bar to doing that kind of monitoring.

Very useful and built into your JVM.

Short Tour Testing

This is a integration or system-level testing technique that scales well and works at the unit testing level as well.

I originally discovered it in an article by Tom Cargill in C++ Report many moons ago (see below). I have not found any electronic descriptions of the technique so I figured I would revive it for those who would find it useful.

Cargill appears to have originally derived the process from a text on validating computer protocols by Holzmann (see below). Having read the book thoroughly, I can see where he derived it from, though would not have occurred to me to do so.

The basic design and concept is very simple: for any mixture of states and transitions that can be walked through in some complex sequence to produce a bug there is a short tour (three to five steps) through the same set of states and transitions that will give you the same bug.

I found this to be invaluable for pounding on APIs in order to validate that they have the correct mixture of correct error handling and correct functionality.

I typically implement this is in Java in the pseudocode looks something like this:

int tour = 1001;

int numberOfMethods = 13;

int numberOfParamSetsPerMethod = 20;

for ( int step = tour; step--; step > 0)

{

int methodIndex = step % numberOfMethods;

int paramSetIndex = step % numberOfParamSetsPerMethod;

invoke(methodIndex , paramSetIndex );

}

The end result of this is a predictable "drunken walk" through the combinations of methods and parameters.

Of course, the code can be made even simpler using the reflection API in Java.

Once the tour code has been designed, the test is invoked and the results of the tour are validated by eye. Typically at that point I save the results of the log of the tour so that it can be programmatically compared against the test results each time.

If the parameter sets are chosen well, this form of testing will go a long way towards discovering interaction issues in the system. I have had great success using scripting languages such as Lua to call C language APIs to do this kind of testing. I have also used it to test service architectures to expose session managementand exception handling issues.

1. Cargill, Tom, "Short Tour Testing", C++ Report, vol 7, no. 2, February 1995, pp 60-62.

2. Holzmann, Gerard, "Design and Validation of Computer Protocols", Prentice Hall (c) 1991

Text processing without the pain

Don't get me wrong, I love sed and awk. I have whole libraries of sed and awk scripts for doing all sorts of things. But some of them took a lot longer to write than they should've.

Last night I was faced with the task of translating a whole bunch of documentation from Tex and Latex to Docbook 5.0 XML. That meant doing multiline matches with sed and some preprocessing with awk, and my spirit rebelled.

I went beserk with online searches for "alternatives to sed awk" "text processing commandline utilities" and so on. The problem is that the standard text processing utilities need so much explanation that tutorials on how to do things with sed and awk are churned out so that they outnumber the alternatives by at least an order of magnitude difference.

I finally did what I should have done in the first place. I went to sourceforge.net and searched for "text processing". And I found Gema (pause for heavenly choir music).

It is not perfect by any means. the documentation in particular is just as cryptic as the original sed man pages. But in less than a half hour I had a script up and running that cleanly handled the multiline matches I needed to do.

As an example:

\\item\[*\]#\\=($1)#\n

will match the following

\\item[First Name]

The first name of the individual

This should not include any periods or commas.

And print out the following:

(First Name) The first name of the individual

This should not include any periods or commas

It's clean, it appears to be quick though none of the files I used it on with that large. It is well worth looking at. You also might want to take a look at this article.

Load Testing Tools

Apologies for the delay is in posting. My company has been dealing with a customer problem that required some significant load testing. The previous set of load tests had appeared to miss something and we had to go back to the drawing board. The person who owned and managed the previous set of load tests was no longer here, and the load test framework, while documented, was not easy to extend.

So we went looking for a load test tool to give us a leg up quickly. Unfortunately, even though open source is my default choice I was not able to get any of the open choice solutions up and running quickly while giving me scope for quickly adapting to different load test scenarios. If I had had a few more days I probably could've put an open-source solution in place that would give me the same responsiveness as the proprietary solutions.

It very quickly came down to the proprietary solutions of NeoLoad and PureLoad. Almost all of the other proprietary solutions were too expensive or took too long to set up and configure.

Of the two, NeoLoad passed the five-minute test with flying colors. After downloading the install. It took me less than four minutes to start running load tests. If you are unfamiliar with the language of load testing, it may take you as long as a half hour to understand the documentation. I recommend looking at the Wikipedia entries for load testing first before attempting to use any load testing tool. The terminology can be misleading when you're first exposed to it.

PureLoad almost passed that test. I was able to get the free (PureTest) component that allows you to record and run the tests themselves up and running in five minutes.They have a separate tool called PureLoad tool that allows you to run the same tests as load tests. I was not able to get that up and running in the types of scenarios I wanted and my call for pre-sales support went 48 hours before I got a response. By that time, I had already committed to going with NeoLoad.

With NeoLoad, we have the license for 60 simultaneous virtual users within an hour after they received a purchase order. I was up and running load tests within the next half hour. The fact that they had an eval license that allowed me to prepare all of the tasks ahead of time and run them with three simultaneous users, allowed me to ramp up before I even got the full license. That is what I like!

Sidenote: What was truly frustrating was the fact that when I did get a presales support call back from PureLoad, the individual in question tried to convince me to change my mind and use the PureLoad product on the basis of price even when I highlighted to him that the reason for going with NeoLoad was speed of setup and speed of response. I pointed this out to him two times. After the second time, when he again brought up the subject of price, I had to make it clear that no means no. This is why I consider communication and listening skills to be critical for any kind of presales support. If, for some reason, NeoLoad does not work out I will be reluctant to go back to the PureLoad people simply because I don't want to deal with people that don't listen.