Taking things public

About two weeks ago I realized that I was creating this project as an open source project but not taking it out into the open source community.

Anybody see a small disconnect there?

So, I have started a project (called saltation) on SourceForge and I will be checking in code within 24 hours.

And, I have already run into my first roadblock (or at least a speed bump).

Specifically, it is the build system. I use a common build framework for all of my projects which I call appropriately enough, common-build. It uses Ant and Ivy for the build and dependency management. For a little more information on that see this posting in my other blog : Ant versus Maven.

Both Maven and Ivy use an external repository for storing dependencies such as Java libraries. SourceForge still hasn't worked out all the ways to interact with this. so the bottom line is one of my first tasks is going to be making the build system require a minimum of setup (. I.e. you should be able to build right out of the box).

That will probably take me a day or two counting testing.

Till then,

- Jim

More in the world of unused code detection : Klocwork

As noted in a previous posting, I have a major refactoring task ahead of me with the code base that I am now the owner of.

Recently we had an intermittent problem that may have been the result of a resource leak. Because we were unable to reproduce it, we put some processes in place for the next time and I did what I always do. Look to see if there is some tool out there that will allow me to detect resource leaks in the code in the current branch of the code base.

The two that most people seem to recommend are Coverity and Klocwork. A number of my acquaintances have said that Klocwork was better at detecting resource leaks so I decided to try it.

Here is the good news: the tool mostly does the job.

The bad news : the company doesn't quite have it together.

In one key way it does: they are curteous, efficient and smart. The sheer efficiency and overall effectiveness highlights even more their marketing/sales deficiencies.

I think the key thing that I kept bumping into is that there were limitations and conditions that were not clearly spelled out during the purchasing process. Klocwork had a salesperson and a technician on the phone with me at their request to evaluate my needs. The decision was made for me to go with the Klocwork Solo product rather than the honking big enterprise licensed behemoth that is their main line product. When I pulled down the demo version of Klocwork Solo I discovered that the temporary license covered more than a month but that the demo version could only handle 99 files at a pop and the Solo product could only run under Windows even though it is a Java application ( it appears to spawn a Windows executable as part of its analysis process).

Neither of these facts came to the surface in the initial call.

Personally, I would recommend that when you are making available a trial application of a code analysis tool, that you make it a short period of time AND allow it to handle an enterprise class number of files. After all, you know people are going to use it on their code base, and will need to do so in order to demonstrate its worth to the powers that be.

The 99 file limitation was a pain, but I evaluated the tool enough to determine but it was probably worth the $100 it costs to get it in and try it out on a larger code base.

I purchased the $100 Solo product and discovered that it was limited to 1000 files (but apparently I can call Klocwork and get that expanded). Luckily, I was able to limit the code base I was interested in to 1000 files.

But the other thing I discovered is that the licensing tool appears to talk with the Klocwork mothership very time I start up Eclipse and the license is only valid for a year. That was another detail that was not presented upfront.

I don't want to leave you with the idea that I think that Klocwork is intentionally misleading people. I don't think that. But I do think that the management of licenses and sales are oriented toward a different scale of user than a small shop of 3 to 5 developers.

I don't think I will be recommending their tools until they rapidly revise their licensing and license management.

- Jim

Technology choices revisited

In one of my earlier posts Technology Choices I had looked at what user interface toolkit I should be using. What I've discovered in the intervening time is that this application is going to put a premium on flexibility in presentation. As a result, I am seriously reconsidering the user interface toolkit. The Eclipse RCP is a very powerful platform upon which to build an application. Unfortunately it does have a very strong set of metaphors upon which it is built (Views and Workspaces) which seem too rigid for what I am attempting.

So I am seriously looking at using QT with the Java bindings.

Any thoughts?

Going to and fro

The other part of the last few months has been looking at importing and exporting and synchronizing.

Spent a great deal of time looking at the iCalendar specification and all of the recurrence rules. And then I read all of the use cases for recurrence rules interoperability that have been put together by some incredibly diligent groups of people. The people that work on those specs definitely come under my heading of unsung heroes.

As often appears to be the case in the area of calendaring, there are really no clean answers. And, going back and forth between the icalender format and a more flexible internal model is going to require a great deal of detailed work. But, it is very clear that iCalendar is the de facto standard for now and I see nothing on the horizon yet that will replace it.

And personally, I think it gives the biggest bang for the buck. If I can support CalDAV and ICalendar to any significant degree I will be able to publish calendars to and from Google Calendar , as well as many others applications.

So my immediate attention is to support import/export for ICalendar , and then support using CalDAV to publish.

- Jim

When things don't fit into a neatly modeled the world

I am back after a long hiatus. A combination of work plus a course I'm teaching has kept me very busy.

But always in the background I am thinking about the PIM. That's how we referred to it in my household "The PIM". I would wonder if I'm a little obsessed except I'm having so much fun.

I've been coding some commandline utilities to test the data model and, as I expected, I have run into the limitations of my original model. I had originally started with calendars, and agendas and so on being first class objects. What I mean by that is that they have some distinct existence in the real world , and that is why I am modeling them. What I have discovered is that calendars, address books, task lists, journals and so on are all simply semantically themed collections of stuff we care about.

There are few assertions I can make about what a calendar is that are universal enough to be easily recognizable to everyone.

So I stepped back and thought about it for a bit and this is what I have come up with: we have an ocean of first-class objects that we would like to track (incense, tasks, promises, contact information, little notes) and so on. And then we spend our time trying to organize them into multiple collections they give us easy access to what we want to do when we want to do.

So I am experimenting with having any of these collections simply be taxonomies. Just like described in an earlier post. This would mean that many calendars have a relationship to each other. An example would be the calendar I maintain for my children's schedule all together which could have two related calendars ( one for Aidan and one for Trent ) . both Trent and Aidan's calendar would refer to events in the Boy Scout calendar.

The same would be true of address books. I could have an address book that references my friends and a related address book that only has the subset of my college friends.

The same appears to work well for journals, agendas, sets of conversations, and the collection of resources and notes that I refer to as Data Mines or just Mines.

New packaging equals new tools

Sometimes a new package gives new life to existing functionality.

I am working to track down a resource leak in our project at work, that only occurs during rare instances. Of course, I am using NeoLoad to stress the system, but since it appears to be a resource leak. I was planning to use the built-in JVM monitoring tools for doing what I need to do.

Those tools are generally commandline tools and I always have to refresh my memory about how they work.

But lo and behold, Sun had a brilliant idea in the package a front end interface called "Java Visual VM" that combines the profiling, monitoring, and heap dump capabilities all in one.

Go ahead and try it out. It should probably become a habit to routinely monitor applications you're working on when running unit tests and such. And since it is built into the 6.0 JDK , Sun has just lowered the bar to doing that kind of monitoring.

Very useful and built into your JVM.

Short Tour Testing

This is a integration or system-level testing technique that scales well and works at the unit testing level as well.

I originally discovered it in an article by Tom Cargill in C++ Report many moons ago (see below). I have not found any electronic descriptions of the technique so I figured I would revive it for those who would find it useful.

Cargill appears to have originally derived the process from a text on validating computer protocols by Holzmann (see below). Having read the book thoroughly, I can see where he derived it from, though would not have occurred to me to do so.

The basic design and concept is very simple: for any mixture of states and transitions that can be walked through in some complex sequence to produce a bug there is a short tour (three to five steps) through the same set of states and transitions that will give you the same bug.

I found this to be invaluable for pounding on APIs in order to validate that they have the correct mixture of correct error handling and correct functionality.

I typically implement this is in Java in the pseudocode looks something like this:

int tour = 1001;

int numberOfMethods = 13;

int numberOfParamSetsPerMethod = 20;

for ( int step = tour; step--; step > 0)

{

int methodIndex = step % numberOfMethods;

int paramSetIndex = step % numberOfParamSetsPerMethod;

invoke(methodIndex , paramSetIndex );

}

The end result of this is a predictable "drunken walk" through the combinations of methods and parameters.

Of course, the code can be made even simpler using the reflection API in Java.

Once the tour code has been designed, the test is invoked and the results of the tour are validated by eye. Typically at that point I save the results of the log of the tour so that it can be programmatically compared against the test results each time.

If the parameter sets are chosen well, this form of testing will go a long way towards discovering interaction issues in the system. I have had great success using scripting languages such as Lua to call C language APIs to do this kind of testing. I have also used it to test service architectures to expose session managementand exception handling issues.

1. Cargill, Tom, "Short Tour Testing", C++ Report, vol 7, no. 2, February 1995, pp 60-62.

2. Holzmann, Gerard, "Design and Validation of Computer Protocols", Prentice Hall (c) 1991