Domain Model: Conversations

I spend many hours each week tracking conversations. Not the conversations like "hi how are you ....I am fine....", I am talking about conversations that call for actions. They can be actions on my part or actions that i am requesting of others. ..... Most of those conversations have the same pattern centered around making a request, or making a promise and tracking its completion. 

Ideally I would like to track the state of these conversations and many of these conversations occur in email and some occur in IM so any tool I write for managing conversations would reflect that common pattern.

For the academics out there the known model of interaction that encompasses my day to day conversations is based on something called "Speech Action" or "Language Action Perspective". A commercial tool (The Coordinator) has been built using it and seems to have a great deal to recommend it.  The basic ideas behind the Language/Action Perspective:

All human results are achieved through conversations (typically written and spoken)that invoke actions.  Most of us don't need to track conversations that don't lead to actions. Those conversations typically involve exchanges of information  .... 

 

Conversations that lead to actions usually require being tracked....and are effectively a dance...

Those conversations start with either a request for an action or an offer of an action. They can also start off with a "What If" type of conversation such as "Wouldn't it be cool if" and then result in requests or offers of action.

People can accept a request, decline a request, or make a counter offer).

Accepting a request typically involves making a promise. The promise can be completed or revoked.

At some point the conversation for action is complete.

So a tool for  tracking and managing  conversations would allow the following...

1. Opening a Conversation

1a.Generating a Request OR 
1b. Generating an Offer OR
1c. Starting a conversation for a Possibility.

2. Responding 

2a. Accept a request OR

2b. Decline a request OR

2c. Make a counteroffer to a request OR

2d. Comment on or annotate the conversation  OR

2e. Acknowledge someone

2f. Putting off a decision ( i.e. giving a promise by when you will respond)

2g. Make an interim report

2e. Report that the conversation and actions are complete

3. Track conversations (and their participants progress) by their status:

New conversations 

    Conversations missing my response

    Conversations missing other's response

My Promises 

My Requests

Commitments

This may seem incredibly structured , or even artificial, but if you look at the conversations that go into causing projects you will see that they all fall into this structure. Of course, most users would prefer not to use this format for all conversations (or even most of them).  So this should be something optional, where you can did note an e-mail thread as a conversation for action and then track the status of the conversation from that point forward.

What activities should a PIM support

When I first started this blog I naïvely thought that because I have been thinking about this for so long that I had it all firmly clear in my head, and it would pretty much come out in some sensible order.  

Instead I have discovered that along the way I will end up going back and documenting something that seems so self-evident that I have forgotten about it completely.

One example of that is really documenting what are the activities I expect to be able to do with the PIM. This is implicit in a lot of the previous posts, but it is definitely worth making it explicit here.

Basic activities that I expect a user to be able to do include the following:

Capturing and evaluating input: Being able to triage and classify events, tasks, contact information, and other references and notes as they come in. Being able to quickly decide what needs to be handled immediately, what can wait, and if something needs to be scheduled immediately the user should be able to do that quickly even if it means classifying it or scheduling it approximately. "oh look, a dinner invitation, I don't even need to look at it until next week,  so I will throw it into next week.". Being able to throw a note down into the system to be converted to an action or a task later.

Organizing: Going beyond the initial triage to the point where tasks and events are being scheduled in time (or at least the order is being set), as well as coordinating with others. Being able to overlay your own taxonomy/ontology/organizational structure onto the raw objects in the domain ( events, tasks, promises, actions, contacts, etc.). Being able to extend the raw objects of the domain so that they can be personalized to carry the information that the user needs.

Collaborating: Not only sharing calendars, but tracking conversations associated with shared activities.    Tracking the status and progress of projects from organizing your grandmothers birthday party all the way to coordinating the training of a large group of people.

Squirreling things away so the nuts can be found again:  one of the biggest battles for users of a PIM  is storing the data (documents and pictures and other information) associated with an event, a task,a person, or a group in such a way that it can be easily found when needed. 

Finding: Looking through information that you stored off to the side with the intention of finding it again.   Being able to reconstruct timelines based on e-mail conversations and scheduled events. 

Presenting: Visualizing and reporting on the data in a way that is natural for the user. Being able to generate reports on the status of projects from scheduling birthdays and tracking who is coming all the way to larger projects like the implementation of a training regimen for a group of trainers.

Playing with the data model

Hello folks, I am back.

I have been working on the PIM for a bit, but without the space to actually blog about it.

I started experimenting with actually code. I have been working on the basic data model classes and how well they model real-world situations. I am not yet at the point of publishing the code, but I'm starting to get to the point where I can actually think about it.

One of the things I've been working on was trying out the java persistence technology Known as DB 4O. 

My intention has been to find a database storage technology that handled hierarchical data well. Relational databases manage many situations very well but even the best ORM (Object Relational Mapping) solution tends to break down when handling deep, flexible and higly modifiable hierarchies.

In order to handle the kinds of events and extensibility that I am looking for.  I need to be able to store and quickly retrieve objects that are referenced from hierarchies/taxonomies where the objects themselvesstore general attributes.

To give you an example I want to be able to generate taxonomies that have a fairly deep structure ( friends-> college friends->the guys-> the wedding party) and be able to walk that taxonomy and quickly collect all the elements it points to.  Then, if necessary, I would like to be able to rearrange the hierarchy.  This is doable with a relational database and an ORM but no matter how carefully you balance the design, and inevitably there will be performance corner cases that are drastically inconsistent with the performance of the rest of the system. 

In addition, I want users to be able tocreate their own classes of events and tasks. For example, I would like users to be able to create specific events and add their own attributes. if a user would like to distinguish between kids birthday parties and general birthday parties, they should be able to simply create a kid's birthday party event with customized properties and then be able to search within those customized properties.

Generic attributes such as this often present headaches for ORM frameworks, though not nearly as bad as managing hierarchical data.

So I was pretty sold on using a decent object storage framework.  If I could find one! 

DB4O has pretty much answered my requirements.

Through a mixture of byte code instrumentation and good design the DB40 folks have achieved a minimally intrusive persistence framework that effectively speeds up development, unit testing, and general maintainability of the code.

To give you a code example:



// Open the DB

ObjectContainer db;

Configuration configuration = Db4o.newConfiguration();

configuration.generateUUIDs(ConfigScope.GLOBALLY);
configuration.generateVersionNumbers(ConfigScope.GLOBALLY);

db = Db4o.openFile( configuration, filename);

//
// Create a bunch of objects (no matter how deep the hierarchy
//

List items = methodThatCreatesAllTheObjects();

// Store the objects

db.store(items);

// Close the transaction

db.commit();



As you can see there is a minimum of explicit code by default. So I don't have to explicitly open a transaction, that is done when I attempt to store the items. 

There are of course, ways to explicitly do all of the actions necessary, but so far I haven't needed to go outside of the defaults. 

Metadata and Taxonomies (aka How do I find it my way?)

Some of the very best PIMs allow you to organize your data according to your needs. Usually that involves using tags or categories. And a great deal of discussion has gone into whether tagging or categorization is more effective for people who are tracking large amounts of information.

What is the difference? Tagging is like sewing little labels on your socks, categorizing is like picking the specific drawer to pick each type of sock in. I would suggest that the best overview document is Shirkey's document here. He very effectively details the strengths and weaknesses of both methods. I would also suggest Christian Ricci's discussion for very clean examples of solid uses of Taxonomies.

After reading Shirkey's notes you may be left with the idea that Tags are the "One True and Useful Way". To the extent that he talks about (i.e. the world of the web) that may be very true. But in the world of PIM the many weaknesses of Taxonomies become strengths for an individual user. The first person I found that seems to have firmly grasped that is Paul Chen here. And so what I am aiming for is the use of Taxonomies where the user finds it works for them and tags as necessary.

For now I want to talk about Taxonomies (I will expand on tags in another post). First are some definitions. A Taxonomy is a representation of concepts (called Taxa) that have distinct relationships.

For example a Taxonomy of the communities I track in my PIM would have Work, Church, Family, and Friends. Friends would have a distinct subset of "College Friends" and Family would have related subsets for each family grouping for my immediate family, my wife's family and my ex-wife's family (see graphic above).

A taxonomy encodes the relationships between the concepts. These include:

• is-a-type-of relationships like spaghetti is-a-type-of pasta.
• is-an-instance-of-a relationship like "my spaghetti dinner" is-an-instance-of-a "spaghetti dinner".
• is-part-of relationship like Rome is-a-part-of Italy.
• is-associated-with relationship like Spaghetti is-associated-with Italy.

These relationships could be the key to achieving smooth access to the information you need when you need it. For example, if I had a tree of communities associated with each other such as the picture above then when I went to reply to an email from my brother I could quickly be given the oppurtunity to email it to my immediate family or the next highest group up beyond that. If a PIM supported multiple user defined Taxonomies such as a Community Taxonomy or a Calendar Taxonomy then the user could view each level of the taxonomy using the views that made sense for them.

As an example, a taxonomy could have the following levels and relationships Calendar->Jim, Janice, Boys, Church, Work with additional subgroupings of Church->Board of Trustees, Worship Committee. Any type of object could be associated with any level such as Work but only those objects that have an Aspect of LIT would show up in a calendar view or a schedule view. Only those objects with the aspect of LIS would show up in a map view.

Of course, as a user my preference would be for being able to create and manipulate taxonomies quickly as well as associate searches with Taxa for the ability to create flexible taxonomies.

Tailoring the domain model.

In the previous post I stated that I believe the path to a usable PIM lies in allowing the user to tailor the domain model to match how they work. That is a great ideal and very laudable but the world is afloat with software that is so tailorable that it is unusable. How to manage this so that the user is not overwhelmed, annoyed or just plain pissed of is going to take some thought. So the first thing to look at is at what will not vary.

There are some things that I believe form the foundation of the things to be managed in a PIM:

• Events: Things that happen
• Actions: Thing that are done or performed.
• Notes: Things that are recorded for later.
• Contacts: Info on people.

And, after looking at the model for awhile I believe that there are some "aspects" or "potential behaviors" that can be associated with each object. The aspects that I can see immediately are:]

Can be Located in Time abbreviated LIT
We can place this on some sort of chronology (Calendar, Schedule, etc)

Can occupy time abbreviated OT
We can view its duration. This may be a logical part of the LIT aspect above

Can be Located in Space abbreviated LIS
We can place this on some sort of map

Can be tracked abbreviated TRK
We can track its progress

Can have a lifespan abbreviated SPN
We can see the object become relevant and expire.

Much of what determines what is visible in a given context is the aspects associated with the object. And those aspects can be applied or not as necessary. The Chandler project refers to this as "Stamping". For example an Event may be made "mappable" by stamping it as LIS. The user may choose to leave its location indeterminate but it is the stamping of the object as LIS that makes it mappable, not that it is given a location. With these building blocks the user has the potential to create many specialized types as needed. i.e. a Promise could be an Action that can be tracked and has a limited lifespan. In addition, the user can then display a Calendar which contains all things that are LIT whether they be Notes about an Event, an Event or a Promise.

Of course most users will want to filter things down to what they want. But then, now that we have determined what governs what CAN be displayed we come to looking at how we filter things down narrower then the aspects associated with the objects. And for that we need to look at the thing that PIMs are expected to handle well. And that is metadata; taxonomies, tags and categories. Since that is a substantial topic in and of itself, that will be covered in a seperate post.

"You keep using that word. I do not think it means what you think it means."

In my previous post I talked about how often users adapt to the PIM rather than the reverse. And no matter how intelligent the developers are the end result of their designs often seems to come up with eerily similar results (Chandler and Ecco being a somewhat notable exceptions). Most PIMs seem to end up with some variation on Tasks, a Todo List, one or more calendars, some events and some categories....

But the domain that we are all trying to model doesn't seem to be that clean cut. When I started out modeling the domain I discovered that very little appears to be set in stone. Most clear statements I can make about the domain are of the "may" or "can" type.

Here are some of the objects that I believe need to be modeled (in no particular ordering):

Events, Meetings, Appointments, Promises, Tasks, Birthdays, Anniversaries, People, Organizations, Checklists, Todo lists, Schedules, Calendars, Communities, Projects, Conversations, Goals, Journals, Directions, Notes and references to resources.

And here are some example statements I can make about some of these objects:

• Events may be located in time.
• Events may occupy time.
• Events probably will be located in space (have a location)
• Events may have participants
• Events may have an owner

• Actions may have an owner
• Actions may be located in time.
• Actions may occupy time.
• Actions may have a location.
• Actions may have participants.

For almost every object in the domain there is some ambiguity about what its behaviors and attributes are. The ambiguity seems to come from several areas. Some of it comes from the context you are working in at the time: the "Honey-Do" list at home may be managed and tracked differently than the project todo list at work. Some ambiguity comes from the fuzziness of the words themselves: What is the difference between a Task that is scheduled in time and an Event? And some of the ambiguity comes from the precision the user expects from the word itself: One user may consider Tasks very important objects to be managed in priority order with due dates and other users may expect Tasks to be simple entries in a list that you mark off when you are done.

So it seems inevitable that there will be significant mismatches between the model that a PIM uses and the worldview of the user. I think the key thing I see here is that the PIM's domain model needs to be tailorable to match the user's world view.

Where is flexibility needed ?

People have been very generous with their time and talking about how they manage the details of their life using a PIM or equivalent. I have been astounded with how people have adapted to the PIM of their choice.

And that seems to be the theme. Anyone who is using an electronic PIM seems to have done some significant adaptation to the PIM itself. It may be simply accepting that you can only have 15 categories for all Tasks, Events and so on. It may also involve managing things in multiple applications and sharing data between them.

Some of the issues people have adapted to:

• PIMs that have only one calendar.
• PIMs that require you to share the same set of categories among events, contacts, and tasks.
• PIMs that only allow you to set one category for each contact.
• PIMs that assume tasks can only have a due date. If you want to reserve a time for doing it the user needs to schedule it seperately as an event.
• PIMs that don't allow you to schedule an event until everything is known. When and how long , etc..
• PIMs that make it difficult to move the information in and out.
• PIMs that only allow you to track the members of an organization, not the organization itself.

I could go on but the basic idea seems to be that the limitations appear to be arbitrary and sometimes they appear counter intuitive. Yet it seems unlikely to me that everyone involved in the development of a PIM is making arbitrary decisions for the fun of it. There are some very smart people working in this space. Just look at the teams working on Chandler, the KDE PIM or Evolution.

As far as I can tell the issue seems to be how the PIMs choose to manage complexity. All PIMs atempt to manage complexity for the user. Many of them do so by making assumptions about what a user needs.

The tension is between making managing the complexity that gets presented to the user and making the underpinnings flexible enough to support the whole range of activities.