Cool Tool - Todoist

Back after a long hiatus. The hiatus was not because I wasn't doing lots of interesting things it was because I hadn't figured out how to balance all the interesting things I was doing. Which brings me to the 1st tool that I have blogged about in a long time.

I do a lot each day and I have the memory of a sieve. If I don't record something the minute it occurs to me or I promise to do it the odds are very good that I will not remember it at all. My iPad is my prosthetic memory and everything I have has to sync with my calendars and my task lists so that what I need is accessible everywhere. 

For years now I've been using ToodleDo. I have generally been happy with it. It's biggest strength was that it had tasks, outlines, and notes. And the tasks portion of it could handle tasks within tasks in a limited way.

And since I am a Java programmer the fact that they had a REST API was useful. The API documentation was a pain in the posterior because the documentation, like many REST APIs' documentation, didn't quite come from nothing. But once I figured out how to get programmatic access to my task lists, it made many things relatively smooth.

The ToodleDo web application is solid and I used Pocket Informant to sync up with TooldeDo (including the notes) and that was effective and reliable.

The 2 downsides to ToodleDo that have been gnawing at me:

1. There is only one level of tasks within tasks. So I tended to use the top level tasks as projects and the tasks below that as the actions. But that didn't really cleanly sync up with Pocket Informant.
2. Capturing information or actions or tasks was not fast. It was acceptable but it wasn't fast. And once a single task was entered in the iPad application, I had to go through the same sequence again to enter the next task. So for inserting many tasks I end up using imports via CSV. Very usable from the computer, less so from the iPad.

And things probably would've stayed that way except for one other thing: I lead courses that require generating and tracking many activities over 3 1/2 to 5 months at a time. So I have checklists for pairing and executing the course that I revise based on what worked and didn't work in the last course.

Some of the checklists are required for predictable events that occur in a certain date such as a particular classroom. Some of the checklists are required for repeating events such as when a new coach comes on the team or if someone drops out of the course. 

I've led 12 of these courses since 2008 and this time my mind balked at again hand editing all of the elements I needed to import into ToodleDo so I ended up reevaluating a number of the online task managers I had reviewed years before as well as a bunch of new ones.

Enter Todoist.

Todoist has the concept of projects and within projects you can  have tasks that have up to 4 levels of tasks embedded in them. And, their iPhone and android apps allow for rapid and repeated capture of tasks.

An example: 




The left-hand side entry with the red arrow that says "April 2016 SELP" is a project and within that project is a task called "Eval SELP Tech". And within that on the right-hand side is a task called Spreadsheet with a series of tasks below it dealing with Google App Script or GAS.

In the iPad application, if I want to add a task I need only click the blue '+' at the bottom and start entering. 

The Todoist application has 2 features that make it very easy to enter tasks in sequence. The 1st is that it will enter any new task at the same level that you entered the last task. The 2nd is that if you enter any time reference it will use that as the due date of the task.

For example:




The blue arrow indicates where you are doing the task entry and the red arrow indicates a portion of the UI you can use your finger on to indent the task or move it out. And once you enter the task if you use a term such as tomorrow or next week it will highlight that and use that as the due date for the task as below:



Lastly, both the iPad and web apps have decent performance. I have over 3000 tasks and actions in this one project that I have not seen any performance degradation.

My next few posts will probably deal with using the Todoist REST API, from Java and from within
Google App Script.

What questions do we want answered.

Practices for Deciding what to Test and what to Automate

We test in order to get certain questions answered. We automate tests when we want to get those questions answered with a minimum of human thought. The idea is that we have the human figure out what questions to ask and how to answer them and let the machine do the grunt work.

Choosing what tests to automate is always the result of a cost-benefit analysis. A.k.a. weighing the benefits against the costs. In most projects you can't test and automate everything, so you want tests that answer the questions you want answered the most. We want to automate the tests that will give us the biggest bang for the buck.

I typically document the tests by the questions the tests are designed to answer. A set of tests may be designed to answer whether or not the current build is worth subjecting to further tests. i.e is the current build minimally acceptable. I have some sample questions below.

As you look at these questions below in each case you want to ask yourself to other questions "Do we care?" and  "How much do we care?". Some of these questions are not important to us because of the specific project that we are working on. In almost all cases there is a priority to what questions you want answered first.

Typical Questions

• Do various classes and functions in the code work correctly in isolation. Typically tests that are designed to answer this question fall under the label of unit tests.
• How does a given protocol handle bad state changes. Typically tests that are designed to answer this question referred to as Short Tour Tests.
• Does the SUT (System Under Test) deploy correctly. Typically tests that are designed to answer this fall under the label of smoke tests.
• Does the same code produce the same result when invoked from different platforms.
• Does the current build meet some minimal criteria for being deployed and tested further? Typically tests that are designed to answer this question fall under the label of smoke tests.
• How does the (SUT) perform under specific loads. Tests that answer those questions are typically referred to as performance tests
• Does the SUT past the same test it used to pass? Tests that are designed to answer this question are typically referred to as regression tests.

Once you have decided what questions are important and how important they are, you need to design an automated  testing regimen that answers those questions in that order of priority.

I want to emphasize that the questions to be answered and their priorities are key factors in determining the effectiveness and efficiency of the testing process. If you are creating unit tests, integration tests, and performance tests and they don't answer the questions you need to have answered, you are now maintaining unit tests, integration tests, and performance tests that do not contribute to your business. In other words, you are generating overhead and possibly technical debt.

Once you know the questions you want answered and how important they are to you, it's time to figure out what testing types you will use get those answers and what actions your tests will perform.

One key implication of this is that when you're creating individual tests try to keep them to answering one question at a time. As the system evolves answering that one question may become more complicated or involved but evaluating whether or not the test is well designed and executed can always be evaluated by determining whether or not it allows you to answer that one question.

General rules to keep in mind

Test Early/Test Often

The earlier in the development/testing lifecycle that you get the answer to the question, the less expensive it is to respond to.

Don't buck the tide

Using tools in the way they are designed to be used is far more effective than cramming the tool to fit.

For example: Maven is an automated build tool that is designed around some methodologies that they consider best practices. If you are not taking the actions that Maven expects you to take you can end up spending a great deal of time working on or against Maven rather than working on your tests. By the same token, if you do adhere to what Maven expects, things will be fairly straightforward.

Don't improve the process until you know what you are improving

You can't improve the process unless you have numbers!

You are adding test an existing process it makes sense to test the areas that have the most bugs are the least stable. If you don't have coverage analysis of the code you can't say how much of the code is actually exercised by the testing process. If you don't know what the current performance level of the product is you have no starting point for figuring out where to speed it up.

By the same token, metrics are at best a guideline, if you use a metric for what it's not designed for, it can be dangerous to your sanity.

Context is Decisive

“A best practice is always a best practice; Except when it's not.”

I have recently been involved in a contract that has forced me to really really look at testing practices and how I explain testing practices. 

This particular department of a very large company gives lip service and action to a lot of "Best Practices" in the world of development and testing. SCRUMs, TDD, BDD, etc. are all the words you see floating around in the air there. They are saying all the right things, and even taking many good actions, and the end result is almost worse than if they took no actions at all.

It is, in my world the best illustration of a very basic principle, context is decisive. Another way of saying that is that a best practice is always a best practice, except when it's not.

To illustrate the point: 

In the context of body part the above is a Finger

In the context of number the above is One

Same picture, different interpretation.

This is particularly notable when you start talking about using "Best Practices" out of context.

When I was starting out as a software engineer I worked for an Foliage Software Systems. They had a significantly higher than average success rate with projects and a higher than average satisfaction level of their customers. One of the things that Foliage did that made a difference was that they had a playbook of processes. So they would have a preliminary engagement with the customer where the lead on the project would discover what kind of existing coding and communication processes they already had in place and use the playbook to select the Foliage development and communication processes to fit with what the customer was doing. This came about because they discovered early on that merely producing a technically successful project was not enough to produce a satisfied client. If the development process was not understandable by or compatible with the client they could have an unsatisfied client. The communication process was not understandable by or compatible with the client, they could have an unsatisfied client. And they worked on FAA and FDA certified projects as well as many, less stringent, markets.

By choosing the process to match the context they ended up with a higher number of successful projects.

As far as I can tell the majority of the TDD is Dead discussion comes down to, if you use the TDD methodology out of context, it gets messy and doesn't deliver on what you need. I completely agree. 

I use TDD a lot. It allows me to shake out the design of the individual modules at a low level and alerts me to the impact of re-factoring. I don't test getters and setters. I don't test methods that just delegate to another method. I don't write code for functionality that isn't tested. And I don't forget that the unit test provides me with a minimal contract regarding the usability of the module. For me the context of TDD is to answer some key questions such as "Does this module meet some minimum robustness requirement?", "Do I know how I'm going to use this module?" , "Did I break anything when I re-factored the classes?". So I use TDD to drive the design and implementation of the individual modules and leave me with a test that documents some ways to use the module, specifies the core functionality of the module, and acts as a tripwire for detecting the impact of re-factoring. In that context, TDD is quite successful.

When I was doing research I came across a website describing the school of thought called Context Driven Testing. You may or may not agree with all of the discussions on the webpage. But the heart of it in the basic principles is one of the clearest statements around best practices that I've ever see.

The Basic Principles of Context-Driven Testing

1. The value of any practice depends on its context.
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project’s context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products

In the next entry I will discuss  what questions to ask to determine what to test, and what to automate.

Blogs to watch for useful TestNG info

As I come across the I will update this post with blogs that have some useful info on working with TestNG

http://rationaleemotions.wordpress.com

TestNG : Using Guice for dependency injection

In the previous post I discussed how to use annotations to make it possible for TestNG to rename tests.

The next few posts are going to deal with some useful ways to use dependency injection and TestNG to radically simplify integration testing.

Recently I have been working on a contract that requires a lot of integration testing. And the great challenge of integration testing is that tests can fail because the you are testing against can change (as well as the code). My current client has a framework of VMs that can be deployed in many different configurations with different elements being simulated and some being real.

In addition, a deployed node may have one purpose but, depending on the customer being served the node may use a different protocol. Plus we have some environments where a node may be a single node in one deployment but a cluster in another.  Most of this information is available in the database associated with the deployment.

It is my belief that you never need more than two pieces of information to run an integration test. The first is the name of the test set to be run, the second is the name or address of some machine to act as a point source of information (database server or some configuration server such as Zookeeper).

My goals in this current project are to:

1. Minimize the amount of information the person writing the test needs to know in order to write the tests. If a session needs to be created, they shouldn't need to know what the underlying protocols are.
2. Replace the many diverse configurations files (YAML, Properties and XML) with an IP address and the name of a test set to run. 
3. Allow the test writers to write tests without regard for whether or not they are pointing at a cluster of nodes or a single node in the deployed environment.

For the first the obvious pattern is to hide the implementations behind interfaces.

For the second the configuration information should be made available to the implementations without passing through the test writers hands. 

This all argues for Dependency Injection. 

Since we are using TestNG as our test framework, it is natural to use Google Guice as our DI framework. TestNG has an annotation that allows us to specify the DI binding factory to use when the test starts up.

For this example I'm going to use a simple interface for persistent storage called Repo. Repos are designed to store CoolObjs. Note the use of the Google Guava Optional class. This simply gives you a typesafe way of handling when an object is not returned.

Repo.java

package org.saltations.testng.usingguice;

import com.google.common.base.Optional;

/**
 * Repository Interface. Represents a repository that objects can be saved or
 * recovered from...
 */

public interface Repo {

 /** 
  * Store an object 
  */
 void store(CoolObj obj);

 /**
  * Retrieves an existing object
  *  
  * @param id The id of the {@link CoolObj}
  * 
  * @return An instantiated cool object.
  */
 Optional<CoolObj> retrieve(String id);
}

CoolObj.java 

Note the use of Lombok annotations to simplify POJO development. I mentioned Lombok in a previous post. You can generate getters/setters and constructors by hand.

package org.saltations.testng.usingguice;

import lombok.AllArgsConstructor;
import lombok.Data;

@Data
@AllArgsConstructor
public class CoolObj {

 private String id;
 private String stuff;

}

When testing we are going to use three different implementations of the Repo interface. The first one is the RepoMockImpl use for unit testing. It uses an internal map in memory to store and retrieve the objects. The second one is the LocalRepoImpl that is used for integration testing. It stores all of the objects using a key-value store on disk. The third implementation is the ServerRepoImpl that points to a remote repository server.

The details of each of the implementations of the repository are not important. The key thing is they require different types of configuration information. The ServerRepoImpl requires the IP address/host name  of the machine being pointed to.

The test that uses this looks like:

TestUsingInjectedServices.java

package org.saltations.testng.usingguice;

import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertTrue;

import org.testng.annotations.Guice;
import org.testng.annotations.Test;

import com.google.common.base.Optional;
import com.google.inject.Inject;

@Test
@Guice(moduleFactory=TestDIFactory.class)
public class TestUsingInjectedServices {

 @Inject
 private Repo repo;
 
 public void shouldSaveAndRetrieveACoolObj()
 {
  CoolObj coolObj= new CoolObj("goodObject", "Cool Tidbits");
  
  repo.store(coolObj);
  
  Optional potentialObj = repo.retrieve(coolObj.getId());
  
  assertTrue(potentialObj.isPresent());
  assertEquals(potentialObj.get(), coolObj);
 }
}

The elements that hook everything together are the

1. Guice annotation which tells the TestNG framework to use the TestDIFactory class as the factory used to supply Guice with the Modules that in turn provide the bindings between the Repo service and its implementation.
2. Inject annotation that Guice to inject the appropriately configured service implementation for the Repo interface in the field repo.
3. The groups attribute in the TestNG.xml file that is used by the TestDIFactory to determine which Module implementation is used to supply the bindings.

If this test were run with the group configured as "unit" we would get the unit test bindings. If we run this test with the group configured as "integration" we would get the integration test bindings.

Configuration information comes in through the TestNG.xml file as TestNG parameters and as TestNG groups .

The Guice factory is in a form specified by TestNG.

TestDIFactory.java

package org.saltations.testng.usingguice;

import static java.text.MessageFormat.format;

import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.List;

import lombok.NoArgsConstructor;

import org.testng.IModuleFactory;
import org.testng.ITestContext;

import com.google.common.collect.Lists;
import com.google.inject.Binder;
import com.google.inject.Module;

/**
 * Guice Factory for TestNG Tests. This factory gives back a Guice module that
 * supplies the bindings that are appropriate for the kind of tests being done.
 * i.e. the factory will give back a module that supplies unit test service
 * implementations for unit tests, integration test service implementations for
 * integration tests, etc...
 * 
 * @author jmochel
 */

@NoArgsConstructor
public class TestDIFactory implements IModuleFactory {

 /**
  * Key for the test parameter in TestNG.xml that contains the name of our repo server.
  */

 private static final String REPO_SERVER_NAME = "repo-server-name";

 /**
  * Module that provides unit test service implementations for unit tests
  */

   public class UnitTestServicesProvider extends GeneralModule implements
   Module {

  public UnitTestServicesProvider(ITestContext ctx, Class clazz) {
   super(ctx, clazz);
  }

  public void configure(Binder binder) {
   binder.bind(Repo.class).to(RepoMockImpl.class);
  }
 }

 /**
  * Module that supplies integration test service implementations for
  * integration tests
  */

 private class IntegrationServicesProvider extends GeneralModule implements
   Module {

  public IntegrationServicesProvider(ITestContext ctx, Class clazz) {
   super(ctx, clazz);
  }

  public void configure(Binder binder) {
   binder.bind(Repo.class).to(LocalRepoImpl.class);
  }
 }

 /**
  * Module that supplies real word service implementations for
  * testing the application against a real world system.
  */

 private class WorkingServicesProvider extends GeneralModule implements
   Module {

  /**
   * Repo Server address.
   */

  private InetAddress address;

  public WorkingServicesProvider(ITestContext ctx, Class clazz) {
   super(ctx, clazz);

   /*
    * Confirm that the repository server address exists and that it points somewhere real. 
    */

   String repoServerName = ctx.getCurrentXmlTest().getAllParameters().get(REPO_SERVER_NAME);

   if (repoServerName == null || repoServerName.isEmpty() )
   {
    throw new IllegalArgumentException(format("Unable to find {0} in the test parameters. We expected to find it configured in the TestNG.xml parameters.", REPO_SERVER_NAME));
   }

   try {
    address = InetAddress.getByName(repoServerName);
   } catch (UnknownHostException e) {
    throw new IllegalArgumentException(format("Unable to find host {1} specified by parameter{0} in the test parameters.", REPO_SERVER_NAME, repoServerName));
   } 
  }

  public void configure(Binder binder) {
   binder.bind(Repo.class).toInstance(new ServerRepoImpl(address));
  }
 }

 /*
  * @see org.testng.IModuleFactory#createModule(org.testng.ITestContext,
  * java.lang.Class)
  */

 public Module createModule(ITestContext ctx, Class clazz) {

  /*
   * Get a list of included groups (comes from the TestNG.xml) and choose
   * which Guice module to return based on the types of tests being done.
   */

  List groups = Lists.newArrayList(ctx.getIncludedGroups());

  Module module = null;

  if (groups.contains("unit")) {
   module = new UnitTestServicesProvider(ctx, clazz);
  } else if (groups.contains("integration")) {
   module = new IntegrationServicesProvider(ctx, clazz);
  } else {
   module = new WorkingServicesProvider(ctx, clazz);
  }

  return module;
 }
}

For completeness: GeneralModule.java

package org.saltations.testng.usingguice;

import static com.google.common.base.Preconditions.checkNotNull;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NonNull;

import org.testng.ITestContext;

import com.google.inject.Binder;
import com.google.inject.Module;

/**
 * Parent class for Guice Modules used in TestNG. Contains the ITestContext and
 * Test Class because I so often need the tests to make decisions based on the
 * contents of the TesNG.xml (and made available in the ITestContext current XML
 * parameters).
 */

@Data
@AllArgsConstructor
public abstract class GeneralModule implements Module {

 /**
  * Context for the test
  */

 @NonNull
 private ITestContext ctx;

 /**
  * Class to be tested
  */
 
 @NonNull
 private Class clazz;

 public abstract void configure(Binder binder);
}

What Happens

The tests are started up using the TestNG.XML file. When the TestNG framework instantiates the test it does so after using the TestDIFactory createModule call to determine which Guice Module it will use to configure Guice. In the createModule call we look at the ITestContext and determine what group this test class should be run has. That module is then instantiated and handed back to Guice. Guice then walks through the instantiated classes and when it encounters a @Inject annotation it asks the module for what class or object should be injected into that field. The person writing the tests does not need to know what the Repo service is pointing to.

One of the key gotchas here is that the TestDIFactory and Modules are instantiated on a per test class basis. If we are running five test classes in a unit test group all five classes will have a separate instantiation process. If you wish for a service to be shared as a singleton then you will want to use a static element in the TestDIFactory or the individual module and then bind the interface to the implementation using the "toInstance" method rather than the "to" method.

What if Repo involved a Generic?

If Repo was actually Repo<ObjectType> the the code would look like:

binder.bind(new TypeLiteral<Repo<ObjectClass>>(){}).to(new TypeLiteral<RepoImpl<ObjectClass>>(){});

Pros and Cons

The pros are pretty clear to me. By using this I am able to hide all of the configuration details from the test writers. It is a little bit more work up front but it pays off handsomely in the simplicity of the testing code. From my standpoint one of the big pluses is that it allows me to verify all the configurations upfront and instantiate the classes with the configuration embedded in them situated that configuration can be used for the test services and (as we see in a later post) for configuring the tests themselves.

TestNG: Dynamically Naming Tests from data provider parameters

I have recently been involved in several contracts that require me to stretch my knowledge of TestNG. There a lot of cool things you can do with TestNG that are available but not explicitly documented  (you know, with real source code !).

One of the first examples is that of dynamically naming a test based on incoming data.

If you're using a data provider in TestNG it allows you to define a method that will feed the data into a test method. For example this simple data provider gives three sets of data that a test can be run with:

@DataProvider(name="rawDP")
public Object[][] sampleDataProvider()
{
 Object[][] rawData = {
  
   {"SCENARIO_1","First Test Scenario"},
   {"SCENARIO_2","Second Test Scenario"},
   {"SCENARIO_3","Third Test Scenario"}
 };
 
 return rawData;
}


@Test(dataProvider="rawDP")
public void shouldHaveTestNamesBasedOnMethodName(String arg1, String arg2)
{
}

The above method will be passed all three sets of data in sequence with the first argument going into the first string of the method in the second argument going into the second string of the method. When you run the test in eclipse the console output will show

PASSED: shouldHaveTestNamesBasedOnMethodName("SCENARIO_1", "First Test Scenario")
PASSED: shouldHaveTestNamesBasedOnMethodName("SCENARIO_2", "Second Test Scenario")
PASSED: shouldHaveTestNamesBasedOnMethodName("SCENARIO_3", "Third Test Scenario")

But you may want the scenario name to show up in the HTML report or  in some other way.

Through the magic of some TestNG interfaces we can make it so that those arguments are intercepted and used to name the instance of the test before the test method is run.

To do so we define a custom annotation UseAsTestName that is made available at run time.
The annotation has an attribute that indicates which parameter of the parameter set should be used as the test name.

/**
 * Annotation used as an indicator that the Test Method should use the indexed
 * parameter as the test instance name
 *
 * @author jmochel
 */

@Retention(RetentionPolicy.RUNTIME)
public @interface UseAsTestName {

 /**
  * Index of the parameter to use as the Test Case ID.
  */

 int idx() default 0;

}

Then we have code in the test's parent class that runs before each method is run (as indicated by the @BeforeMethod annotation).

public class UseAsTestName_TestBase implements ITest {
 
 /**
  * Name of the current test. Used to implement {@link ITest#getTestName()}
  */

 private String testInstanceName = "";

 /**
  * Allows us to set the current test name internally to this class so that
  * the TestNG framework can use the {@link ITest} implementation for naming
  * tests.
  *
  * @param testName
  */

 private void setTestName(String anInstanceName) {
  testInstanceName = anInstanceName;
 }

 /**
  * See {@link ITest#getTestName()}
  */

 public String getTestName() {
  return testInstanceName;
 }

 /**
  * Method to transform the name of tests when they are called with the
  * testname as one of the parameters. Only takes effect if method has
  * {@link UseAsTestName} annotation on it..
  *
  * @param method
  *            The method being called.
  *
  * @param parameterBlob
  *            The set of test data being passed to that method.
  */

 @BeforeMethod(alwaysRun = true)
 public void extractTestNameFromParameters(Method method, Object[] parameters) {

  /*
   * Verify Parameters
   */

  checkNotNull(method);
  checkNotNull(parameters);

  /*
   * Empty out the name from the previous test
   */

  setTestName(method.getName());

  /*
   * If there is a UseAsTestCaseID annotation on the method, use it to get
   * a new test name
   */

  UseAsTestName useAsTestName = method
    .getAnnotation(UseAsTestName.class);

  if (useAsTestName != null) {
   
   /*
    * Check that the index it uses is viable.
    */

   if (useAsTestName.idx() > parameters.length - 1) {
    throw new IllegalArgumentException(
      format("We have been asked to use an incorrect parameter as a Test Case ID. The {0} annotation on method {1} is asking us to use the parameter at index {2} in the array and there are only {3} parameters in the array.",
        UseAsTestName.class.getSimpleName(),
        method.getName(), useAsTestName.idx(),
        parameters.length));
   }

   /*
    * Is the parameter it points to assignable as a string.
    */

   Object parmAsObj = parameters[useAsTestName.idx()];

   if (!String.class.isAssignableFrom(parmAsObj.getClass())) {
    throw new IllegalArgumentException(
      format("We have been asked to use a parameter of an incorrect type as a Test Case Name. The {0} annotation on method {1} is asking us to use the parameter at index {2} in the array that parameter is not usable as a string. It is of type {3}",
        UseAsTestName.class.getSimpleName(),
        method.getName(), useAsTestName.idx(),
        parmAsObj.getClass().getSimpleName()));
   }

   /*
    * Get the parameter at the specified index and use it.
    */

   String testCaseId = (String) parameters[useAsTestName.idx()];

   setTestName(testCaseId);
  }
 }

}

Because we have a Method type in the methods parameter list, TestNG automatically inserts the method object pertaining to the method being called. The same for the Object[] which TestNG automatically inserts the row of data associated with this invocation of the test method.

@BeforeMethod(alwaysRun = true)
public void extractTestNameFromParameters(Method method, Object[] parameters) {

In the before method we clear out the old test name and use the information provided by the method parameter and the parameters parameter to create a new test name and set that.  When the XML and eclipse Console reports are being generated by the tests they use the ITest getTestName() method to get the name of the test.

The tests that use this look like:

public class UseAsTestNameTest extends TestBase {
 
 @DataProvider(name="rawDP")
 public Object[][] sampleDataProvider()
 {
  Object[][] rawData = {
    {"SCENARIO_1","First Test Scenario"}, 
    {"SCENARIO_2","Second Test Scenario"},
    {"SCENARIO_3","Third Test Scenario"}
  };
  
  return rawData;
 }
 
 
 @Test(dataProvider="rawDP")
 public void shouldHaveTestNamesBasedOnMethodName(String arg1, String arg2)
 {
 }
 
 @UseAsTestName()
 @Test(dataProvider="rawDP")
 public void shouldHaveTestNamesStartingWithANA(String arg1, String arg2)
 {
  getTestName().equals(arg1);
 }
 
 @UseAsTestName(idx=1)
 @Test(dataProvider="rawDP")
 public void shouldHaveTestNamesStartingWithThe(String arg1, String arg2)
 {
  getTestName().equals(arg2);
 } 
}

The output looks something like:

PASSED: shouldHaveTestNamesBasedOnMethodName("SCENARIO_1", "First Test Scenario")
PASSED: shouldHaveTestNamesBasedOnMethodName("SCENARIO_2", "Second Test Scenario")
PASSED: shouldHaveTestNamesBasedOnMethodName("SCENARIO_3", "Third Test Scenario")
PASSED: SCENARIO_1("SCENARIO_1", "First Test Scenario")
PASSED: SCENARIO_2("SCENARIO_2", "Second Test Scenario")
PASSED: SCENARIO_3("SCENARIO_3", "Third Test Scenario")
PASSED: First Test Scenario("SCENARIO_1", "First Test Scenario")
PASSED: Second Test Scenario("SCENARIO_2", "Second Test Scenario")
PASSED: Third Test Scenario("SCENARIO_3", "Third Test Scenario")

The one gotcha in all of this is that the reports that are generated for the results are generated from the XML that TestNG generates.

The XML uses the name we generate and puts it in what's called the "instance name" attribute. There are many different HTML versions of the reports and some of them correctly use the instance name and some of them don't. The HTML reports run on Jenkins correctly use the instance names and show the tests with that as their name. The default HTML reports that get generated will only run it from eclipse don't use the instance name correctly. The eclipse console does correctly use the instance name thus we can see it there.
Some HTML reports will show it in some will not.

Getting things out of your head

Most task management methodologies share some way to deal with getting information out of someone's head and into a persistent form as quickly as possible without evaluating it. Some methodologies use some notebook or tool that you carry around everywhere and you just write things down in them, some others simply have you write things on a whole sheet of paper and throw it into it inbox. These actions are intended to relieve you of the mental burden of keeping track of stuff. When you write something down and you know you can find it again later when you need it, it stops filling the space in the back of your brain. A great deal of research has shown that the more thought that has to go into categorizing or figuring out where something has to go at the moment when you need to capture it, the more likely it is that it will be lost (due to interruptions, indecision, etc.)

Unfortunately, most PIM's require you to know exactly what you going to do with the data before you ever capture it. And when you capture it, if you need to change it into something else, it can be a real pain that requires cut-and-paste.

The scenarios below deal with capturing information as well as evaluating and transforming it later.

Scenario: A parent is assuming out the door when their son reminds them that he needs to be at the newly scheduled soccer practice on Saturday. When the parent jumps in the car they open the PIM application on their cell phone and simply speak "Trent has a soccer practice next Saturday at 4 PM". The PIM generates a note and automatically puts it into a list to be triaged.

That evening when the parent opens the PIM on their desktop or tablet it notes that there are several items to be triaged. When the parent looks at the note they use a finger gesture or mouse gesture to transform it to an event. The event is generated with the subject "Trent has a soccer practice", and a date of next Saturday at a time of 4 PM.

Scenario: An entrepreneur is on the road and uses a digital voice recorder to record a bunch of brilliant ideas that they absolutely don't want to forget. When they come to rest they open up their laptop, open the PIM and import the various sound files from the voice recorder. A note gets created for each sound
file with the sound file attached. They automatically go into the triage queue and the entrepreneur can use voice recognition to extract the text or simply use the attached voice file as is.

Scenario: A vice president of a small company wants her engineering team to stay on top of the latest development technologies and subscribes to the newsfeeds of several training and development companies. She has set things up so that whenever something new is available it automatically gets put into her morning to do list as a task that says "Evaluate: New and Cool Course" where "New and Cool Course" was the subject of the newsfeed entry.

The VP evaluates a bunch of new courses and decides that 3 of them look interesting and then uses a mouse gesture or a click to transform all three of those into "Shared Votes" that are visible to the engineering team. The engineering team members each give a thumbs up or thumbs down to the particular course and at the end of the "New and Cool Course" has three thumbs up.

The VP then uses a mouse gesture or a click to turn the "Shared Vote" into an task "Schedule New and Cool Course for the engineering team".

Scenario: A security consultant has been asked to step in and have a conversation with a general consultant's client about their security needs. He enters this into the PIM as a note "Call so-and-so about security for whats-his- name". That night he tells the PIM that this note is now both an event (which is
not yet scheduled) AND a promise. These are not two separate entities. The single item is both an event and a promise. The event does not yet have a hard and fast date so it needs to be scheduled. And when it is complete, and the consultant notes that is complete, and email goes out to the general consultant
to let them know that that's complete.

And the security consultant can even share the event with the general consultant so that the general consultant can see when the event has actually been scheduled.

One size does not fit all

Here are some scenarios pertaining to personalizing your PIM.

Scenario: A divorced parent is creating an event in their PIM for one of their kids birthdays. When they create the event they include associated actions and events such as an email to be sent out two months ahead to check which parent is going to be throwing the party, a reminder a month ahead to make sure that both parents have the kid's "birthday list", A time to be scheduled a week or two ahead to do the shopping for the gift, and a time and checklist for the kid to write thank you notes afterwards.

After they have created that event, they can simply tell the PIM that this is a new type of event called "My Kid's Birthday" and the next time they schedule an event with that type it will include all of those associated actions and events and reminders.

Some of the follow-up videos include entering the "Birthday List" and automatically sharing it with the other family members and parents of kids who have been invited to the event. It is then visible as a checklist (somewhat like a wedding registry) that others can view and click off on.

Scenario: Someone using the PIM for the first time who is already an avid user of some methodology such as GTD (Getting Things Done) or Mission Control or Steve Covey... simply picks their choice of  methodology and their PIM is populated with the typical types used by that methodology. So a user of GTD would have types that included "Ticklers" and monthly reviews. A user of Mission Control would have the "Not Doing Now" and "Never Doing Now" lists.

And the user would still be able to add types of actions and events that are either associated with or derived from the types in the methodology.

Scenario: A teacher using the PIM had previously scheduled and taught a new course. The teacher had set up the classrooms in such a way that they included actions that automatically sent out the homework for that classroom the next day as well as a homework reminder to the students two days before each classroom reminding them that the homework was due as well as including the homework for that classroom.

The school asks the teacher to teach that course again and the teacher simply goes back into the PIM, selects all of the events from the previous course and then tells the PIM that the entire course is the new type "Really Cool Course". When the course gets scheduled the teacher goes into the PIM and  schedules a new course using the type "Really Cool Course" and substituting in the new dates. All of the actions and reminders and checklists and agendas from the previous course are included.

Scenario: A consultant makes a promise to a client to let them know when they finished some research and to set up a new time to talk when that is complete. The consultant already has a type "Research promise and follow-up" in their PIM. The consultant creates a new promise from that type and types in the subject of the research and selects the client to be notified. The consultant clicks on the promise and notes that it's complete. The PIM generates an email notifying the client that the task is complete, puts it into the email queue to be edited and then sent out. The email contains a link to the consultants favorite scheduling system (Timetrade, Doodle, etc..) that allows the client to schedule their follow-up.

In order to make these scenarios a reality the PIM needs to support

1. The ability for users to create their own types.
2. The ability for users to create types from existing events, tasks, and actions.
3. The ability for types to include information that must be entered when a new instance of that type is created (such as the dates in the teacher's course, or the client used in the consultants promise)

I am not hallucinating but...

I have, in the past, had problems with company vision statements. I could never see how any
of the actions of most of the companies that I was dealing with were connected to the vision statement.

When I think about a vision I think of it as someone crawling their way through the desert and hallucinating about the oasis. And the oasis is truly beautiful; it shimmers. It is like a vision of the promised land (Cue heavenly choir).

As this vision starts to take shape is mostly coming to me as pictures of what I would like to see in the world. Somewhat like visual "User Stories". In some cases they are simple images and in other cases they are mental videos.

But all of them are about what it would look like for a user to manage their life with tools that are built to fit their life.

In the next few posts I'm going to be describing the scenarios that make up my vision (this is the moment when I really regret a lack of artistic talent) and what I think is required to make the vision a reality.

The scenarios of these visions come from real life needs. Some come from my real life as a divorced parent who has created a really effective collaboration with my ex-spouse so that we are in a partnership to raise our children that includes our current spouses and involves being aware of and working with all of the parents and kids schedules. Some come from my life as someone who devotes a great deal of time to coaching people and working with other people schedules. Some come from the people I have talked to over the years regarding their challenges in managing a busy distributed life while maintaining connections with people.

The one thing that all of these visions have in common is that the model that they are based on is much richer in connections, data types, and metadata than the typical models we see in use in most existing PIMs. To make this happen, one of the requirements will be that this data model and servers that support this data model will have to be ubiquitous. I have prototyped and experimented and I
have confirmed that I can go back and forth between a rich and hierarchical model of events to something that can be represented in an iCal format but the rich nature of the hierarchical model would be lost.

So the first underpinning of this vision is that there is a rich model that is ubiquitous.

And so it begins...

In the last 14 days I've had over 60 conversations with people about what it will take to kick this project off and have it be successful.

I have talked to people about what they want out of a PIM, who they know that might be an expert on something, who they know that might be willing to contribute money to making this happen, and what would be the market for it.

And a path forward has started to emerge.

Along the way I have discovered that I have a lot of opinions that could get in the way of what I want to create. When I started this I was thinking purely in terms of producing an open source project and finding a way to fund it so that I could produce something that will shift the way people deal with their communities, their schedules and to do lists.

What I have discovered is that for many people an open source project alone is not going to provide what they want and need. With everyone I talked to, many of whom are open source boosters, I kept bumping into people's interest in the commercial availability of these tools.

A lot of startup owners, consultants and small business owners are definitely interested in having SaaS providers of these services so that they don't have to install and manage the software. Everywhere I went I kept bumping into that if I want to make the kind of difference I am looking at I will be dealing with people that want to deal with businesses that provide services.

I find myself resisting the very feedback I had gone looking for. That's like doing a marketing survey and then ignoring the results. I did not want to get involved in any activity that looks like a business. I really had to confront that my opinions about businesses and business owners were getting in the way.

All of my opinions boil down to: When you start selling something you start going down the slippery slope towards lying and losing what little integrity you might have. And of course, I have lots of evidence to support this opinion. I have been in many startups and other companies dealing with sales and marketing people who have "Sold the Sizzle" when the functionality hasn't been even put in
the project plan. And I have dealt with executives in marketing or sales who have outright lied and directed their salespeople to lie. And I know many software engineers who can say the same. And when I have encountered single salespeople or even whole sales organizations that have integrity I have discounted them as exceptions rather than the rule.

I have a commitment to creating tools that make a difference in how people deal with life. In order to do that I can see that I am going to have to constantly examine my opinions and where they get in the way of that commitment.

Taking on something radical

Declaring something.

For years now I've been (playing with the design of, thinking about, contemplating, prototyping) a personal information manager and now I am taking it on.

Whats happening right now

Currently, PIMs have hit a "semi-sweet" spot. Good enough to do the job but not "rock your world" great. They are capable and with some poking, effort, technical savvy, and working within their limits you can manage your life. 
Many people find a rough, workable, set of tools and settle on it because no cleaner path presents itself. I know a very highly effective executive coach who constantly deals with breakdowns in synchronization between her different tools. And she is not alone. Everyone from small business owners to teachers to parents and students working in a more complex, fast moving world with more and varied commitments and opportunities for collaboration wrestle with the inevitable friction of getting the tools to do what they need them to do. The busier the person the more the friction is felt. 

As more and more things need to be managed with increasing speed and accuracy the friction becomes more apparent until the user feels like they are swimming upstream. 

The right PIM is not "The Answer". But as people figure out ways to manage their live's powerfully the right PIM will empower them in using those methods. They will make the difference between feeling like you are swimming upstream or swimming with the current.

So for the people who want to manage their contacts and relationships with people, who have multiple accountabilities and calendars to juggle, and who need to track lots of tasks and the state of conversations, the less they have to deal with the tool the freer they are to do what they intend to do. Another consultant I know summed it up quite effectively: "How successful I am in life and business is a function of how many conversations I can effectively manage."

What the future predictably holds.

There is a lot of inertia in this area. Getting to the point that we can reliably exchange calendar events and contact information has taken awhile and most people have gotten used to tools that don't do everything they want. There is some friction but it is not so uncomfortable that most of us feel it strongly enough or often enough to do more than get wistful about something better.

The "good enough, not too painful" place the users are in doesn't call for powerful innovation.

The commercial products are limited by existing standards. Some of the key standards in the area of calendaring and exchanging contacts and tasks date from the 90s and interoperability has been poor. Calconnect, the Calendaring and Scheduling Consortium, was formed in 2004 and one of its major victories was clarifying the iCalendar standard and making it possible to test how well various products allow us to exchange meeting or event information. Some of the biggest players (Google, Microsoft, Apple) in the scheduling space really have no need to do more than provide some fairly basic, fairly usable exchange of calendar information via iCalendar. And there doesn't appear to be a lot of urgent and visible market needs that typically drive innovation.


So predictably we will see a series of incremental improvements with no radical changes over the short haul.

But the market is changing quietly but rapidly. The world has shifted to a vary loose and collaborative footing. Personal communications, projects and collaboration are now routinely spanning countries, time zones, and languages. Sharing calendars, events and tasks become important collaborative processes even in families. The use of online services to find,  locate, purchase retrieve and use products and services that weren't available 10 years ago. School age kids have daytimers to help them manage their schedules. Schools have online homework and grade reporting. Hair salons, banks and retail outlets allow you to schedule appoinments online. Online schools allow you to take course and do homework online and schedule calls with advisors you have never met. Some businesses are composed of coworkers working at home collaborating with people they have never met.


Our existing PIMs and the standards they rest upon are insufficient to support us in that type of world and now is the moment for something new to come into being.

What is possible

Capturing things quickly. 

I envision a PIM that allows you to capture information quickly. You start typing and it creates a note. You can triage the note later or turn it into a task or event immediately. The intention is that the you be able to get the information out of your head and into the system for dealing at your leisure.

Personalizing life 

One size does not fit all.  How we take on life colors how we describe the things in our life and how we track those things. Some of us have specific actions and events that others don't. I have "Calls", "Coaching Calls", "Sales Calls", "Calls to generate leads", etc.. and each has its own agenda or checklist. Some people have "Promises" rather than just tasks. We all have specialized events like "Family Reunions", "Training Weekends" , "Tax Audits", "Employee Reviews", etc..
The incident "Kickoff call with Carl" may be both an event with a start time and date and duration, as well as an task to be completed as part of another task or project. A given type of meeting may have an agenda, a separate checklist, and action items generated by the meeting. And each of those actions may or may not be themselves scheduled in time. When you are looking at the meeting, you want to be able to get access to the agendas and checklists easily.A birthday party usually includes actions like buying birthday cards, buying a gift, etc. And things change over time, a meeting or appointment may become a conference call or an email exchange.

Life has depth.  

When you schedule a class and it has 16 sessions, if you cancel the class you should be able to cancel the class as a whole, not each session individually. Each course may have classes which contain multiple sections each with their own agendas and checklists. Tasks often include multiple tasks which may include other tasks. Communities generally have a  hierarchical relationship. An organization such as a company may have specific departments within it and the departments themselves may have other included groupings. There may be more than one set of such hierarchies. The corporation may have a hierarchy for departments, a hierarchy for roles within all departments, and a hierarchy for current projects that spans across the departments.

Life is fuzzy.  

You may say to someone, I will do it next week but I don't yet know exactly when. Putting it in the calendar at an exact time is misleading. Not putting it into the calendar until you know when could lead to someting else taking up the needed time. Some things are fuzzy in people's heads but need to be remembered until they can be clarified. The PIM should allow you to declare the fuzziness so that you are clear on what needs to be clarified.

Life is repetitive.  

Many of the most complex actions we take on in life are repetitive. I lead courses which have the same structure (the same number of sessions, many of the same checklist to be used each time, etc.). To schedule the course typically requires re-creating the actions by hand or reimporting the data with modified dates. Ideally, sets of actions, and/or events should be something that you could create a template for and then use whatever you are going to schedule those actions or tasks. Ideally you would be able to simply plunk down something like the course starts at this date, put all the standard actions, preparation tasks and events into existence and notify me of any conflicts.

Life involves collaboration. 

Most activities in life involve collaboration. You may schedule a call and want an email or SMS reminder to go out to them 15 minutes ahead of time. You may make a promise to complete an action and report it back to the person you made the promise to. The PIM could email or text that person when you completed that action. You may make a request via email and want to track who replied and who said yes and who said no. You may also wnat to share certain events or actions with others.

Memories are not infinite. 

If you are a teacher you may want to remember key details of your students or your interactions with them years later. If you are a consultant you may want to remember details about the client that you may not want in your CRM system but make it easier for you to have a conversation with them. If you are coaching someone you may wnat to track the ongoing promises they make to you so you can check with them at the next call.

What you see is what you need.  

People can only keep (GROK) a limited number (5-7) of items ( or groupings of items) in their brain at a time. How well the PIM presents the data governs the usefullness and ease of use. The user must be able to tailor their display to fit how they work.

Context is decisive.  

Having the application be sensitive to whether you are home and online or home and off-line or at work and online etc. is key to determining what actions can be performed a given situation. So eventually the application will be able to use the fact that you are in your home office to determine that you have resources such as phone, computer, Internet, etc. available to you.                 

What is next

In the next few weeks I will be blogging about my vision, my actions in creating the funding to take this on as well and technical topics centered around it.

Changing over to two factor hardware authentication.

Over the past two years I have become very concerned about security.

First of all a disclaimer. I know just enough about security to be worried. I am not a security expert but I do understand enough to understand some Bruce Schneier articles which is probably enough to qualify me to play a security expert on TV.

I was one of the people whose Dropbox accounts was hacked and attempts were made on several of my Gmail accounts as well as my bank account immediately afterwards. Luckily I do not use even vaguely similar passwords on the different systems.
 
In addition, I am seeing a huge increase in attempts on my home system through the firewall. My personal development server at home is accessible only through the SSH port and that port has shown a 173% increase in attempts in the last year alone.
 
On that server are several hundred gigabytes of scans of various documents and photographs on the family document management system. Of course all of this is backed up to the cloud but I would rather deal with disaster prevention than disaster recovery.
 
At work I am dealing with how to manage a large number (on the close order of 100) of uername/password/SSH Key/PGP combinations for various servers (development, testing, and production) in a way that allows me 24/7 access in case of emergencies without compromising security.

When it comes to security I have become as nervous as one of our forebears on the African savanna hearing a rustle in the tall grasses. I know that it may just be the breeze and it may mean nothing, but our ancestorss have already demonstrated the value of panicking early and often. The ones that did lived to contribute to the gene pool. In this area, false positives are a survival feature.
 
As a result I have settled upon setting up two factor authentication based on something known (a password) and something possessed (a hardware key).

Any convenient hardware key alone will typically be subject to man in the middle attacks so to make that more difficult I was looking for something time-based like a RSA SecurId or something that generated a one-time password or OTP.

LastPass 

The first part of my solution is LastPass. LastPass is a online service that stores user names and passwords (along with other secure data pieces of data). The data is stored in their servers as a single encrypted block and that block of data is kept in sync as a block with your local system. In other words it is never encrypted or decrypted except on your local machine. Your passwords don't pass over the wire. They don't have the password, you do.  All encryption and decryption is done locally (usually via a browser plugin) or a mobile application.
 
This is allowed me to change over to using automatically generated passwords that are very large and impossible for me to remember. By using LastPass I have drastically decreased the cognitive burden of keeping track of passwords. Though, of course I still have to enter them into LastPass but the browser plug-in makes that very doable. In addition, I don't have access to them when the Internet  is down.  Of course, if it is, most of the systems I need to connect with are inaccessible anyway.

On the plus side it has mobile apps for both the Android and iPad, and the cost for the premium package that is $12 a year.
  
The key downside to this is that it puts a premium on protecting the single password for LastPass. And it is still only one factor authentication (i.e. you only need the password).

Enter the Yubikey. 

It is sold by Yubico  and is a hardware key that provides two factor authentication. It is significantly slimmer (2mm) than a standard USB thumb drive and acts as a USB keyboard. Is used by inserting the key into a USB port and then pressing a button on the key that causes it to generate a one-time password (OTP)  that can then be validated against the Yubico security servers.
 
The standard keys are $25.00 apiece and can be purchased in larger numbers such as 10 for $200 or 50 for $750.
 
There are more advanced keys that do cooler things. More about the various "coolnesses" in a moment.
 
LastPass has built-in support for Yubikeys and you can register up to 5 Yubikeys with your LastPass account. In That means that if for some reason you lose one of the keys, you can simply deregister it from your LastPass account and use your backup. In addition you can configure what types of systems require the Yubikey (mobile and desktop and so on) versus those that you don't want to use it on.

I am currently using a Yubikey on my android phone and all desktop systems, but not my venerable original iPad.
 
Using it is simple. I log into the LastPass count on my system using the browser plug-in (Available for Chrome, Firefox, Safari, and IE).

After the browser plug-in has validated my local password it then asks me to enter the Yubikey one-time password.
 

I put the key into the USB port and press a button. And I am now authenticated until my LastPass  authentication expires normally ( I have it set differently on different systems).

For using it with my Android phone I use the NEO NFC (Near Field Communication) feature to simply hold the key to the back of my phone and it automatically brings up LastPass and asks me for my password and logs me in.

I have purchased 2 NEOs and a standard key and after two weeks I am very happy with the result. The biggest con to all of this is building the habit of always carrying my hardware key with you. But that is a small price to pay to drastically decrease my exposure to security breaches.

More on the Yubikey coolness 

The Yubikey comes in several different forms: Standard, NEO, Nano, VIP

A quick summary of what they're good for is below:

• Standard - One-time password support and an additional "slot" for multiple configurations such as OATH. 
• NEO - Same as the Standard with the addition of NFC to allow authentication to those mobile devices that support it. Also has other cool features that I don't yet understand.
• Nano - Same as the Standard but ultra-tiny with additional support for the newer iPad camera docking station (This appears to be the only way to use the key with the iPad)
• VIP - Same as Standard with additional features that allow it to be used with Symantec VIP and PayPal.

As far as software support is concerned there are login authentication solutions for Windows and Linux (I have done nothing to test the Apple based OSes). In addition they provide an API for writing clients that connect their services.

As a special treat for me, they publish notes on how to use the Linux PAM (Pluggable authentication module) with SSH so that the hardware key is required in order to SSH into servers.

I'm going to be trying to set that up next week.

General usability

The LastPass browser plug-ins and integration with Yubikeys is very usable.

The only place where I have bumped up against usability is really in the documentation for the additional Yubikey integrations ( such as the Linux PAM and SSH integration).  They do seem intent on making it easy for third-party application support Yubikey authentication so I expect that will shift over time.

Next comes the real challenge: enrolling my wife and boys into using hardware keys.