We are developing Webnographer using Agile Project Management Techniques. Agile builds the project in very small iterations of between 1 and 4 weeks. At the end of each iteration the software should be working. To give an example with Webnographer in our first iteration the whole system was usable from start to end. I.e. we could build a test, get participants to test a website, and analyse the results.

Not only did we want to just build a tool that could give summative results, for example how long a user takes on a task. But we wanted to build a remote testing tool that could be used for formative testing as well. Webnographer enables this by capturing users interactions on the page, as well as qualitative and quantitative questionnaire data. This means that Webnographer does not just report the state of a system (as in summative evaluations), but also provides insight into where, how and why user errors occur (as in formative evaluations).

So how could we build complex usability testing software in a week? We didn’t. When we started, the building of the test and the analysis of the results had to be done by hand. Over a number of iterations what had to be done by hand got automated.

The Agile technique is a revolt against the traditional waterfall approach. Under the old fashioned waterfall approach the business specifies exactly the software that will be built, the developers build it, and then it gets tested, and hopefully released. The challenge with the waterfall approach is that it often led to software projects running over time and budget.

With Agile each iteration can either add features or rework what has been built before. This has the advantage for developing Webnographer, that as we get feedback from each of the tests that we run with clients, the results feed back into each iteration and add to an improvement of the tool.

Up until now, we have mainly focused on the design of the test set up. The test set up was the most time consuming part and easiest to automate for Webnographer. Before it had its simple interface, to create a test each one had to be hand crafted, taking half a day. Modification of a test was hard, time consuming, and error prone. So we have focused on developing an easy to use interface for the test design. This is now complete.

The next iterations will focus on the analysis part of the product. The driving force is that we want to show people actionable results in an easy to understand way. Sample reports are to follow soon.