This is an update of where we are with Webnographer.

We are developing Webnographer using Agile Project Management Techniques. Agile builds the project in very small iterations of between 1 and 4 weeks. At the end of each iteration the software should be working. To give an example with Webnographer in our first iteration the whole system was usable from start to end. I.e. we could build a test, get participants to test a website, and analyse the results.

Not only did we want to just build a tool that could give summative results, for example how long a user takes on a task. But we wanted to build a remote testing tool that could be used for formative testing as well. Webnographer enables this by capturing users interactions on the page, as well as qualitative and quantitative questionnaire data. This means that Webnographer does not just report the state of a system (as in summative evaluations), but also provides insight into where, how and why user errors occur (as in formative evaluations).

So how could we build complex usability testing software in a week? We didn’t. When we started, the building of the test and the analysis of the results had to be done by hand. Over a number of iterations what had to be done by hand got automated.

The Agile technique is a revolt against the traditional waterfall approach. Under the old fashioned waterfall approach the business specifies exactly the software that will be built, the developers build it, and then it gets tested, and hopefully released. The challenge with the waterfall approach is that it often led to software projects running over time and budget.

With Agile each iteration can either add features or rework what has been built before. This has the advantage for developing Webnographer, that as we get feedback from each of the tests that we run with clients, the results feed back into each iteration and add to an improvement of the tool.

Up until now, we have mainly focused on the design of the test set up. The test set up was the most time consuming part and easiest to automate for Webnographer. Before it had its simple interface, to create a test each one had to be hand crafted, taking half a day. Modification of a test was hard, time consuming, and error prone. So we have focused on developing an easy to use interface for the test design. This is now complete.

The next iterations will focus on the analysis part of the product. The driving force is that we want to show people actionable results in an easy to understand way. Sample reports are to follow soon.

Shopping in Europe by supermuch / James

I was asked last week, if culture effects the user experience and the users’ use of the system.

What we have found is that consumers knowledge has a major impact on online behaviour. For example most Europeans hardly use the internet to shop. Therefore they have both little knowledge and experience of online buying. Using numbers from Jupiter and Forester research shows the UK online user spends twice as much online as the German, and French user.

Rory Sutherland of Ogilvy points out that most marketers base their strategies on US research, Ogilvy has done a study that shows key differences between shopper habits offline in the US and the UK. We believe this is the case online as well.

Other significant differences in behaviour that we have found include that the British find reading a timetable from top to bottom easier. Most Europeans are the opposite, and find reading a timetable left to right faster. We do not know if a similar behaviour would exist with product listings, and maybe a study is needed to find out.

Online Shoping by Gareth Saunders

Many other differences in culture and knowledge also affect behaviour. Including time — some people do not understand the 24 hour clock others do not understand the 12 hour clock. The reading of prices — Europeans use a dot to denote thousands, and coma to denote the decimal point. Anglo Saxons do the opposite. This can lead somebody at a fast glance to think a product is cheaper or more expensive than it is.

To make matters even more complex Europe is becoming more culturally mixed. Over fifty percent of Londoners where not born in London. There are so many French people living in London, that President Sarkozy of France visited London during his election bid. He claimed that “London, has become one of the great French cities”. A Polish, or German, or Scandinavian Politician would make the same claim for their country. This cultural mix doesn’t just apply to London, but holds true for many other countries, and cites in Europe, including Spain, Italy, and France.

We have found from previous studies that a Dutch person living in London, is quite willing to go through the whole process of purchasing online in English until they reached the terms and conditions which they want to read in Dutch. They could have switched languages earlier in the process. And the Dutch user wants far more reassurances over being protected from online fraud than the Italians.

What this means for an e-commerce site is that it needs to work with people with different cultural backgrounds in different cities in different countries. It needs to work for an English people living in Spain, just as much that it needs to work for a Dutch person living in Berlin.

Not only have we found many issues on culture, but additionally the configuration of the users computer effects usability. The size of the screen, and if you are displaying rich media, the type and version of the browser, and the power of the processor. Most lab studies are done with one standard set-up of the computer, using the same size monitor.

As Dr. Harry Brignull points out it is critical to use the right participants for your usability testing. If you restricted yourself to a lab, but your users are distributed all around the world, then the question is: “Are you testing the right users?”

Webnographer can help identifying cultural issues, and many more. Using some of our own analytic techniques we can gain insight into qualitative feedback and satisfaction ratings from the questions asked of the participants, but also gain knowledge from the participants behaviour while carrying out the task, by tracking their interactions on the page.