Monument to Henry the Navigator

FeraLabs and UX Lisbon are organising a one day workshop, spread over two days on Remote Usability between the 12th to the 13th of May. The workshop is being held in conjunction with the conference.

The purpose of both workshops is to enable you to turn data and observations from Remote Testing into valuable insights which can be actioned. After attending both workshops you will know what questions to ask the data, and be able to prioritize the feedback.

The cost of the remote workshops if booked by the 28th of February is € 595.00, the price includes entry to the Ux Lisbon Conference as well. For just € 795 you can attend an extra 2 workshops and the conference. These prices are valid until 28th of February. There is a discount of 10% for Usability Professionals Association, Information Architecture Institute, Interaction Design Association, Association for Computing Machinery, and 15% for UxBrighton members.

As we at Feralabs want to spread the message about Remote Usability, we will offer you a voucher worth €225 off your next remote usability study, valid until the end of 2010 if you attend the workshop.

To book your place for the workshop simply go to http://www.ux-lx.com/registration.html and email lisbon@webnographer.com so that your place in the workshop is confirmed.

Workshop 1: An Introduction to Remote Usability

Remote Usability is a collection of methods that over the last year has become increasingly popular. Most of us are designing and developing products that will be used anywhere in the world, but most of our research methods are tied to a physical location. Remote sets the researcher free from being tied to a place, and also enables the testing of more people. By the end of the session you will become familiar with a number of remote research technologies, from remote ethnography to synchronous and asynchronous tools, and will be able to explain when such tools are suitable for user research.

The workshop will cover :-

• The different methods 1.30
• Remote ethnography
• Surveys
• Synchronous
• Asynchronous
• Study design
• Recruitment Methods
• Basic Analysis

Workshop 2: Asynchronous Remote Usability testing in detail

Asynchronous Remote testing is when the participant is separated from the evaluator by time and place. Asynchronous testing is significantly different from the usual user research methods, in this half a day workshop James and Sabrina will take you through the steps and theory of how to carry out a study from the design to the analysis of the results. The workshop will demystify the basic statistics that will help you understand the results. You will also learn the basics about how to carry out a International Study, from the legal issues to how to deal with translation.

• Study design
• True intent studies
• Task based studies
• Simple Statistics
• Advanced Analysis
• Analysing task based and true intend studies
• Importance of filtering
• Coding responses
• International studies
• Legal

Our new website has gone live for Webnographer, the remote usability testing tool that we have developed. We spent some time working on the Information Architecture of the site with Danny Hope, of UX Brighton fame.

The challenge we faced, which is common for many sites, is that our audience has many levels of knowledge both about Usability, and Remote Usability.  Information Architecture is critical for any website, that is getting the conceptual structure and logical organization of the site right.

So, how did we go about it?

We put the user at the heart of our design. This is usually done in the world of usability and information architecture through reasearch and creating personas. These personas are fictitious characters created to represent the different user types that might use a site.

There are many heated discussions to be found on the subject on the IxDA mailing list, with many different arguments for and against personas.

Here is our take on it. We don’t use them. Instead, we use real people! The important thing is that you think about your user, a person. Not a target market, that is a non-tangible thing. Thinking about a real person helps build empaphy, helps focus your design process, and lets you design better products. Personas help encourage this. But they take time to create as they are fictional creations of a user type. Real people work just as well, and even better, because you don’t have to imagine whether they would like something, as you can just give them a call and ask them.

The process of using real people

When we started designing the website, the first step was to map out the target audience that we had identified to date. As you can see on the picture of the white board, we broke our user base down by the industry sector they worked in (see picture of white board). For example in our case: In-house UX specialists,  Agencies, and Usability Professionals, Developers, etc.  Then we picked people who were representative for the culture of a sector, put their names on the board. (To protect their identity I have smudged out their names.) We know these representative users, as we have met them and keep meeting them throughout the concurrent ethnographic research that we are carying out.

The hard part in the exercise was to identify what are peoples motivations and feelings.  This is always hard as people have varied backgrounds, knowledge and views. Even people in the same sector have different mental models. Using real people to motivate the design, meant that if we where unsure about their motivation, all it took was a phone call, or meet up with them to find out.

Once we had our hypothesis of peoples motivations, we then could start creating the structure of the site and different message for communication. As you can see on the photo on the left the design started taking place, and the design for our website evolved.

So, are we finished?

No. We work agile and want to keep improving our site. People and technology will keep changing, and so too will the Webnographer website. Also, we are aware that some of the language used on the site is still geeky, and needs to be “translated” into non-geek language. This is something we are working on at the moment. Also we are very keen on getting feedback from our users, so if there is anything you like, dislike, or fell is missing, please feel free to leave you comments here. We are listening!

We would like to thank Danny for all the enlightenment in structuring the ideas and the insight that he gave us. Just look at our old holding page above and compare that to our new website. Its a great improvement!

Every time we talk about remote usability testing, many people have many different ideas of what remote testing is.  And there are many different remote testing methods.

Rex Hartson in 1996, defined remote usability testing as the method “wherein the evaluator, performing observation and analysis, is separated in space and/or time from the user”.

Two simple distinguishing factors, which are now commonly accepted, are that remote testing falls into two main categories:

1. Synchronous / moderated remote testing, where the evaluator has direct communication with the participant during the evaluation. This is normally carried out using screen sharing software, or talking over the phone.
2. Asynchronous / automated remote testing/ automated remote testing, where the evaluator has no contact with test participants. Asynchronous remote testing is the method used by Webnographer.

Asynchronous remote testing falls into three sub categories.

The simplest is analog remote testing, such as critical incident reporting, where the participant is sent a paper questionnaire which is filled in after carrying out the test. While this may seem like an easy method for evaluators, this is a method which can be very time consuming for the participant.

A bit more advanced are task-based online questionnaires, which are online tools that query the user for feedback after carrying out the task. Yet, those do not track any interaction on the page which makes it difficult to infer why problems are occurred. They therefore can only offer summative results such as success rates, time on task and satisfaction.

The most advanced tools use the CARUT methodology (Combined Asynchronous Remote Testing methodology), which combine logs of the user interaction during the task (hovering, clicks, key press, etc.) with questionnaires, which record rich qualitative feedback. This allows inferences about where and why problems occurred and offers both summative and formative results. Webnographer uses the CARUT methodology.

Yet, a challenge with most of the remote tracking tools which use the CARUT methodology is that they require the test participant to download software for it to work. We have found most people, worried about viruses and trojans, do not want to download software, especially one that tracks them. The other disadvantage of downloaded software is that it normally only works with certain types of browsers and operating systems. We have found that being able to work with different combinations of computers and browsers is important as we found the test participant’s configuration of their computers has a major impact on how they use a site.

Webnographer does not hold the above challenges. It works with nearly all computers, and with the most browsers. The user does not have to download anything and the website does not need to be modified. Webnographer gives therefore full freedom to carry out any test, including tests on your competitor’s website for competitive analysis.