What makes remote usability testing so exciting is that it reaches a
broader range of people and ages. We have even had a 89 year old
helping in our research.

Another aspect that makes the remote research a fascinating quest is
the phenomenon of context collapse. Users behave very differently when
they are alone, than when a researcher looks them over the shoulder.

This role is a great opportunity for you to apply creative and
analytical skills to the user research field. Creativity and knowledge
of different methods is required for the study design.

You will be working collaboratively with the research, marketing and
technology departments of our clients to design studies all over the
world, and then analysing the results, and presenting the findings back to
them.

There will be a steep learning curve as “remote” is very different
than normal usability testing.

You don’t need to know anything about remote usability testing yet,
but a strong interest in Psychology, Sociology, Anthropology,
Linguistics, or Behavioural Economics will be an advantage.

The location of work will be Hastings in East Sussex, UK for the
intense training period, with the potential to work remotely once the
training is completed.

If you are interested email jobs@feralabs.com with your CV and
cover letter, including descriptions of research projects you have
been involved in.

Monument to Henry the Navigator

FeraLabs and UX Lisbon are organising a one day workshop, spread over two days on Remote Usability between the 12th to the 13th of May. The workshop is being held in conjunction with the conference.

The purpose of both workshops is to enable you to turn data and observations from Remote Testing into valuable insights which can be actioned. After attending both workshops you will know what questions to ask the data, and be able to prioritize the feedback.

The cost of the remote workshops if booked by the 28th of February is € 595.00, the price includes entry to the Ux Lisbon Conference as well. For just € 795 you can attend an extra 2 workshops and the conference. These prices are valid until 28th of February. There is a discount of 10% for Usability Professionals Association, Information Architecture Institute, Interaction Design Association, Association for Computing Machinery, and 15% for UxBrighton members.

As we at Feralabs want to spread the message about Remote Usability, we will offer you a voucher worth €225 off your next remote usability study, valid until the end of 2010 if you attend the workshop.

To book your place for the workshop simply go to http://www.ux-lx.com/registration.html and email lisbon@webnographer.com so that your place in the workshop is confirmed.

Workshop 1: An Introduction to Remote Usability

Remote Usability is a collection of methods that over the last year has become increasingly popular. Most of us are designing and developing products that will be used anywhere in the world, but most of our research methods are tied to a physical location. Remote sets the researcher free from being tied to a place, and also enables the testing of more people. By the end of the session you will become familiar with a number of remote research technologies, from remote ethnography to synchronous and asynchronous tools, and will be able to explain when such tools are suitable for user research.

The workshop will cover :-

• The different methods 1.30
• Remote ethnography
• Surveys
• Synchronous
• Asynchronous
• Study design
• Recruitment Methods
• Basic Analysis

Workshop 2: Asynchronous Remote Usability testing in detail

Asynchronous Remote testing is when the participant is separated from the evaluator by time and place. Asynchronous testing is significantly different from the usual user research methods, in this half a day workshop James and Sabrina will take you through the steps and theory of how to carry out a study from the design to the analysis of the results. The workshop will demystify the basic statistics that will help you understand the results. You will also learn the basics about how to carry out a International Study, from the legal issues to how to deal with translation.

• Study design
• True intent studies
• Task based studies
• Simple Statistics
• Advanced Analysis
• Analysing task based and true intend studies
• Importance of filtering
• Coding responses
• International studies
• Legal

Our new website has gone live for Webnographer, the remote usability testing tool that we have developed. We spent some time working on the Information Architecture of the site with Danny Hope, of UX Brighton fame.

The challenge we faced, which is common for many sites, is that our audience has many levels of knowledge both about Usability, and Remote Usability.  Information Architecture is critical for any website, that is getting the conceptual structure and logical organization of the site right.

So, how did we go about it?

We put the user at the heart of our design. This is usually done in the world of usability and information architecture through reasearch and creating personas. These personas are fictitious characters created to represent the different user types that might use a site.

There are many heated discussions to be found on the subject on the IxDA mailing list, with many different arguments for and against personas.

Here is our take on it. We don’t use them. Instead, we use real people! The important thing is that you think about your user, a person. Not a target market, that is a non-tangible thing. Thinking about a real person helps build empaphy, helps focus your design process, and lets you design better products. Personas help encourage this. But they take time to create as they are fictional creations of a user type. Real people work just as well, and even better, because you don’t have to imagine whether they would like something, as you can just give them a call and ask them.

The process of using real people

When we started designing the website, the first step was to map out the target audience that we had identified to date. As you can see on the picture of the white board, we broke our user base down by the industry sector they worked in (see picture of white board). For example in our case: In-house UX specialists,  Agencies, and Usability Professionals, Developers, etc.  Then we picked people who were representative for the culture of a sector, put their names on the board. (To protect their identity I have smudged out their names.) We know these representative users, as we have met them and keep meeting them throughout the concurrent ethnographic research that we are carying out.

The hard part in the exercise was to identify what are peoples motivations and feelings.  This is always hard as people have varied backgrounds, knowledge and views. Even people in the same sector have different mental models. Using real people to motivate the design, meant that if we where unsure about their motivation, all it took was a phone call, or meet up with them to find out.

Once we had our hypothesis of peoples motivations, we then could start creating the structure of the site and different message for communication. As you can see on the photo on the left the design started taking place, and the design for our website evolved.

So, are we finished?

No. We work agile and want to keep improving our site. People and technology will keep changing, and so too will the Webnographer website. Also, we are aware that some of the language used on the site is still geeky, and needs to be “translated” into non-geek language. This is something we are working on at the moment. Also we are very keen on getting feedback from our users, so if there is anything you like, dislike, or fell is missing, please feel free to leave you comments here. We are listening!

We would like to thank Danny for all the enlightenment in structuring the ideas and the insight that he gave us. Just look at our old holding page above and compare that to our new website. Its a great improvement!

Last Tuesday I went to the first Groovy and Grails meet up in Brighton. Graeme Rocher, the founder of Grails lives in Brighton, so it was very exciting for the Brighton geek crowd to have their first very own meet up down at the coast and outside London. The most exciting thing was that Graeme was going to re-create Twitter in 40 minutes of live programming.

FeraLabs helped to organize the event, so I went along, even though I am not a programmer at all. My background is in design and HCI.  The thought of a geek programmer meeting was slightly disheartening, as I was worried that I was going to be sitting there with blank eyes, not understanding a word.

But to my surprise, my preconceptions were found to be wrong. I really enjoyed the talk.

Graeme started his presentation with a short introduction about Groovy and Grails, including the Grails philosophy, which did sound a little bit like a usability recipe for programming:

• build on the shoulders of giants
• embrace convention over configuration
• use sensible defaults
• achieve simplicity without sacrificing flexibility

Graeme also highlighted a few geek facts about Grails:

• Grails is 3-5 times faster then Ruby on Rails
• in March 2008 Grails had 7000 downloads a month, in March 2009 it had increased immensely to 70.000 downloads a month

Black window of horror

Then the programming action began. Graeme opened the black window of horror and created a new app. He explained the different folders that were created, and what the different items in them are. For a programmer this may have been a bit basic, but I got excited, as it was something that I recognized.

FeraLabs is building Webnographer in Grails. This means that I see these folders everyday. I work with them and around the code, working on html myself and trying not to break the Grails code.

Graeme Rocher

Graeme whizzed through the different steps of creating Twitter. He achieved this mostly through plug-ins, so it looked really easy. There was of course some manual programming, and Graeme explained the different steps and what the lines of codes were meant to do. 40 minutes later he had recreated the functionality of Twitter. It was ugly, but it did work.

A personal note about Grails: I liked that it seems to be using mainly natural language for its commands. This certainly helped me following the presentation, and will also be useful in my future work in getting a vague notion of what the different lines of code do and refer to.

In conclusion, for me as a designer, it was great to find out what all the bits and pieces of code do. I work with/around it every day and I am definitely curios. This does not mean that I will convert to being a programmer. Yet, this little insight that I got at the talk can definitely be beneficial to my future work, as well as other designers. It can help the communication between programmers and designers, encourage mutual understanding, and result in a better working relationship between both.

So I my question to the programmers is, are you willing to try seeing things from the design and user’s perspective too? And what do other designers think about hanging out with programmers a bit more? Do you think that cross mingling at different events can encourage better working relationships between designers and programmers?

Please leave your comments below. I’m keen to find out what you think.

The other night, after the Connecting Innovation event I got into a conversation with Professor John Carroll from the University of Sussex about the contribution of Sussex to advancing the internet.

I had read somewhere that the first transatlantic email had been sent from the University of Sussex campus. But nobody knew about this important event. Using Danny Hope’s iPhone in the pub afterwards, finding the information took ages, because of a total lack of any web pages mentioning it. But luckily I found a reference to it in a obituary in the The Daily Telegraph.

According to the article, in September 1973 Professor Dick Grimsdale and a group of American academics sent the world’s first transatlantic email message from the University of Sussex campus. This press release locates the event to a computer in a lecture room in the Engg 2 building. The computer was linked with a network of inter-connecting mainframe computers in the USA.

At the same conference, Vint Cerf and Bob Kahn presented a paper called “A Protocol for Packet Network Interconnection” (pdf). This paper laid the foundation for TCP/IP which is bedrock for how the internet works. TCP/IP is the language that computers on the Internet talk to each other.

In the UK, we have a tradition of placing a blue plaque, on buildings where somebody famous lived, or an important event happened. This post is to gather support for a blue plaque to be erected on the Engg 2 building on University of Sussex campus to commemorate the sending of the world’s first transatlantic email. If you agree, or disagree please put your name into the comments at the end of this post, stating your support, or non support, and we will start the ball rolling.

Post Updated with a photo of Engg II building by Graham McAllister.

Update II spoke with English Heritage and found out that they only do plaques inside London, outside it is done by the local council.

Ben Schneiderman from the University of Maryland, gave a fascinating talk in Cambridge on 5th March 2009 about the topic of “Information Visualization for Knowledge Discovery.”

Ben has authored many books and papers on human computer interaction, and was the founder of the Human Computer Interaction Laboratory at the University of Maryland. His keen interest is the field of information visualisation.

During his talk, Ben pointed out that in contrast to scientific visualization information visualization is a relatively young field as information visualization conferences have only been going for about 15 years. He added that, the challenge with information visualization is that the information keeps changing over time.

Ben presented his conceptual break down of information visualization tasks: “Overview -> zoom and filter -> details on demand.” What he means is that one should provide an overview first, showing all the information, for example complex graphs, diagrams and maps. This allows the user to orientate themselves and get the big picture. Then allow the user to zoom into more detail and filter out any unwanted information. Finally, allow the user to select an item and get more detail about it when required.

The most enlightening point that Ben made during his talk was that: “Information visualization gives you answers to questions you didn’t even know.” He went on to argue that “there should be a move from opportunistic discovery to a more systematic discovery of knowledge.”

Ben illustrated his argument with a number of demonstrations and screenshots of projects that he and his students have developed over the years. Each guides knowledge discovery thought the visualization of different patterns in the data. Ben emphasised that for information visualization “the interest is not in a particular value, but an overall view and patterns in the data”. Yet, he also emphazised the importance “trying to see the violations in the data that are contrary to your expectations”.

Ben demonstrated his famous treemap that has now been modified by many commercial companies. It has been modified for example  to visually show the constantly changing landscape of the google news aggregator, and even the New York Times has used it to show changes in truck and car sales.

Other tools which Ben showed were the ShapeSearcher which finds spikes in the data, Scattergrams which provide the opportunity of hunting for stuff, the alignment tool which can filter by event and show what happened before and after this event, and another tool which identifes gaps in the data.

The most intriguing example of finding patterns in data was Ben’s demonstration of the SocialAction tool, which uncovers hidden structures in social networks over time. The visualization presented the correlation between US senators voting the same way. It showed a strong that democrats and republicans vote the same way. Only four republicans sometimes voted similar to the democrats. Yet the most surprising finding through this visualization was that the correlation for democrats voting the same way was far stronger then republican voting the same way.

"The social network of the U.S. Senators voting patterns. Here, the threshold is raised to 290 votes. The Democrats' relationships are much more intact than the Republicans. Details-on-demand are provided for Senator Whitehouse, the senator with the highest degree at this threshold." (by Ben Schneiderman)

Ben concluded his talk with three key points for information visualization to guide knowledge discovery:
1.    Rank-by-Feature Framework, i.e. rank by what people want to know
2.    Decomposition of complex problems into multiple simpler problems
3.    Ranking guides discovery. It is important to provide systematic
       approaches for discovery.

Challenges of visual literacy

A theme that kept popping up in the talk and particular in the questions afterwards, was the challenge of visual literacy. Words can help to clarify matters of information visualization, but Ben explained that textual information is only good for simple queries (such as a rank list in Google search results). Visual tools on the other hand are better for complex queries.

For anyone who is interested in finding out more about the challenges of visual literacy, Ben recommended the work of Colin Ware, a perceptional psychologist, who looks at the challenges of understanding visual information.

Interesting reads about information visualisation:

Bederson, B. and Shneiderman, B. (2003) The Craft of Information Visualization: Readings and Reflections, Morgan Kaufmann Publ., San Francisco, CA. Amazon UK, Amazon US

Card, S., Mackinlay, J., and Shneiderman, B. (1999) Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann Publ., San Francisco, CA. Amazon UK, Amazon US

Tufte, Edward (1983) The Visual Display of Quantitative Information, Graphics Press, Cheshire, CT. Amazon UK, Amazon US

Tufte, Edward (1990) Envisioning Information, Graphics Press, Cheshire, CT. Amazon UK, Amazon US

Tufte, Edward (1997) Visual Explanations: Images and Quantities, Evidence and Narrative, Graphics Press, Cheshire, CT. Amazon UK, Amazon US

Ware, Colin (2004) Information Visualization, Second Edition: Perception for Design (Interactive Technologies), Morgan Kaufmann Publ., San Francisco, CA. Amazon UK, Amazon US

Ware, Colin (2008) Visual Thinking for Design, Morgan Kaufman, Burlington, MA. Amazon UK, Amazon US

We are helping to organise a Groovy and Grails meet up in Brighton on the 24 March at 7pm.

Grails is one of the fastest growing web frameworks, and as Brighton has many meet ups for other frameworks and programming languages including Rails and Python, so we thought that it was time that there would be one for Grails and Groovy.  Another important reason is the creator of Grails, Graeme Rocher, has just moved down to Brighton.

Grails and Groovy

The reason that Wired magazine, Pepsi Cola, Sky TV use Grails, and why we at FeraLabs chose to develop Webnographer in Grails, is that it allows one to build an application quickly.

Grails uses the Groovy programming language, which is another creation from the United Kingdom. Groovy runs on the Java platform (which means it will run on most computers), and has a unique syntax that means it easy for both somebody from a Java background to start programming in it, and somebody more used to other dynamic languages like Python, Ruby, and Smalltalk.

The Book

Graeme has just written a book on Grails. He will bring a couple along to sell to the event, but you can order the book The Definitive Guide to Grails 2nd Edition (Expert’s Voice in Web Development) on Amazon as well.

The meet up in Brighton

At the meet up, Graeme will demonstrate the speed of developing in the framework by building a Twitter clone in less than 40 minutes. In my personal opinion not only is Grails fast to develop a web application in, it also has a very low learning curve for both the experienced programmer and the novice.

When and Where?

Danny Hope of #UXBRI (the meetup for User Experiance in Brighton) has kindly arranged for us to use the historic Regency TownHouse.

24 March @ 7pm
Regency Town House
13 Brunswick Square
Hove, East Sussex
BN3 1EH

Sign up for the event on Upcoming to secure your place.

Every time we talk about remote usability testing, many people have many different ideas of what remote testing is.  And there are many different remote testing methods.

Rex Hartson in 1996, defined remote usability testing as the method “wherein the evaluator, performing observation and analysis, is separated in space and/or time from the user”.

Two simple distinguishing factors, which are now commonly accepted, are that remote testing falls into two main categories:

1. Synchronous / moderated remote testing, where the evaluator has direct communication with the participant during the evaluation. This is normally carried out using screen sharing software, or talking over the phone.
2. Asynchronous / automated remote testing/ automated remote testing, where the evaluator has no contact with test participants. Asynchronous remote testing is the method used by Webnographer.

Asynchronous remote testing falls into three sub categories.

The simplest is analog remote testing, such as critical incident reporting, where the participant is sent a paper questionnaire which is filled in after carrying out the test. While this may seem like an easy method for evaluators, this is a method which can be very time consuming for the participant.

A bit more advanced are task-based online questionnaires, which are online tools that query the user for feedback after carrying out the task. Yet, those do not track any interaction on the page which makes it difficult to infer why problems are occurred. They therefore can only offer summative results such as success rates, time on task and satisfaction.

The most advanced tools use the CARUT methodology (Combined Asynchronous Remote Testing methodology), which combine logs of the user interaction during the task (hovering, clicks, key press, etc.) with questionnaires, which record rich qualitative feedback. This allows inferences about where and why problems occurred and offers both summative and formative results. Webnographer uses the CARUT methodology.

Yet, a challenge with most of the remote tracking tools which use the CARUT methodology is that they require the test participant to download software for it to work. We have found most people, worried about viruses and trojans, do not want to download software, especially one that tracks them. The other disadvantage of downloaded software is that it normally only works with certain types of browsers and operating systems. We have found that being able to work with different combinations of computers and browsers is important as we found the test participant’s configuration of their computers has a major impact on how they use a site.

Webnographer does not hold the above challenges. It works with nearly all computers, and with the most browsers. The user does not have to download anything and the website does not need to be modified. Webnographer gives therefore full freedom to carry out any test, including tests on your competitor’s website for competitive analysis.

We are developing Webnographer using Agile Project Management Techniques. Agile builds the project in very small iterations of between 1 and 4 weeks. At the end of each iteration the software should be working. To give an example with Webnographer in our first iteration the whole system was usable from start to end. I.e. we could build a test, get participants to test a website, and analyse the results.

Not only did we want to just build a tool that could give summative results, for example how long a user takes on a task. But we wanted to build a remote testing tool that could be used for formative testing as well. Webnographer enables this by capturing users interactions on the page, as well as qualitative and quantitative questionnaire data. This means that Webnographer does not just report the state of a system (as in summative evaluations), but also provides insight into where, how and why user errors occur (as in formative evaluations).

So how could we build complex usability testing software in a week? We didn’t. When we started, the building of the test and the analysis of the results had to be done by hand. Over a number of iterations what had to be done by hand got automated.

The Agile technique is a revolt against the traditional waterfall approach. Under the old fashioned waterfall approach the business specifies exactly the software that will be built, the developers build it, and then it gets tested, and hopefully released. The challenge with the waterfall approach is that it often led to software projects running over time and budget.

With Agile each iteration can either add features or rework what has been built before. This has the advantage for developing Webnographer, that as we get feedback from each of the tests that we run with clients, the results feed back into each iteration and add to an improvement of the tool.

Up until now, we have mainly focused on the design of the test set up. The test set up was the most time consuming part and easiest to automate for Webnographer. Before it had its simple interface, to create a test each one had to be hand crafted, taking half a day. Modification of a test was hard, time consuming, and error prone. So we have focused on developing an easy to use interface for the test design. This is now complete.

The next iterations will focus on the analysis part of the product. The driving force is that we want to show people actionable results in an easy to understand way. Sample reports are to follow soon.

Without innovation the internet would still look like this. (Gutenberg's printing press. Photograph by Matthias Kabel)

The dilemma of visual convention vs. ground breaking new design seems to be a fearsome concern for usability specialists.  In a recent blog post on the Concept 7 blog, Stefan Wobben quotes a paper by Luis Santa-Maria and Mary C. Dyson form the University of Reading that investigated the impact of violating visual conventions on user’s performance and orientation. Santa-Maria and Dyson explain:

“Although initially violating visual conventions might hinder user performance and leave users disoriented this experiment indicates their experiment indicates that these problems can be short-lived and users can adapt reasonably fast to a new set of visual conventions.”

This is good to hear, yet its no news as such. If design had always only followed convention we would not have progressed from the written word to the printing press to computers and the internet.

“So the decision to whether conform or violate visual conventions when designing a website should ponder that although users might adapt quickly to novelty there is an initial performance hindrance and disorientation.”

They have a good point there in encouraging those violations. Too many studies focus on first time use, but not repeat users, how behaviour changes over time, and the experience and use of the system by expert users. A single lab study as a Q+A exercise just before the launch of your website is not going to do the trick in gaining this understanding. Usability is an ongoing process, not a one of label of approval. As Harry Brignal pointed out on his blog: “A UX designer’s job is never done.”

The one great thing about remote usability testing is that it is cost efficient and can therefore be carried out more often, than a lab study. Webnographer, as a remote usability testing tool, makes ongoing testing simple and affordable. It makes it easy to test the learning curve and behaviour and satisfaction of experienced users.

As Jared Spool explained back in 2003 small ongoing changes carry far less risk, then a major relaunch and re-design, which is very likely going to fail. As with the prinitng press, changes in improvements were small streched out over a lenght of time and we ented up with the internet, which makes the spreading of ideas and information easier than ever. For website design, the small changes allow you to measure the effect of that change on your users, and you will find out whether the change has made your site better or just different, and how it affects your users over time.