Archive for January, 2009

Designers Dilema: visual convention vs. breaking new ground

Wednesday, January 21st, 2009
Without innovation the internet would still look like this. (Gutenberg's printing press. Photograph by Matthias Kabel)

Without innovation the internet would still look like this. (Gutenberg's printing press. Photograph by Matthias Kabel)

The dilemma of visual convention vs. ground breaking new design seems to be a fearsome concern for usability specialists.  In a recent blog post on the Concept 7 blog, Stefan Wobben quotes a paper by Luis Santa-Maria and Mary C. Dyson form the University of Reading that investigated the impact of violating visual conventions on user’s performance and orientation. Santa-Maria and Dyson explain:

“Although initially violating visual conventions might hinder user performance and leave users disoriented this experiment indicates their experiment indicates that these problems can be short-lived and users can adapt reasonably fast to a new set of visual conventions.”

This is good to hear, yet its no news as such. If design had always only followed convention we would not have progressed from the written word to the printing press to computers and the internet.

“So the decision to whether conform or violate visual conventions when designing a website should ponder that although users might adapt quickly to novelty there is an initial performance hindrance and disorientation.”

They have a good point there in encouraging those violations. Too many studies focus on first time use, but not repeat users, how behaviour changes over time, and the experience and use of the system by expert users. A single lab study as a Q+A exercise just before the launch of your website is not going to do the trick in gaining this understanding. Usability is an ongoing process, not a one of label of approval. As Harry Brignal pointed out on his blog: “A UX designer’s job is never done.”

The one great thing about remote usability testing is that it is cost efficient and can therefore be carried out more often, than a lab study. Webnographer, as a remote usability testing tool, makes ongoing testing simple and affordable. It makes it easy to test the learning curve and behaviour and satisfaction of experienced users.

As Jared Spool explained back in 2003 small ongoing changes carry far less risk, then a major relaunch and re-design, which is very likely going to fail. As with the prinitng press, changes in improvements were small streched out over a lenght of time and we ented up with the internet, which makes the spreading of ideas and information easier than ever. For website design, the small changes allow you to measure the effect of that change on your users, and you will find out whether the change has made your site better or just different, and how it affects your users over time.

Does culture affect online behaviour?

Monday, January 19th, 2009
Shoping in Europe

Shopping in Europe by supermuch / James

I was asked last week, if culture effects the user experience and the users’ use of the system.

What we have found is that consumers knowledge has a major impact on online behaviour. For example most Europeans hardly use the internet to shop. Therefore they have both little knowledge and experience of online buying. Using numbers from Jupiter and Forester research shows the UK online user spends twice as much online as the German, and French user.

Rory Sutherland of Ogilvy points out that most marketers base their strategies on US research, Ogilvy has done a study that shows key differences between shopper habits offline in the US and the UK. We believe this is the case online as well.

Other significant differences in behaviour that we have found include that the British find reading a timetable from top to bottom easier. Most Europeans are the opposite, and find reading a timetable left to right faster. We do not know if a similar behaviour would exist with product listings, and maybe a study is needed to find out.

Online Shoping by Gareth Saunders

Many other differences in culture and knowledge also affect behaviour. Including time — some people do not understand the 24 hour clock others do not understand the 12 hour clock. The reading of prices — Europeans use a dot to denote thousands, and coma to denote the decimal point. Anglo Saxons do the opposite. This can lead somebody at a fast glance to think a product is cheaper or more expensive than it is.

To make matters even more complex Europe is becoming more culturally mixed. Over fifty percent of Londoners where not born in London. There are so many French people living in London, that President Sarkozy of France visited London during his election bid. He claimed that “London, has become one of the great French cities”. A Polish, or German, or Scandinavian Politician would make the same claim for their country. This cultural mix doesn’t just apply to London, but holds true for many other countries, and cites in Europe, including Spain, Italy, and France.

We have found from previous studies that a Dutch person living in London, is quite willing to go through the whole process of purchasing online in English until they reached the terms and conditions which they want to read in Dutch. They could have switched languages earlier in the process. And the Dutch user wants far more reassurances over being protected from online fraud than the Italians.

What this means for an e-commerce site is that it needs to work with people with different cultural backgrounds in different cities in different countries. It needs to work for an English people living in Spain, just as much that it needs to work for a Dutch person living in Berlin.

Not only have we found many issues on culture, but additionally the configuration of the users computer effects usability. The size of the screen, and if you are displaying rich media, the type and version of the browser, and the power of the processor. Most lab studies are done with one standard set-up of the computer, using the same size monitor.

As Dr. Harry Brignull points out it is critical to use the right participants for your usability testing. If you restricted yourself to a lab, but your users are distributed all around the world, then the question is: “Are you testing the right users?”

Webnographer can help identifying cultural issues, and many more. Using some of our own analytic techniques we can gain insight into qualitative feedback and satisfaction ratings from the questions asked of the participants, but also gain knowledge from the participants behaviour while carrying out the task, by tracking their interactions on the page.

For cardsorting is 20 people enough?

Wednesday, January 14th, 2009

Kath Straub aka “DriveBy” points out Tom Tullis and Larry Wood’s research, How Many Users Are Enough for a Card-Sorting Study?

A card sort exercise is used in Information Architecture so that a common set of categories and relationships emerges that can be used to organise such areas as website navigation.

To perform a card sort, a participant is given a set of index cards with terms written on them. This person sorts the cards either into a predefined category or chooses there own name for the category name.

Tullis & Wood’s research in 2004 used 168 participants, with 48 cards.  Kath Straub draws the following conclusion from the paper:-

Based on their simulations, Tullis & Woods concluded cardsorting studies with just 20-30 participants are robust and predictive.

Tullis and Wood actually point to some area of concern about their research, namely that :-

As always, we must exercise appropriate caution in generalizing results from one study. Results will obviously differ as a function of the homogeneity of the participants in a sample and such things as the instructions given to the participants for the card-sorting task.

The 168 participants, even though they where from all over the world, they all worked at one corporation. Fidelity Investments. Corporations have cultures. If you where to do the study at another company would the numbers be the same?

What always matters in these sorts of studies is the variance of the data. Living in Europe one quickly realises that people are not homogeneous. The Germans have many ways of categorising bread, while the British just have three, brown, white, and hovis. There is the famous global example of the Eskimos having 48 different names for snow. How an Eskimos will categorise the weather will be very different than a person living in the desert.

Even within one field there can be much disagreement in how to categorise data. If you take the field of Taxonomy, which is the science of classification, there is much disagreement on how to categorise things.  One of the reasons we do not know how many livings species there are is because of a disagreement on what belongs to what species. For example scientists can not agree on the number of genius of the Eucalyptus.

So how many people are enough. Well it depends on the homogeneity of your target market and what you want them to classify. Twenty participants may be enough and may not be. What effects how people will categorise items depends on their knowledge and opinion. The more diverse the knowledge and opinion the more participants you will need. The only way in finding out is doing some tests and look at the variance.

In a later post we will discuss this more about the number of particpants needed for different kinds of tests.