Often when I’m introduced to a new project team I’m introduced as the usability expert, which makes me cringe a little. Why is that I hear you ask, well the evidence suggest that there is no such thing as a usability expert.
Back in 1994 a chap called Rolf Molich (and friends) decided to see if us usability people were consistent in our ability to identify usability problems. He devised a simple test. Ask 9 usability experts to evaluate the usability of Microsoft’s free email service Hotmail and come up with a list of usability issues.
Rolf then aggregated all 310 usability problems uncovered from the experts and found something rather surprising.
Usability experts are not equal
It seems not all usability experts agree. Of the 310 usability issues found only two, that’s 2, were found by more than 6 experts. Of all the problems identified 75% were identified by only one expert. Usability it seems is in the eye of the beholder.
So what are we usability folk to do?
Rolf’s old partner is crime, Jakob Nielsen (Jakob and Rolf came up with the original usability heuristics) conducted a wonderful piece of research and found that 5 users will identify 70% of all usability problems. Notice the similarity in those numbers? 75% of problems are identified by 6 experts vs. 70% of problems found by 5 users.
The recommendation: run a usability test with real users. It need not be expensive, only 5 users are needed after all, it need not be exhaustive, a hour is more than enough.
Us usability people can help organise and interpret the results and most important of all make recommendations to fix the usability problems identified. A usability expert has techniques and theory but can’t always uncover usability issues any better than a user can.
Finally, be wary of people describing themselves “usability experts”, we now know there is no such thing.
8 replies on “Usability expert, me? Never.”
Hi Joe
I’d love to see the Molich 1994 study run again. It should be easily replicable, and we can find out if things moved on a lot since then.
Would you be interested in participating in a re-run of this study? I’m sure we can get 7 other companies interested without much trouble!
My colleague Walt has shared the latest from Rolf and co: http://www.dialogdesign.dk/CUE.html
Harry: be an interesting experiment (sorry for taking 2 years to reply!)
@mrjoe I’ve written a lot about research being everyone’s job but there are definite reasons for a specialist rather than a generalist.
@emmaboulton @mrjoe being *involved* in the research is everyones job, but it should be led by someone who knows what they’re doing.
@leisa @mrjoe absolutely.
@emmaboulton @leisa It’s the difference between a specialist & an expert. Experts ‘know’ the answers, specialists know how to get t/ answer
@mrjoe @leisa yeah I suppose that comes with more and more experience and seniority. It’s a bit more of a grey area for me – a continuum.
@mrjoe @emmaboulton @leisa Semantics? A good practitioner has much knowledge, but also knows the limitations of what they know. You know? 😉