Jakob Nielsen's Alertbox, January 2, 2007:
Fast, Cheap, and Good: Yes, You Can Have It All
Summary:
The sooner you complete a usability study, the higher its impact on the design process. Slower methods should be deferred to an annual usability checkup.
The consulting business has an old saying: "Fast, cheap, and good -- pick any two." The notion is that if you want something done quickly and inexpensively, it'll be of poor quality; if you want it quickly and done well, it'll be expensive, and so on. Although true in many areas, this maxim doesn't hold for one important aspect of usability: methodology.
In usability, the fastest and cheapest methods are often the best.
Of course, in any discussion involving value judgments like "good" and "best," we must define the quality criteria. My main quality criterion for usability is that it change the world. In other words, usability methods must set the product development directions and result in significant improvements to the shipping design.
Another criterion that's sometimes relevant is the quality and depth of the insights you derive into user behavior. In terms of insight, you can't be fast, cheap, and good at the same time. Truly deep insights require advanced usability methods, extensive research, and sufficient time to ponder the data. For example, you should conduct field studies where you observe users in their natural habitat. Unfortunately, this is expensive and time-consuming.
When to Go Fast and Cheap
For everyday design projects, discount usability methods are the best. In fact, generally, the faster and cheaper the study, the bigger its impact because the results will be available early enough to change your system's fundamental architecture.This is why I'm such a strong proponent of paper prototypes: mock up your design ideas on a few screens before you invest the resources on detailed design and implementation. Test each design with 5 users. And run many rounds of user testing. The cheaper each round, the more rounds you can fit within your budget, and the more you'll learn about user needs.
Some people complain because cheap and fast usability studies don't teach you everything about a design. But that's irrelevant. Yes, a bigger study would yield more results, but you'd get those results too late to influence the big design decisions. Also, the second or third rounds of testing will reveal anything you might have missed in the first, fast study.
Annual Usability Checkup
Fast and cheap usability is usually best, but sometimes you need to step back and get a bigger perspective. For important projects, you should do this once a year. For less important or slow-moving projects, it might be enough to have a checkup every two or three years.The big-picture checkup has three components:
- An independent review by an outside expert. Usability rests on your ability to consider the outside perspective, since your customers are all outsiders. A key downside to working for the same company year-round is that you eventually become acclimated to its thinking and assume that the way your website works is natural and intuitive. But "intuitive" basically means "what you've grown accustomed to." Having someone analyze your design from a fresh perspective shakes things up and offers a more neutral evaluation of your usability level compared to the rest of the world. That's also why it's a good idea to ask any new hires in your usability group to immediately write usability reviews of your design while they still have an outside perspective.
- If you run an intranet, your users are obviously insiders and it's less of a problem that you're embedded in the company. Still, it's valuable to have your intranet's usability independently assessed by someone who has studied a lot of other intranets.
- A competitive study that compares your design with that of three competitors. These studies are expensive and take time, but they're the best way to get broad insight into customer behavior and strategic ideas for the next year's design projects. Your competitors spent a lot of time and money designing solutions to approximately the same problems as your own; relative to their investments, it's dirt cheap to spend $40,000 on a competitive study to find out which of their ideas work and which flop.
- Sadly, intranet teams can't conduct competitive studies. Instead, read the Intranet Design Annual each year to hold your own intranet up against ten award-winning designs.
- A benchmark study to track your usability metrics and see how much better you're doing than last year. Quantitative studies are the most expensive of all and subject to many pitfalls unless you employ impeccable methodology. Thus, they are only for companies at the high end of organizational maturity with respect to usability.