It's been 11 years since our original studies of e-commerce usability — long enough that it's definitely worth revisiting the topic. The bottom line? The number of usability guidelines for e-commerce sites has increased from 207 in the first edition of the report to 874 in the new edition. Using this rough metric, we now know 4.2 times as much about e-commerce user experience as we did during the dot-com bubble.
Usability: Marked Improvements
In our study 11 years ago, we recorded a success rate of 56% across 496 task attempts on the e-commerce sites of the day. In our new research, we observed 507 e-commerce task attempts and measured a success rate of 72%.
In other words, during the dot-com bubble, users attempting to shop on e-commerce sites failed almost half the time. No wonder the bubble burst, with sites that bad. Now, users fail slightly more than a quarter of the time. Sites are still leaving plenty of money on the table, but not as much.
Today, our main reason to recommend usability improvements for e-commerce sites is the competitive pressure from other sites that keep getting better. Today's consumers aren't satisfied with sites that simply make it possible to shop; the experience must also be pleasant. So, while you could argue for improving design purely to reduce user failures, it's now necessary to look beyond simple success rates — even though a user's ability to complete tasks is the obvious first requirement.
Search remains a sore point, even though it has improved somewhat. In our first study, users succeeded in their initial search attempt on an e-commerce site 51% of the time. In the new study, users' first within-site query was successful 64% of the time.
However, user expectations for search quality are far beyond what today's websites actually deliver. As with most aspects of web usability, user expectations are set by their aggregated user experience from around the web. (As Jakob's Law states: users spend the majority of their time on other sites than yours.) In the case of search, this means Google and the other major search engines — which, while not perfect, do work pretty well.
When users search an e-commerce site and don't find what they want, they often assume that the site doesn't have the desired product. Users have poor search skills and will typically leave a site rather than figure out how to reformulate their queries.
Old Study Findings Stand
Our first study reported on lab-based usability testing of 20 websites across 7 product categories: clothing, department stores, entertainment, flowers, food, furniture, and toys. We conducted tests in 2 countries: the US and Denmark. Although relatively limited, this research was more than anybody else had done at the time. And, of the resulting 207 early design guidelines, 206 were confirmed in our recent — and much more elaborate — research.
The one guideline we retracted? Offer a special welcome page for new shoppers. Today, it's safe to assume that by the time users arrive at your site, they've probably already shopped elsewhere. E-commerce is no longer new, and users no longer need to be told what it is. As long as the site is easy to use, people will use it.
The fact that 99.5% of our 11-year-old guidelines were confirmed is an indication of the longevity of usability findings. Our design recommendations are based on the characteristics of the human mind, which change much more slowly than the technology that seems to fascinate so many people in the field.
New User Research
Our new research encompassed 3 usability methods: traditional user testing (as in the original study), eyetracking, and field research in the form of a diary study . We conducted studies in 3 countries: most sessions were in the US (in Georgia, Indiana, and New York), with a smaller number of users in China and the UK.
In total, users tested 206 sites — more than 10 times as many as were tested in our original research. As I noted above, we identified "only" 4.2 times as many usability guidelines, which indicates some degree of diminishing returns from enlarging user research studies.
The tested sites covered an incredibly wide range of industries and products, from high culture to low (Paris Museum Pass and NASCAR), from cheap to expensive (Walmart and Tiffany's), from virtual to physical (Ticketmaster and The Container Store), and from general interest to highly specialized (Zappos and Lightbulbs.com).
Except for the diary studies, all studies were conducted as direct empirical observation of users' actual behavior as they engaged in online shopping. We sat next to users, one-on-one, and asked them to think out loud as they performed specific tasks. This research approach provides deep insights into why users behave the way they do and results in findings that are not available from other methods.
Some test tasks were highly directed, assessing the degree to which the design supported users who arrived at a site with a predetermined goal. For example: "Buy an air conditioner to put in the window of a room that is 10 feet by 20 feet (200 square feet). Get one that is energy-efficient and that has a remote. Buy it from www.homedepot.com."
Other tasks were broader, assessing the degree to which the site could inspire users who didn't have a particular need in mind. For example: "You just got a promotion and a bonus, and you want to treat yourself. Buy yourself something. Spend no more than £200 at Links of London."
We also tested web-wide tasks that didn't specify which site the user should visit. For example, in one task, we gave users a burned-out light bulb and the following instructions: "The light bulb in your desk lamp just burned out. Get a replacement for it."
Finally, we tested a range of customer service tasks. For example: "Can you get a refund for tickets you buy from cinema.com.hk if there is a typhoon signal?"
Supporting Different Types of Shoppers
Our diary study looked at why and how people shop on their own, when not given test tasks to perform. In total, diary-writers recorded 263 visits to e-commerce sites.
2/3 of the time, users visited a site with a pre-determined goal: 35% of visits were to look for a particular type of product (without having a specific product in mind), and 27% of visits were to look for a specific product.
1/3 of the time, users visited a site to see what the site had to offer. Such visits were often prompted by the receipt of an email newsletter or otherwise learning about sales or special offers.
Sites must support all four types of use:
category research, where users identify and buy products that best match their needs;
bargain -hunting; and
browsing for inspiration.
Finally, some users are one-time shoppers. They don't know the site, and they don't intend to return, but they might shop there once (because they received a gift card, for example, or a relative wanted a gift the site carried).
Bad Content Kill Sales
The first rule of e-commerce design remains: if the customer cannot find the product, the customer cannot buy the product.
But in our new studies, the main problem was not so much finding the product as it was finding information about the product. Indeed, 55% of the 143 user failures we observed were caused by bad content — typically, incomplete or unclear information or uninformative error messages. Users sometimes indicated that they planned to call or email the site, a clear sign that the company had neglected the opportunity to answer their questions on the site itself.
Content can be verbal or visual. Either way, it must provide the information users need to both decide on products and feel comfortable trusting the site with their money.
The key downside of e-commerce is that users cannot touch, feel, see, taste, or smell the offerings. Nor do customers benefit from the essential credibility boost of having the purchase in hand before paying the price. That is, there's no tactile experience. Online shopping is purely an information experience. (Or user experience, as we like to say.) This again places a huge premium on good content. And many sites fail to deliver.
Finally, how many times do we have to repeat the guideline that when users ask for a bigger product photo, the site should show a dramatically bigger picture? Inadequate photo enlargement was #10 on the list of top-10 web-design mistakes as long ago as 2005. Although an old guideline, it's still frequently violated.
Loyalty = Business
One of our study participants said, "If I have a good experience with something, I'll stick with it forever." Not all users are that loyal, but our research does indicate tremendous benefits from fostering customer loyalty in e-commerce.
In the web-wide tasks, we didn't specify which site users should visit to make purchases. Not surprisingly, half of the users went straight to a search engine. But it was a bit of a surprise that the other half of users went directly to some site they already knew.
Bypassing the search engines' tollbooth on the information highway is user loyalty's first benefit. But the advantages reach much further.
Of users who started by searching the web, only 39% completed their task on the first site they selected from the SERP (search engine results page). That is, almost 2/3 of search users abandoned their first love and proceeded to do business elsewhere. This outcome demonstrates that SEO and good search engine ranking are necessary but not sufficient for Internet business success. It's actually more important to satisfy users once they arrive at your site. Search users exhibit little loyalty to sites they happen to click on.
In contrast, users who bypassed search and went directly to a preferred site overwhelmingly gave their money to that site: 71% did so, while only 29% completed their task elsewhere. (Of course, the latter group is a significant amount of lost business; clearly, you can't take e-commerce users for granted, even when they're loyal to your site.)
The benefits of loyalty might make you push aggressive registration requirements, but that would be a mistake. You must convert first-time shoppers before they can become long-time shoppers, and users strongly resent up-front registration. On the other hand, users frequently complained about the drudgingly large amount of data entry required to complete a purchase. So, by reminding repeat users of the time savings, you can nudge them to register eventually.
In general, sites would benefit from a longer-term perspective on the full sales cycle and total user experience. A transmedia design strategy should reach beyond the main website to encompass a mobile site, an email newsletter strategy, and good customer service (including good confirmation messages). Yes, e-commerce user experience has come far, but it has even further to go to truly meet customer needs.
Full research fuindings with all the design guidelines for e-commerce sites are available for download as a series of 13 reports. (Note that this link leads to a newer version of the report than the one described in this article. However, most of the findings were confirmed and are still discussed in the new report.)