Jakob Nielsen's Alertbox, February 28, 2011

Mobile Content Is Twice as Difficult

Summary:
When reading from an iPhone-sized screen, comprehension scores for complex Web content were 48% of desktop monitor scores.

It's more painful to use the Web on mobile phones than on desktop computers for many reasons:

Slower downloads
No physical keyboard for data entry
No mouse for selection; no mouse buttons to issue commands and access contextual menus (indeed fewer signaling states, as discussed further in our seminar on Applying HCI Principles to Real World Problems: a touchscreen only signals "finger-down/up," whereas a mouse has hover state in addition to button press/release)
Small screen (often with tiny text)
Websites designed for desktop access instead of following the usability guidelines for mobile
Whacky app UIs that lack consistency

New research by R.I. Singh and colleagues from the University of Alberta provides one more reason: it's much harder to understand complicated information when you're reading through a peephole.

Singh and colleagues ran a Cloze test on the privacy policies of 10 popular websites: eBay, Facebook, Google, Microsoft, Myspace, Orkut, Wikipedia, WindowsLive, Yahoo!, and YouTube.

I did a quick analysis of Facebook's privacy policy, which features:

5,789 words, or 35 times the number of words users read during an average page visit.
13^th grade reading level, so only people with a year or more of university would find the text easy to read.
Nicely formatted for Web reading, including a good use of subheads, bulleted lists, and highlighted keywords, in keeping with guidelines for writing for the Web. (That said, these guidelines also call for short text and an 8^th-grade reading level when targeting a broad consumer audience and not just Harvard students.)

(Last week, Facebook posted a draft of a rewritten privacy policy. The content is now written at an 11^th-grade reading level, which is a nice improvement. Even better, the endlessly scrolling document has been broken up into multiple pages, with a simple internal navigation system and clear summaries that give users an overview of the information. Adding structure and navigation to turn a nightmarishly long, linear document into a tight information space is an example of what I call "mini-IA." Well done.)

In any case, there's no doubt that privacy policies count as complicated Web content.

In Singh's study, 50 test participants completed Cloze tests while reading the privacy policies on either a desktop-sized screen or an iPhone-sized screen. (The study didn't use an actual iPhone, but since users didn't perform navigation or any interactions other than reading and scrolling, the specific device shouldn't impact the comprehension results.)

Results:

Desktop screen: 39.18% comprehension score
Mobile screen: 18.93% comprehension score

Test scores must be 60% or higher for a text to be considered easy to understand. Even while reading from a desktop screen, users achieved only 2/3 of the desired comprehension level.

The obvious first conclusion from the study is that major websites have overly complicated content in their privacy policies. Of course, it's not exactly news that privacy policies are incomprehensible. Users know this and usually don't read them.

In our previous research, we have found that users treat "user agreements" and similar site copy with contempt. In approaching such agreement text, users

read 10%
scan 17%
skip 73%

And this is users' behavior during usability studies, where they know they're being video-recorded. At home, I expect they read even less. Basically, people click "I agree" without reading what they're "agreeing" to.

Why Mobile Reading Is Harder

User comprehension scores on the Cloze test were 48% of the desktop level when using the iPhone-sized screen. That is, it's roughly twice as hard to understand complicated content when reading on the smaller screen.

Why? In this case, people were reading only a single page of information, and they were shown that page as part of the study without having to find it. Thus, navigation difficulties or other user interface issues cannot explain the increased difficulty. Also, users were tested in a lab, so there were no issues related to walking around with the phone or being disturbed by noises or other environmental events. (In the real world, such distractions and degradations of the user experience further reduce people's ability to understand mobile phone content during true mobile use.)

The only reason mobile scored lower than desktop is the screen size, since that was the only difference in the study conditions.

A smaller screen hurts comprehension for two reasons:

Users can see less at any given time. Thus, users must rely on their highly fallible memory when trying to understand anything that's not fully explained within the viewable space.
- Less context = less understanding
Users must move around the page more, using scrolling to refer to other parts of the content instead of simply glancing at the text. Scrolling introduces 3 problems:
- It takes more time, thus degrading memory.
- It diverts attention from the problem at hand to the secondary task of locating the required part of the page.
- It introduces the new problem of reacquiring the previous location on the page.

This new research provides striking support for the main conclusion in our usability studies of mobile websites: websites (and intranets) must design a separate mobile version for optimal usability. Specifically, complicated content should be rewritten to be shorter, with secondary information deferred to subsidiary pages.

Learn More

Full-day seminars at the annual Usability Week conference:

Reference

R.I. Singh, M. Sumeeth, and J. Miller: "Evaluating the Readability of Privacy Policies in Mobile Environments," International Journal of Mobile Human Computer Interaction, vol. 3, no. 1 (January–March 2011), pp. 55–78.