Summary: Users have highly emotional reactions to newsletters which feel much more personal than websites. In usability testing, success rates were high for subscribe and unsubscribe tasks, but users were frustrated by newsletters that demanded too much of their time.
The most significant finding from our recent usability testing of ten email newsletters is that users have highly emotional reactions to newsletters . This is in strong contrast to studies of website usability, where users are usually much more oriented towards functionality. Even a website that you visit daily will feel like a tool where you want to get in and get out as quickly as possible and not connect with the site.
Newsletters feel personal because they arrive in your inbox; you have an ongoing relationship with them. In contrast, websites are things you glance at when you need to get something done or find the answer to a specific question.
The positive emotional aspect of newsletters is that they can create much more of a bond between user and company than a website can. The negative aspect is that usability problems have much stronger impact on the customer relationship than they normally do.
For example, one user received an error message which read "Email address is not valid." Even though this would be a poor error message in any user interface, the emotional aspect to newsletters increased the user's anger: "Mine's as valid as the next person's! [...] It's questioning my validity as an entity in cyberspace."
High Nominal Usability
Our test users experienced unprecedented high levels of task completion in their attempts to subscribe and unsubscribe to the newsletters in the study: 78% for subscribing and 92% for unsubscribing . When you consider that most usability studies find success rates around 50-60% for other areas of Web design, the success rates for newsletter usability are incredibly high, even though they are still lower than anything we would deem to be a truly great user experience.
Given these success rates, a newsletter with 50,000 subscribers could add an estimated 14,000 subscribers on average if everybody could operate its subscription interface correctly.
There are probably two reasons for the fairly high success rates we found in the newsletter study. First, the functionality is very simple: Get on or off a mailing list. In fact, the main failures came on websites that had more complex functionality, such as combining newsletter subscriptions with site registration. In general, it is easier to design a simple user interface when the underlying functionality is simple.
The second reason that the subscription process had much better usability than other Web designs is that newsletter designs are highly accountable . In many other areas of Web design, project managers can delude themselves and their bosses that there are benefits from user-hostile designs such as splash pages. Create a design where people can't find what they want, and page views may even go up as users wander aimlessly before they leave and give up doing business with the company. With a newsletter subscription design, either users subscribe or they don't. In the latter case, websites will sooner or later tone down the excesses of their design and focus on simplicity, after which subscriptions will increase. If a site were to replace a simple design with a complex one, it would soon notice a lack of new subscriptions and revert to the previous design, writing off the bad design as an expensive lesson in usability.
Low Perceived Usability
Even though users were able to successfully unsubscribe 92% of the time during the test sessions, they often refrained from even trying to get off mailing lists that they didn't want any more.
The three main reasons people didn't attempt to unsubscribe were:
- Emotional attachment to the newsletter: It doesn't feel good to sever the relationship, even when you don't read the mailings any more.
- Low expectations for usability on the website: People assumed that it would be difficult and time-consuming to unsubscribe, so they postponed the job for another day and simply deleted the current issue of the newsletter.
- Fear that unsubscribing would not work and would subject the user to getting even more mail: Many people have heard that asking to get off spam lists only serves to confirm the validity of your address to the spammers, and this notion has turned into an urban legend that contaminates their mental model of legitimate newsletter operators as well.
Whatever the reasons, it is clear that mailing list owners should not assume that all of their subscribers actually want to receive the newsletter. Many users may simply have neglected to unsubscribe for one of the stated reasons.
Some newsletters deliberately make it difficult to unsubscribe by hiding the instructions or making them overly complex. The theory is probably to retain as many subscribers as possible in order to have maximum reach for permission marketing programs. But in reality, you don't have a user's "permission" once that user stops wanting the newsletter, whether or not the user jumps through the hoops to get off the list. If users keep getting unwanted newsletters, the messages will start to backfire and become regular reminders that your company is annoying. Better to let them go.
On average across the newsletters we studied, the subscribe process took five minutes , and the unsubscribe process took three minutes. Even though these task times are not prohibitive, they are much too long for the simple functionality that's involved.
We recommend setting a usability goal of allowing an existing user to unsubscribe in less than one minute, assuming that the user has a recent copy of the newsletter at hand. New subscriptions should also take less than a minute for newsletters that don't require any information beyond the user's email address. Even if additional information is required, users should be able to subscribe to free newsletters in less than two minutes. Only newsletters that involve a subscription fee should be allowed so many steps to their subscription process that the average user can't subscribe in two minutes.
Users are very demanding with respect to the efficiency of operations like subscribing or unsubscribing. For both tasks, we found extremely strong correlations between the task time and the users' subjective satisfaction: r = -.71 and -.96, respectively.
These correlations basically say that the slower the subscribe or unsubscribe process, the less people will like the site. For each additional minute it takes to subscribe, you will lose 0.3 satisfaction points on a one to seven scale, and for each additional minute it takes to unsubscribe, you will lose 0.6 satisfaction points. As indicated by the numbers, users are substantially more critical of slow unsubscribe processes . Once you want out, you want out quickly.
A perfect satisfaction rating of 7 would require instantaneous task performance according to the regression estimates. It seems impossible to create a design that allows users to subscribe and unsubscribe in zero seconds, but that's ultimately what users want. It's nobody's goal in life to "manage your subscriptions," so any overhead becomes an annoyance. Ultimate simplicity and ease of use are necessary to make a positive impact on customers.
Extreme Platform Diversity
The Web is getting to feel like a science-fictional World of Clones with virtually no biodiversity, as almost all users have the same browser these days. It is still good practice to check that Web designs work on additional platforms beyond Internet Explorer and Windows, but the vast majority of users do use this combination. In any case, from an interaction design perspective, Netscape is a pale imitation of IE, and Macintosh is a higher-priced dolled-up variant of Windows. The differences between Web browsing platforms are like those between Indian and African elephants and not like the differences between crabs and eagles.
In contrast, email newsletters have to contend with platform diversity that is much more like the biodiversity of the Cretaceous Period (before the comet hit). Our test users were almost evenly distributed between AOL, Hotmail, Netscape mail, Outlook, and Yahoo mail, and it is also common to find people using Eudora, Lotus Notes, and a broad variety of mainframe systems and variants of Unix mail.
Each email platform has a different way of displaying the From line, the Subject line, and the content of a newsletter, as well as different approaches to spam filtering and other ways of influencing the user experience of the newsletter subscriber. This diversity makes it very important for newsletter designers to consider the many different platforms and test their subscribe and unsubscribe processes as well as the delivery and display of the actual newsletter on all major email platforms.
Newsletters Must be Simple
People get a lot of email. They don't have time to read a lot of text. Newsletters must be designed to facilitate scanning . In our study, only 23% of the newsletters were read thoroughly . The remaining newsletters were skimmed, read partly, or not even opened -- a fate that befell 27% of the newsletters.
The only newsletter in the study that was consistently read every time it was received was Dictionary.com's Word of the Day , which is very short and also has an engaging layout that doesn't intimidate users with a wall of text.
In subjective comments, users basically said that newsletters are bad if they take too much time or demand too much work of the user. Newsletters are good if they cut down the time it takes users to accomplish something or if they are quick reads that do not feel frivolous.
Users will often avoid signing up for newsletters because they feel crushed by information overload. It is the job of the newsletter publisher to convince users that the newsletter will be simple, useful, and easy to deal with.
A predictable publication frequency that is not too aggressive is usually best, except for newsletters that report breaking news. Not only are users more likely to sign up for newsletters that feel less intimidating, but a regular publication schedule lets users know when to look for the newsletter and reduces the probability that it will be deleted because it is confused with spam.
Writing good subject lines is especially important, both to encourage users to open the newsletter and to distinguish the newsletter from spam. We recommend including some actual content from the individual issue in each subject line , even though it's a difficult job to write good microcontent within the 50-60 character limit that is imposed by many email services.
The full report on email newsletter usability with 221 design guidelines for newsletters is available for download.