Jakob Nielsen's Alertbox, November 17, 2008:  

Agile Development Projects and Usability

Summary:
Agile methods aim to overcome usability barriers in traditional development, but pose new threats to user experience quality. By modifying Agile approaches, however, many companies have realized the benefits without the pain.

Depending on how they're handled, Rapid Application Development (RAD) processes such as Agile and Scrum can enhance or threaten user experience quality.

The Promise of Agile Methods

Agile methods address three issues that have long vexed usability professionals:

• For 50 years, almost all experiences have shown that traditional waterfall development methods result in a poor user experience. The reason is simple: requirement specifications are always wrong.
  • At best — when derived with care — the requirements might reflect what users want. More commonly, however, they reflect the desires of user "representatives" who are too far removed from the coalface to know the details of the real work. In any case, what users want and what users need are two different things, which is why it's long been a primary usability guideline to watch what users do, rather than listen to what they say.
  • If years go by between writing the requirements and delivering the product, the users' needs will have likely changed, putting even more distance between the requirements and the needs.
  • Over the past 25 years, work in usability has shown that one of the best ways to evaluate a design's quality is by watching users interact with it (through either a functional or mocked-up screen). Again, if years go by before the developers do this, most of their development effort will have been spent producing the wrong design.
• Documents further down the waterfall also cause problems. The only thing worse than having developers deviate from the design specs is having developers implement the design specs to the letter. Many issues arise during an interaction design's detailed implementation; developers can't resolve these issues in ways that enhance usability when the design work was completed long ago and the user experience professionals have long since left the building.
• Since 1989, the "discount usability engineering" movement has demonstrated that fast and cheap usability methods are the best way to increase user experience quality because developers can use them frequently throughout the development process. However, this doesn't happen if there's only one milestone in the process that calls for usability input.

Agile methods hold promise for addressing the many ways in which traditional development methodologies erect systematic barriers to good usability practice.

The Threat of Agile Methods

Agile's biggest threat to system quality stems from the fact that it's a method proposed by programmers and mainly addresses the implementation side of system development. As a result, it often overlooks interaction design and usability, which are left to happen as a side effect of the coding. This, of course, contradicts all experience of the last 30 years, in which user experience's importance in system development has steadily increased as we moved from mainframes to PCs to the Web. As the user base and the use cases have expanded, the need for top-notch usability has grown.

To construct a quality user experience, development teams need interaction design and usability methods. For smaller teams, this doesn't necessarily require dedicated designers and usability professionals. It's perfectly feasible for developers to do interaction design and usability. But a team must recognize these two activities as explicit development methodology components, whether the people doing them have design or usability as their main job or simply as one of several roles they perform.

For a project to take interaction design and usability seriously, it must assign them "story points" (i.e., resources) on an equal footing with the coding.

Another issue is that, with Agile, a product's development is broken down into smaller parts that are completed one at a time. Such an approach risks undermining the concept of an integrated total user experience, where the different features work consistently and help users build a coherent conceptual model of the system. At worst, the user interface can end up resembling a patchwork.

To address this, teams can design storyboards and prototypes that embed the user interface architecture and use these tools as reference points for designing individual features. To avoid spending too much time up front, teams can design low-fidelity prototypes — such as paper prototypes — that don't require coding. Just like we've always advocated.

Agile teams typically build features during fairly brief "sprints" that usually last around 3 weeks. With such tight deadlines, developers might bypass usability because they assume there's no time to do testing or other user research.

The solutions here are threefold:

• Perform usability activities, such as user testing, in a few days. One fruitful approach is to plan for testing before you know exactly what will be available for testing. Weekly tests are completely feasible and give you a surefire way to integrate several user feedback rounds within even the shortest sprint.
  • We have a 3-day course on how to perform a complete round of user testing by actually testing the team on its own project. You can do this type of quick testing in a day. And, it takes less than a day to both prepare the test and analyze its findings.
• Most successful teams have adopted a parallel track approach, where the user experience work is continuously done one step ahead of the implementation work. So, by the time such teams start to develop a feature, the initial user experience work on it has just completed.
• Finally, we need foundational user research that goes beyond feature development. Ideally, organizations should conduct this work before a development project even starts. Also, bigger companies should house basic knowledge about user work flows, personas, and usability guidelines outside individual projects so it can be reused for years across many projects.

Making Agile and Usability Work

There are good reasons to believe that usability and Agile development methods can work together and improve user experience quality:

• Agile offers many opportunities for overcoming problems with traditional development methods that have long impeded usability.
• Approaching Agile narrowly, as a programming methodology rather than a system development methodology, threatens to destroy the last decade's progress in integrating usability and development. But, as outlined above, there are ways around each of these threats. So long as teams recognize the threats as explicit issues, they need not harm product quality.
• Finally, we know from our research that many companies have made things work swimmingly — once they adapted the Agile methodology to suit quality-focused system development.

This assessment of the opportunities, threats, and empirical facts about what works is based on two rounds of research:

1. A survey of 105 design and development professionals.
2. In-depth case studies on Agile projects at 12 companies.

Although the data shows much promise, we also found several cases of poor outcomes, emphasizing the need for companies to take explicit steps to integrate Agile and usability.

For user experience practitioners who support Agile teams, the main change is in mindset. Having good, general user experience knowledge will help you understand how to change traditional design and evaluation methods to meet your Agile team's different focus. Ultimately, however, you must both believe in yourself and embrace Agile development concepts if you want to succeed.

If you're prepared to change your practices and take on the responsibility, there are great opportunities to improve your effectiveness and your impact on the teams you support.

Learn More

119-page report on Best Practices for User Experience on Agile Development Projects is available for download. (Note that the current report is the second edition and is based on additional research beyond the work summarized in this article. However, the main findings remain the same. It certainly increases our confidence in the results and recommendations when they are confirmed by two rounds of research.)

Full-day course on Agile Development and Usability in San Francisco April 2012.

2-day course on Application Usability and a 3-day course on Designing Complex Applications and Websites at the annual Usability Week conference. (Topics differ by city, so check your preferred location's agenda for an exact list of seminars.)