Jakob Nielsen's Alertbox, November 4, 2009:  

Agile User Experience Projects

Summary:
Agile projects aren't yet fully user-driven, but new research shows that developers are actually more bullish on key user experience issues than UX people themselves.

Last year, we conducted a study of best practices in integrating usability methods with Agile development projects.

Usually, it's not worth studying the same problem again just a year later since user behavior doesn't change much. But this particular project didn't concern user behaviors, but rather the best way to run Agile projects to ensure usability.

Because this is still a new field, we decided to supplement last year's research with a new round of more detailed studies focused on additional organizations that have had more time to discover better ways to manage Agile user experience (UX).

UX: The Gatekeeper Role

The two main recommendations for ensuring good usability in Agile projects remain the same as in our original research:

• Separate design and development, and have the user interface team progress one step ahead of the implementation team. That way, when it comes time to build something, it's already been designed and tested. (And yes, you can do both in a week or two by using paper prototypes and discount user testing.)
• Maintain a coherent vision of the user interface architecture. Create the initial vision during a "sprint zero" period — before any implementation has started — and maintain it through annual (or semi-annual) design vision sprints. You can't just design individual features; they have to fit together into a coherent whole — a whole that must be designed as well. Bottom-up user interface design equals a confused total user experience (the Linux syndrome).

In both rounds of research, these two ideas proved useful across many of the different companies we studied. One modification became clear in the second round, prompted by the PayPal case study: it's important to designate a gatekeeper to track requirements and communications between the UX team and the other project teams to keep everybody on track (even though those tracks are parallel).

Circle D Design used a variation on this, designating an "anchor person" for each project's UX communication. As new UX specialists are needed on a project, they're paired with the anchor. The anchor principle also supports long-term cross-communication among teams as anchor people are periodically rotated.

Decline of the Centralized UX Department

The debate over how usability, interaction design, technical writing, and any other specialized discipline should be positioned on the org chart is endless. There are two main options:

• Centralized structures create a single team that "owns" its discipline and supplies it to development teams on projects across the organization. A centralized usability group, for example, would conduct all user testing and other user research; a centralized design team would supply all interaction design and visual design; while a centralized user experience team would supply all design and research. Individual project teams would then take the centralized team's design and/or research and turn it into an actual product.
• Distributed structures forgo centralization and instead assign specialized staff members to work directly on individual project teams. In this structure, each project team has its own usability specialist, interaction designer, visual designer, information architect, technical writer, and so on for all user experience disciplines. In fact, big projects might include several such specialists on the team.

An obvious disadvantage of a distributed structure is that companies might not have enough user experience specialists to assign one or more from each discipline to each project team. Thus, teams would either share specialists or some would go without.

The centralized structure provides a "home" for specialized staff members, who often appreciate having close colleagues within their own discipline. This makes it easier to manage and promote specialized staff. For example, the manager of a centralized usability group is typically a senior usability specialist, while the manager of a centralized design group is a senior designer, and the director of UX is an even more senior designer or usability person. Such managers understand their staff members' tasks and needs. (In contrast, a usability specialist who reports to a development manager will often find that the manager has no idea whether a study was run well or poorly.)

A centralized department can also support strategic initiatives that cut across individual development projects. Examples include writing and maintaining user interface standards or guidelines, building a usability lab, collecting longitudinal or comparative UX metrics, and advancing organizational maturity — for example, by communicating strategic UX issues to upper management and hopefully convincing them to invest more in usability.

All this is very well, but is problematic when it comes to integrating usability and good UX with Agile development teams. All experiences from our case studies indicate that UX people must be co-located with developers and other project team members. Indeed, UX should be considered a part of the project team, not an outside department.

Distributing your UX personnel doesn't mean you have to abandon all the benefits of having a centralized, specialized group. Often, a matrix structure provides a good compromise, making UX professionals part of individual projects on a day-to-day basis, but still offering some company-wide coordination.

Agile UX is Good, But Can Get Better

This year, we asked study participants how extensively UX was integrated into their projects and how satisfied they were working on projects with a particular development methodology. They indicated their answers on a 1–5 scale, with 5 indicating the highest level of integration or satisfaction:

Project Methodology	Integration of User Experience	Satisfaction with the Method
Waterfall	2.5	2.9
Agile	3.1	3.7
Iterative	3.2	3.8

Clearly, Agile is considerably better than the old Waterfall method. Good riddance to that one. However, the professionals in our new study still felt that Iterative Design was marginally better than Agile; there's still work to be done to make Agile projects more user-driven.

In better news, the latest data provides some evidence that we're moving beyond "them and us" — overall, developers were more bullish than UX people on a couple of key UX opinion metrics.

Developers rated the user experience impact on the deliverable's quality at 4.3, whereas UX people rated it at 4.0 (again on a 1–5 scale, with 5 best). Developers said productivity increased somewhat with UX involvement (3.3), whereas UX rated this only slightly higher at 3.4. Both developers and UX professionals expressed a strong desire for more UX involvement in the project. In those companies that bother to integrate usability and Agile, things are not yet perfect, but they're already pretty good.

Learn More

119-page report on Best Practices for User Experience on Agile Development Projects is available for download.

Full-day course on Agile Development and Usability in San Francisco April 2012.

2-day course on Application Usability and a 3-day course on Designing Complex Applications and Websites at the annual Usability Week conference. (Topics differ by city, so check your preferred location's agenda for an exact list of seminars.)