Usability Process Challenges in a Web Product Cycle

Gayna Williams

Microsoft Corporation

One Microsoft Way

Redmond, WA 98052

gaynaw@microsoft.com

ABSTRACT

Improved accessibility to the Internet by product teams and end users is changing the product development cycle and the tools to support it. The changes create opportunities for usability engineers and others engaged in user-centered design, but introduces new workload and scheduling problems.

KEYWORDS:

Internet time, product development cycle, usability engineering

INTRODUCTION

Microsoft has six usability groups and over 80 usability engineers, each group reporting into a product development division. As elsewhere, usability engineers choose from a diverse set of established techniques based on a team's experience with methods, the product being produced, the phase of the project, and the individuals involved in the process. The timing of activities depends on windows of opportunity for influencing development.

The development of products in "Internet time"-very compressed development and testing schedules-dramatically changes this process. The use of the Internet to release beta versions, the existence of Usenet newsgroups devoted to supporting specific products, the World Wide Web (WWW) and release of products partly written in HTML, and intranetsin software corporations greatly increase information exchange. These present tremendous opportunities for feedback and communication, but can also conflict with existing usability procedures practices. This paper describes such challenges and opportunities, focusing on experiences with Internet Explorer (IE).

BETA RELEASES

The WWW enables product teams to use the Internet to distribute beta versions to an international audience of eager end users. The principal goal may be "bug bashing," but usability feedback is also possible, on a massive scale and in a time frame in development when human-computer interface changes can still be made. In addition, competitor's beta releases facilitate comparison testing. However, managing this great opportunity effectively is a considerable challenge, for the following reasons.

Web beta releases add to the workload for usability engineers, who now are asked to determine whether users can locate the beta on the web site, understand the download requirements, download it, and find technical support information. In addition, where a full field study was formerly done only at product release, it is now sought after each beta release. Moreover, trade publications now comment on usability in published reviews of beta releases, so usability engineers are under pressure not to wait for feedback from beta users. To manage field tests of each beta, we have trained IE team members in field study techniques, so that they can help conduct and utilize the studies with minimal lead time.

Beta testers do not necessarily understand how to provide useful usability feedback. A new group was formed within the IE product team to manage beta releases and elicit feedback. Educating beta testers on how to report usability problems and accurately capturing, quantifying, and communicating to teams the rich information from this extensive, but not entirely representative, set of users are two more new tasks for usability engineers.

HTML: THE PROTOTYPE IS THE CODE

Iterative testing has utilized prototypes up to the code completion deadline, after which there is no hope of change until the next version. In previous product cycles the prototype was the specification (Sullivan, 1996) but now the prototype is often the code: Parts of the interface are developed in HTML, enabling non-developers to make changes right up to product release. This improves the ease of iterative testing and change. This means testing is needed to confirm that changes are in fact improvements right up to the "eleventh hour". Another challenge is to manage this activity alongside other activities that traditionally filled the schedule after code completion, such as field studies.

NEWSGROUPS AND INTRANETS: USEFUL, CHALLENGING The Internet provides developers (as well as usability engineers) with an ever-open window on the "real world," rather than periodic glimpses through marketing feedback, printed product reviews, and usability tests. Special newsgroups exist to support products-"Read the newsgroups" is often a mandate from product management. The challenge for usability engineers is to ensure that information from newsgroups is used appropriately.Newsgroups attract certain types of users, whose concerns may not be widespread. A developer may make changes based on what a single "real user" wants, while neglecting more significant problems reported by usability engineers.

These pressures interact. In responding to feedback from newsgroups and external customers, some IE 4.0 feature components were being changed at the time of the beta 2 release and required usability evaluation work, work that was required at a time when a field study had been planned. The field study was canceled to iterate on the interface.

In addition to the external Internet, the Microsoft intranet has improved product development dramatically, enabling rapid dissemination of schedules, specifications, daily versions of the product called "builds," vision statements, and remote viewing of usability tests. Managing the coordination of this information remains a challenge for usa bility and the product teams.

REFERENCES

Sullivan, K (1996). The Windows 95 User Interface: A Case Study in Usability

Engineering, in Proceedings of CHI '96 (Vancouver, Canada), ACM Press,

473-480.