jameshom.com | Usability Methods Toolbox | Formal Usability Inspections

Formal Usability Inspections

What is it?

Formal Usability Inspection takes the software inspection methodology and adapts it to usability evaluation. Software inspections, more commonly known as code inspections, started at IBM as a way to formalize the discovery and recording of software problems ("defects" in quality jargon, "bugs" in the vernacular). The technique also provided quantitative measurements that could be tracked using statistical process control methods. Code inspections were also adapted to check and track documentation defects, and usability defects were a logical next step.

Formal usability inspections include aspects of other inspection methods too. Heuristics are used to help non-usability professionals find usability defects. Inspectors walkthrough tasks with the user's goals and purpose in mind, similar to cognitive walkthroughs, although the emphasis is less on cognitive theory and more on encountering defects.

How do I do it?

This method formalizes the review of a specification or early prototype. The basic steps are to assemble a team of four to eight inspectors, assign each a special role in the context of the inspection, distribute the design documents to be inspected and instructions, have the inspectors go off on their own to do their inspection, and convene later in a formal inspection meeting. Defects found are assigned to responsible parties to be fixed, and the cycle continues.

Assemble the team. Pick a team of interested people, that is, people that have a stake in making the design more usable. This usually includes engineers from the design, quality assurance, documentation, training, and technical support groups. Each person brings a diverse viewpoint to look at the design, and the potential to discover usability defects is greater with a diverse team.

Assign roles. The formal usability inspection methodology borrows the inspection roles concept from code inspections. Each person on the team, besides having to inspect the design, has a role to play during the formal meeting. These roles are the following:

Moderator: Runs the meeting. Distributes and collects any materials needed. Schedules meetings, and coordinates defect assignment.

Owner: Designer of the product to be inspected. Usually the person to which defects are assigned. Fixes the defects.

Recorder (sometimes called Scribe): Logs defects during the formal meeting.

Inspectors: Everybody else. Inspects the design and reports any defects found. Everyone's an inspector regardless of their other role.

Distribute documents. For code inspections, this would be a code listing with line numbers plus instructions on what to look for--bad choice of syntax, variable problems, etc. For usability inspections, these include descriptions of the product, including screen mockups if any, user profiles, typical user tasks, heuristics to use, and a defect logging form.

Inspect the design. The inspectors work alone through the design and log the defects they find on the provided form. Having a form with an agreed-upon format for logging helps later during the formal meeting when the defects are discussed with the other inspectors. Each inspector assumes the role of a specific user from the user profile and walks through the tasks of a particular scenario. Prior to inspection, each inspector should review the heuristics and keep them in mind during their solo inspection sessions. Sometimes the form can be adapted to incorporate the heuristics as a checklist. Defects are logged according to the task the inspector was trying to execute and the location of the defect. With code inspections, the defect is located by line number--however, line numbers aren't usually present in interfaces. Defect location can be given as the screen and field or control name, or by the command and option attempted.

Hold the formal meeting. During the meeting, the moderator walks the team through each task/scenario as a group. Inspectors chime in at each step with the defects they found during their own inspection. Often, a lot of new defects are found as the inspectors discuss each defect--different aspects one inspector might not have thought of are brought up during the meeting. Everybody agrees on the recorder's logging of the defect--this formal log will be tracked later.

Inspectors might be tempted to think up solutions during the meeting, or the owner might take umbrage at the pronounced defects and protest each entry. These delays make the meeting run less smoothly and hurt the method's chance of success. Part of the mediator's role is to reduce these distractions so the defects can be agreggated and logged. There'll be plenty of time to fix them later.

Prioritize and fix the defects. Defects logged during the meeting are assigned to responsible persons to be fixed. The moderator often coordinates this effort, tracking fixed and open defects, and arranging solution-brainstorming meetings if necessary.

When should I use this technique?

Like other inspection methods, this technique is designed to reduce the time required to discover defects in a tight product design cycle. Since the inspectors can work with merely a specification or paper mockups, the technique lends itself well to early stages of development.

Who can tell me more?

Click on any of the following links for more information:

Kahn, Michael, and Prail, Amanda, "Formal Usability Inspections," in Nielsen, Jakob, and Mack, R. eds, Usability Inspection Methods, 1994, John Wiley and Sons, New York, NY. ISBN 0-471-01877-5 (hardcover)

Code Inspections

Freedman, Daniel, and Weinberg, Gerald M, 1990, Handbook of Walkthroughs, Inspections, and Technical Reviews : Evaluating Programs, Projects, and Products, Dorset House, ISBN: 0932633196

Gilb, Tom, Graham, Dorothy, and Finzi, Susannah, Software Inspection, 1993, Addison-Wesley Pub Co, ISBN: 0201631814

Wheeler, David A. (Ed.), Software Inspection : An Industry Best Practice, 1996, IEEE Computer Society, ISBN: 0818673400