Usability Study

Conducting a qualitative usability study for a major online retailer

Challenge

Our client, REI, had recently redesigned its online gift card purchase experience in an effort to simplify the selection process, reduce the need for users to seek online or telephone assistance, and to increase conversion. We worked for the REI gift card team to help assess any usability issues with the revised gift card purchase experience. The client was specifically interested in measuring the discoverability of the gift card page and assessing whether users are able to navigate the gift card purchasing process without resorting to customer assistance. Secondary goals for the research were to learn about user reactions to the gift card purchase process and uncover suggestions to improve the overall experience. 

Process

Our team conducted an initial cognitive walkthrough to identify research questions for exploration. We then designed a usability study plan that included the following:

• Online recruiting screener survey to screen participants for our research criteria

• Pre-test questionnaire to qualify participants and to understand their online shopping and gift card buying behavior

• Usability test with 3 tasks to be conducted using Think Aloud Protocol and post-task Likert ratings

• Post-test questionnaire that covered feedback on the overall user experience

Our team recruited 9 participants to each complete a one hour usability test in the Lute Research Lab on the University of Washington's campus. During each test, the moderator led the participant through the test, while the notetaker watched a mirrored display to record behaviors and observations. In another room, two observers watched the sessions using Morae Observer and recorded quotes and observations on sticky notes placed on printouts of key screens. Our team shared responsibilities, with each member serving as moderator, notetaker and observer for at least 2 participants. 

Outcome

After reviewing the footage of the tests, our team collaboratively analyzed the data using affinity diagrams to help uncover patterns in user behavior. We identified six usability issues and gave them severity rankings based on their impact on the gift card shopping experience. For each of these issues we created video clips from the user testing sessions and identified representative quotes from our users. We shared our findings with REI in a written report that included an executive summary, study plan, data analysis and research findings, and recommendations for improving the user experience with annotated screenshots of the relevant pages. We also shared our research plan and results in a short presentation to fellow Master's students in the Human Centered Design and Engineering program. A copy of our final report is available here. 

Project Team: Tomo Matsumoto, Kat Brookshier, and Dorothy Wong