Usability studies show how real people will use a product, software or web site in the real world.

The goals of usability testing vary by study, but they usually include:

• Identifying problems in the design of the product or service

• Uncovering opportunities to improve

• Learning about the target user’s behavior and preferences

Open

Open

Why do we need to do usability testing? Won’t a good professional UX designer know how to design a great user interface? Even the best UX designers can’t design a perfect — or even good enough — user experience without iterative design driven by observations of real users and of their interactions with the design.

There are many variables in designing a modern user interface and there are even more variables in the human brain. The total number of combinations is huge. The only way to get UX design right is to test it.

Elements of Usability Testing

There are many different types of usability testing, but the core elements in most usability tests are the facilitator, the tasks, and the participant.

Open

The facilitator administers tasks to the participant. As the participant performs these tasks, the facilitator observes the participant’s behavior and listens for feedback. The facilitator may also ask followup questions to elicit detail from the participant.

Facilitator

The facilitator guides the participant through the test process. She gives instructions, answers the participant’s questions, and asks followup questions.

The facilitator works to ensure that the test results in high-quality, valid data, without accidentally influencing the participant’s behavior. Achieving this balance is difficult and requires training.

(In one form of remote usability testing, called remote unmoderated testing, an application may perform some of the facilitator’s roles.)

Tasks

The tasks in a usability test are realistic activities that the participant might perform in real life. They can be very specific or very open-ended, depending on the research questions and the type of usability testing.

Examples of tasks from real usability studies:

• Your printer is showing “Error 5200”. How can you get rid of the error message?

• You're considering opening a new credit card with Wells Fargo. Please visit http://wellsfargo.com and decide which credit card you might want to open, if any.

• You’ve been told you need to speak to Tyler Smith from the Project Management department. Use the intranet to find out where they are located. Tell the researcher your answer.

Task wording is very important in usability testing. Small errors in the phrasing of a task can cause the participant to misunderstand what they’re asked to do or can influence how participants perform the task (a psychological phenomenon called priming).

Task instructions can be delivered to the participant verbally (the facilitator might read them) or can be handed to a participant written on task sheets. We often ask participants to read the task instructions out loud. This helps ensure that the participant reads the instructions completely, and helps the researchers with their notetaking, because they always know which task the user is performing.

Participant

The participant should be a realistic user of the product or service being studied. That might mean that the user is already using the product or service in real life. Alternatively, in some cases, the participant might just have a similar background to the target user group, or might have the same needs, even if he isn’t already a user of the product.

Participants are often asked to think out loud during usability testing (called the “think-aloud method”). The facilitator might ask the participants to narrate their actions and thoughts as they perform tasks. The goal of this approach is to understand participants’ behaviors, goals, thoughts, and motivations.

Remote Usability Testing

Remote usability sessions don’t require either the participant or the facilitator to travel. As such, remote testing is a great solution for teams with limited budget, or for testing products whose users are geographically dispersed. Scheduling a series of online studies can be preferable and far less costly than traveling around the country or the world.

Online testing is also a good solution in a tight timeframe — travel doesn’t have to be coordinated, and facilities for testing don’t have to be secured. Further, participants can be from any geographic area rather than concentrated in one location, which can make recruiting faster and easier.

Remote research allows participants to use their own computers for the study, letting you and your team see how they set up their desktop, navigate between programs, and use tabs, for instance. This insight into how people work with their own machines is valuable, but it also makes it more difficult to troubleshoot any problems participants have with the remote tools needed to conduct the study.

In moderated remote testing, users and facilitators are in the same “virtual” space at the same time — the facilitator is watching the usability test remotely as it happens, and communicating directly with the participant via the telephone, email, chat, or a combination of methods. In an unmoderated remote session, the participant completes the study on his or her own schedule, recording the session for later review by the usability expert.

Moderated Studies

Moderated sessions allow for back and forth between the participant and facilitator, because both are online simultaneously. Facilitators can ask questions for clarification or dive into issues through additional questions after tasks are completed.

It can be difficult to know when to ask a question in a remote study, however. Silence on the other end of the line may mean that the user is confused, immersed in content, looking around the page, or distracted. It can be difficult to find the balance between letting users know you are listening and interrupting them. Although the same is true in face-to-face studies, the problem can be magnified in remote studies.

Unmoderated Studies

Unmoderated usability sessions are completed alone by the participant. Although there is no real-time interaction with the participant, some tools for remote testing allow predefined follow-up questions to be built into the study, to be shown after each task or at the end of the session. Questions can also be emailed to be completed after the user has finished her session. In both cases, questions are the same across users. There is no opportunity to ask detailed questions specific to the user’s actions.

Users don’t have real-time support if they have a question, need clarification, or can’t get the technology to work, although you can provide them with an email address or phone number to contact someone for assistance. This disconnect also means you don’t know what the session was like until it’s finished. If a user did run into a problem, skipped tasks, or failed to complete what was asked, you don’t know until it’s over. Some sessions may end up being unusable or less valuable, depending on the issue.

Unmoderated tests can also be quieter than moderated tests. We typically use the think-aloud protocol in usability testing, asking users to talk us through what they’re doing as they’re doing it. In a moderated study, the facilitator can gently nudge a quiet participant to share more about what he’s doing. In an unmoderated study, you can ask a user to think aloud, but there is no one there to remind her if she doesn’t do it.

Because of the lack of detailed follow-up, it is preferable to use unmoderated remote usability tests when the main focus of the study is a few specific elements, rather than an overall review. Remote studies are great for gathering data on an element or widget or for seeing the impact of a relatively minor change.

Unmoderated studies can also be good for tight timeframes: users can complete sessions on their own schedules and even simultaneously, rather than trying to fit into scheduled time slots.

Tips for Remote Studies

Practice the technology. Even if you’ve used your company’s tools a million times before, test them with someone you know outside the company, mocking up a real test situation. Make sure the instructions for signing in are clear. Practice sending URLs or tasks to your user and make sure you know how the technology works on your end — and theirs.

Rewrite everything. Write tasks far enough in advance to pilot-test them. In a moderated session, the facilitator can get a user back on track if a task is misunderstood. In an unmoderated session, there is no safety net. The written instructions need to stand on their own. Every instruction, task, and question needs to be fine-tuned to eliminate the potential for misunderstanding. As we know from every study ever done on instructional design, anything that can be misunderstood will be misunderstood.

Be available. Even for unmoderated studies, be available by email (if not by phone) as much as possible to help with any potential user questions. In moderated sessions, sign in to the testing tool early, in order to know when your participant arrives and to troubleshoot if needed.

Recruit more users. No-show rates for any remote study can be higher than for in-person studies. You also don’t know the quality of an unmoderated session until you’ve watched it. It’s better to add a few more users than you think you need in order to accommodate such problems.