Chapter 3
7 Essential usability testing methods for UX insights
To create intuitive products and exceptional user experiences, you need to understand how real people interact with your designs; meaning, you need to run usability tests.
In this chapter, we explore the types of usability testing, how usability tests are conducted, essential methods to know about, and what data they collect.
3 Types of usability testing formats
Before we dive into specific methods, it’s important to note that, like all UX research, individual usability testing methods fall within different types of research:
- Qualitative or quantitative
- Moderated or unmoderated
- Remove or in-person
While these aren’t methods as such, they are different formats for conducting usability testing, and will provide you with different types of data.
While we won’t be covering these in-depth now, you might also hear about formative vs. summative usability testing which is broadly early-stage (formative) usability testing to refine features, and pre-launch testing that evaluates (summative) a product’s overall.
You should also consider whether you’re conducting exploratory research (trying to understand something new) or comparative research (comparing two things) when planning usability studies.
These approaches often overlap with the types of usability testing above, which is why these are the types we'll be digging into here.
🤔 Expecting to hear about generative or evaluative UX research? Check out our detailed guides.
1. Quantitative vs. qualitative usability testing
Any kind of data collected during usability testing will be either quantitative or qualitative data.
- Quantitative data provides measurable, numerical key metrics about user interactions, such as task completion times, error rates, and user satisfaction ratings. This data looks at the ‘what’ to identify patterns and trends for further investigation.
- Qualitative data offers descriptive insights into user experiences, motivations, and emotions. It often provides the ‘why’ behind quantitative data’s ‘what’, and enables teams to dig deeper to understand user thoughts and feelings.
Neither type of data is empirically better, and most research benefits from both (mixed methods research). Think about them as different players on the same team—a shared goal of user insights, but different approaches to obtaining it.
The type of data you collect will depend a lot on the goal and hypothesis of your test. When you know what you want to test and you identify the goals of your test, you will know what type of data you need to collect.
Vaida Pakulyte
UX Lead at Technigo
Share
2. Moderated vs. unmoderated usability testing
Whether a usability study is moderated or unmoderated comes down to whether a researcher is present or not:
- In moderated usability testing, a researcher guides participant(s) through tasks, observes their interactions, and asks follow-up questions in real-time. This approach provides rich data and allows for immediate exploration of user behavior.
- On the other hand, unmoderated usability testing has participants complete tasks independently without direct interaction or guidance.
Both moderated and unmoderated research have their place—moderated research typically gathers more in-depth, rich data, while unmoderated usability testing allows for greater sample sizes and efficiency.
Unmoderated tests are useful for large-sample studies, such as quantitative testing, or when I need the results quicker.
Vaida Pakulyte
UX researcher and Designer at Electrolux
3. Remote vs. in-person usability testing
Whether you opt for remote vs. in-person usability testing is fairly self-explanatory, and typically comes down to your available resources and chosen UX research methods.
- Remote usability testing involves conducting research virtually—typically via video conferencing or a usability testing tool. Suitable for both moderated and unmoderated testing, remote testing is a more accessible and lower-cost option for conducting usability tests.
- In-person usability testing takes place in a controlled environment, such as a lab or the participant's workplace. Researchers can directly observe participants' behavior, including nonverbal cues and reactions. However, it’s more costly due to providing a suitable environment, participant reimbursement for travel, researcher time.
💡 Learn more
We look at remote vs. in-person usability testing in more detail in our chapter on Remote Usability Testing.
Now we’ve covered types of usability test, let’s dive into the most common usability testing methods to test your product and find out what works (and what doesn't).
1. Lab usability testing
In a lab usability test, participants are tasked with completing a set of tasks on a computer or mobile device under the supervision of a trained moderator. The moderator observes their interactions, asks questions, and replies to their feedback in real time.
As the name suggests, lab usability testing takes place in a purpose-built testing environment. Typically, this has two rooms divided by a one-way mirrored window to allow note-takers and observers to watch the test without being seen by the participants. Sessions can also be recorded for later review and in-depth analysis.
When to use lab usability testing: If you’re looking to get extensive verbal and observational feedback from participants
✅ Rich qualitative data for deep understanding of user experience ✅ Controlled environment for focused observation ✅ Direct interaction with participants for clarification ✅ Ability to record sessions for detailed analysis | ❌ High setup and execution costs ❌ Potential for cognitive biases to influence behavior due to lab setting ❌ Small number of test participants (5-10 participants per research round) limits generalizability ❌ Time-consuming process |
2. Contextual inquiry
The contextual inquiry method involves observing people in their natural contexts, such as their office or home, as they interact with the product the way they would usually do. The researcher watches how users perform their activities and asks them questions to understand the reasons behind those actions.
As the research takes place in the users’ natural environment, this method is an excellent way to get rich, reliable information about how users interact with products in their day-to-day lives.
These insights are essential at the beginning of a project when evaluating requirements, personas, features, architecture, and content strategy. However, you can also use this method after a product release to test the success and efficiency of your solution.
There are four main principles of contextual inquiries:
1. Context: Interviews are conducted in the user's natural environment
2. Partnership: The researcher should collaborate with the user, observe their actions, and discuss what they did and why
3. Interpretation: During the interview, the researcher should share interpretations and insights with the user, who may expand or correct the researcher's understanding
4. Focus: It's important to plan for the inquiry and have a clear understanding of the research goal
When to use contextual inquiries: When it’s critical that insights reflect an organic scenario in your users’ real-world circumstance
✅ Observe users in their natural environment for authentic behavior ✅ Identify hidden needs, pain points, and workarounds ✅ Applicable throughout the product life-cycle | ❌ Requires planning and travel ❌ Small sample size doesn’t represent your target audience accurately ❌ Can be disruptive to the user's workflow |
3. Guerrilla usability testing
Guerrilla usability testing is a quick and low-cost way of testing a product with real users. Instead of recruiting specific research participants, testers are approached in public places and asked to perform a quick usability test—often in exchange for a small gift, such as a coffee. The sessions usually last between 10 and 15 minutes and cover fewer tasks than other usability methods.
This usability testing method is useful for gathering quick feedback without investing too much time and resources—and works best at an early stage of the UX design process when you need to validate assumptions and identify core usability issues. However, if you need more fine-tuned user feedback, it’s better to complement your research with other usability testing methods.
When to use guerrilla testing: If you’re looking for a low-cost testing method to gather results quickly
✅ Low-cost method, requiring minimal financial investment ✅ Provides valuable insights from natural, real-world environments ✅ Can gather diverse opinions from random participants | ❌ May not yield high-quality feedback due to lack of participant interest ❌ Testing in public spaces can lead to interruptions and distractions ❌ Limited time with participants can result in shallow insights |
4. Phone/video user interview
Easily tag themes and categorize insights with Maze Interview Studies
A phone or video user interview is a remote usability test where a moderator instructs participants to complete specific tasks on their device and give verbal feedback on the product. All feedback is collected automatically as the user’s interactions are recorded remotely.
Phone and video interviews work best to collect user feedback from test participants scattered around different parts of the world. Another benefit is that they are less expensive and time-consuming than face-to-face interviews, allowing researchers to gather more information in less time.
When to use phone/video interviews: If you need to gather qualitative information from participants in different locations, in a short amount of time
✅ Reach a wider participant pool regardless of location ✅ Participants may feel more relaxed in their own environment ✅ Easily record sessions for later analysis and sharing ✅ Can often be scheduled and conducted more rapidly than in-person tests | ❌ Limited ability to observe nonverbal cues ❌ Issues with internet connection, audio quality, or video quality can disrupt the session ❌ Difficulty in conducting complex tasks that require physical interaction |
🗣️ Want to conduct video interviews for usability testing?
Maze Interview Studies streamlines the interview process with solutions for participant scheduling, video conferencing, interview analysis, and more.
5. Session recording
Session recordings are an efficient way to see exactly how users interact with your site. This method of website usability testing uses software to record the actions that real, anonymous visitors take on a webpage, including mouse movement, clicks, and scrolling. Session recording data can help you understand the most interesting features for your users, discover interaction problems, and see where they stumble or leave.
When to use session recordings: If you want to see how users naturally interact with and navigate your product
✅ Detailed analysis of user behavior ✅ Enables collaboration among team members ✅ Can be used for legal and compliance purposes | ❌ Can be time-consuming to review and analyze recordings without an adequate research tool |
🚧 Testing a live website? Maze Live Website Testing enables you to conduct usability tests on live websites without the hassle of a laboratory set up.
6. Tree Testing
Tree testing helps your researchers and designers assess content organization with users to optimize for the ideal information architecture. It involves instructing users to navigate to a specific page within an existing navigation system to assess how easily they can find it.
This gives insight into their mental models and enables designers to build information architecture that matches user expectations.
When to use tree testing: If you’re designing or restructuring a product’s navigation, content, or information architecture to align with user expectations
✅ Easily involve participants from different locations ✅ Automatically captures and visualizes data for fast interpretation | ❌ Doesn’t show how users interact with content in real-world tasks ❌ Participant approaches can vary, making analysis challenging |
✨ Product tip
Use Maze Tree Testing to get information on user mental models quicker than ever.
7. A/B usability testing
A/B testing, also known as split testing, involves assessing and comparing the usability of two designs in order to determine which is most effective and user-friendly. In A/B tests, two different groups are shown two different versions of the same product and asked to complete a set of tasks (the same across groups). This approach helps test aspects like content placement, information architecture, CTA specifics (like color, phrasing, placement, etc), and more.
Researchers look at the quantitative data—usability metrics like time spent on task, task success rate, heatmaps, and more—and qualitative data—comments and feedback from users during the test—to understand which of the two designs provides the best overall user experience.
This type of testing can also be used to assess the user interface and functionality of multiple designs—this is called multivariate testing (but it works the same!).
When to use A/B usability testing: If you’re comparing two (or more) designs to decide which to move forward with
✅ Answer specific design questions, like which color button works best ✅ Assess designs one step at a time ✅ Get clear, comparative data | ❌ Testing process takes lots of time and resources ❌ Only works in specific scenarios, can’t comprehensively test usability ❌ Data insights are only specific to the test in question and cannot be used elsewhere |
Moving forward
That’s it for our chapter on usability testing methods for your design and development process, head to our next chapters for in-depth breakdowns of the methods discussed above, including remote usability testing, live website testing, and guerrilla testing.
Test usability in hours, not days
Maze lets you quickly test and validate usability across your wireframes and prototypes with real users to get early feedback on your design process.
Frequently asked questions about usability testing methods
What is usability testing?
What is usability testing?
Usability testing is the process of evaluating how easy to use and intuitive a product is by testing it with real users. Usability testing usually involves getting participants to complete a list of tasks while observing and noting their interactions to identify usability issues and areas of improvement.
What are the benefits of usability testing?
What are the benefits of usability testing?
One of the benefits of usability testing is that you can fix usability issues before launching your product and make sure you create the best possible user experience. Ultimately, ensuring users can accomplish their goals smoothly will lead to long-term customer success..
In a nutshell, wireframe testing allows you to:
- Explore different early layout concepts
- Test your ideas rapidly and validate them with users
- Learn valuable information that you can use to guide the design of higher fidelity prototypes.
What are the different types of usability testing methods?
What are the different types of usability testing methods?
Quantitative or qualitative: quantitative testing measures users’ performance on a given task, such as the percentage of users who completed a given task. Qualitative testing involves observing users to understand how they experience your product and why they performed certain actions.
Moderated or unmoderated: in a moderated usability test, a moderator guides the participants through the test. In an unmoderated test, the participants receive instructions and tasks to complete beforehand, but there's no one present as they accomplish the tasks.
Remote or in-person: remote usability tests happen when the participant and the researcher are in separate locations. In-person tests take place in a testing lab or office under the guidance of a moderator.
Other types of usability testing methods include guerrilla testing, lab usability testing, contextual inquiry, phone interview, and session recordings.