Chapter 3

How to run website usability testing: Benefits and best practices

To keep your site relevant, you need to continuously review usability and iterate based on user feedback, even after launching your product.

Website usability testing with a continuous mindset allows you to catch user interface and experience (UI/UX) mistakes, technical bugs, or navigational challenges even after your site is published.

Here’s our comprehensive guide to why, and how, you should run live website testing—plus our best practices for gathering actionable user feedback.

Test your live website effortlessly with Maze

Gather user feedback and improve your website’s usability in real time

Why is live website usability testing important?

Live website testing lets you evaluate the design, usability, and performance of your website. When you run usability tests on your website, you unlock a wealth of insight into how users experience the site in real time—from navigation and ease of use, to task completion, and design elements. This will help you keep your site relevant, retain users, and impress stakeholders at every level. By regular testing usability on your website, you can ensure your product is always serving users’ needs and delivering the experience they expect.

What are the benefits of website usability testing?

The biggest benefit of testing live websites for usability is the ability to react quickly, meet evolving user needs, and identify problems early on. By regularly assessing your site post-launch, you can be continuously informed by data. Testing your live website is also beneficial to:

  • Assess user experience and sentiment continuously
  • Test in real scenarios for realistic, contextual insights
  • Minimize costs by identifying issues early on
  • Build an exceptional reputation and increase customer satisfaction
Benefits of website usability testing

Testing your live website for usability is highly beneficial. You get to gather user insights continuously and use them to boost customer satisfaction.

Assess user experience and sentiment continuously

The adage goes that frustration is a matter of high expectation. However, user satisfaction is about meeting those expectations at every opportunity.

It’s your team’s job to truly get to know your users and identify their pain points, motivations, and needs. Putting your users at the center of your business means giving them a voice, and fueling your product decisions with that input. If you want to have a more robust and data-informed understanding of your users, you’ll find usability testing invaluable.

Test in real scenarios for realistic, contextual insights

The good thing about usability testing is that you can (and should) conduct it at any and all stages of the product development process.

With live website testing, you can expedite this process, safe in the knowledge that you’re able to continuously test, iterate, and launch, even once the website is in the hands of users. In fact, you’ll find your insights become more accessible, and far more relevant, when they’re coming from real users interacting with your live product.

Minimize costs by identifying issues early on

The cost of skipping usability testing has knock-on effects that can be felt through the product and product team. A poorly designed or difficult-to-navigate website can lead to:

  • Poor reviews and damaged reputation
  • User disappointment and frustration
  • High churn rate/low conversion rate
  • Time lost fixing preventable issues

Doing continuous usability testing allows you to mitigate risk as you can monitor north star and baseline metrics, and act quickly should any issues arise.

Usability testing a live website is imperative to identify problems and opportunities for product improvement, and to understand how users behave when using it in a real environment.

Mariana Anjos, Product Designer at Revelo

Build an exceptional reputation and increase customer satisfaction

Testing the overall usability of your designs allows you to cultivate a better user experience, and maintain a solid reputation. When your customers identify that you care and take action based on their opinions, change requests, and suggestions, they’re more likely to keep using your product. They might also be open to suggesting improvements, and eventually, recommending it to others.

How to conduct live usability testing on a website

You’re likely already experienced at running usability tests, but everyone can benefit from a standardized testing process. Here’s our tried-and-tested process to follow when performing usability tests on your live website:

1. Define your testing objectives

The goal of every usability test is to validate how easy it is to use your website. But when you run these types of tests, you need to identify more detailed goals—the data of which may be quantitative or qualitative in nature. For example:

  • Time to complete certain tasks (quantitative)
  • Abandonment rate (quantitative)
  • Number of alternative paths (quantitative)
  • Most clicked button (quantitative)
  • Answers regarding pain points of frustrations (qualitative)
  • How easy or difficult users found the website to understand (qualitative)

2. Determine the usability testing methods

As part of your usability testing plan, once you have a clear outline of your goals, you neeed to choose the right testing method for your needs (we go into this more in chapter five ). Every method will fall under these types of research:

  • Moderated or unmoderated: Choose whether or not you need the testing session to have a moderator. Some researchers prefer a moderator to ask follow-up questions in real time, while others opt for the freedom of unmoderated testing, where participants don’t feel they’re being watched.
  • Qualitative or quantitative: This refers to the type of answers you’ll get from the test. These can be sentiment-focused (qualitative) like quotes or facial expressions from users, or they may be data-focused (quantitative), such as number of clicks, or time to perform a task.
  • Remote or in-person: It’s common for website tests to happen remotely as you can record the user’s screen and monitor clicks. You can, however, also host them in person, if you have a product that requires in-person testing.

Pro tip ✨

Running unmoderated tests is usually easier and less expensive, as people can test from anywhere at a time that suits them, whereas moderated tests require multiple people’s calendars to align so they can conduct the test together.

3. Pick a tool or location

The next step is to choose a usability testing tool that suits your research and product team needs. If you’re testing remotely and want a mix of qualitative and quantitative insights, you can’t go wrong with Maze (even if we say so ourselves).

If you’d rather host moderated and remote usability testing sessions, you can try video conferencing apps like Google Meet or Zoom, and record the sessions. Finally, if you’re hosting tests in person, you can select a relevant location, depending on your needs and participant locations.

4. Recruit participants

Recruiting test participants who are willing to test your product can be challenging. But to gather valuable insights, it’s crucial you get this step right—and if you’re testing frequently, your bank of participants will always be growing.

If you’re using Maze for running remote and unmoderated tests, you can add In-Product Prompts to share mazes directly on your live site. That way, website visitors can take a few minutes to go through your test while they browse. You can also keep track of your users database in Maze’s Reach, or access our participant panels.

Maze in-app test prompts

Here are some ways to recruit participants easily:

  1. Leverage your current users
  2. Recruit people you know
  3. Ask on social media
  4. Make use of a recruitment platform
  5. Use your testing tool’s database

Screening your users

However you recruited participants, you’ll need to screen them and ensure you have a clearly-defined user persona before you start running screeners. You can do that by:

  1. Coming up with a pre-screening survey: Ask about demographics, experience, and any relevant information to your test—e.g. if they’ve ever done a test for you before
  2. Run a pilot test or group interview: Get a small group of users to jump on a call with you before the official usability test, and assess the kind of answers they give you. This is mostly relevant to assess the type of answers you’ll get based on how you phrased each question.

It’s worth keeping track of your participants and categorizing them, so you can easily invite them to future tests.

Product tip ✨

With our database, Reach, you can segment test participants based on type or responsiveness. You can also send your mazes through personalized emails directly from Maze, which helps boost your response rate and get real eyes on your tests fast. Learn more about Reach.

5. Create test scenarios

Create usability test scenarios that are tightly linked to the test goals. These should be real use cases that your users will try to achieve. For example, if you have an e-commerce platform, some scenarios you could try include:

  • Add a pair of green size 2 socks to your cart
  • Buy two size M white t-shirts using a gift card
  • Request a refund because your package never arrived

Share as much detail as you can, but don’t solve the task for them. This means adding accurate descriptions or gift card codes, not writing: “Go to the accessories tab and click on the first product in the fourth row.”

If you’re running moderated qualitative tests, you should also ask people about how they’d do certain tasks, and what they found frustrating or difficult. Make sure these are open-ended questions.

6. Build tests

It’s now time to actually configure the test your users will see. This will vary depending on the platform you’re using and the type of test you’re running. You need to come up with a usability test script and include all the scenarios or questions listed in the previous step.

Running usability tests on a live website with Maze? Check out our Help Center article, or watch this short video explaining how to set up your test:

Pro tip 💡

Test the test itself with a pilot run—ask someone who's unfamiliar with the product to evaluate whether your instructions are clear and easy to follow.

7. Review the results

Once you’ve run your tests, you need to compile your answers and build a report. You should include all the different issues you encountered during research and the various data points that back up those claims.

If you’re using Maze or another research platform, you can download an automatically-generated report with all your test data. Share the PDF with stakeholders, without having to manually analyze the data and design a presentation.

If you’re not using Maze for automatic reporting, here are a few important things to include in your unmoderated research report:

  1. A reminder of the test goals
  2. A brief summary of the type of test you ran, the number of participants, and the methodology you used
  3. List and prioritize found usability issues
  4. Add graphs, stats, and heatmaps per issue
  5. List all key takeaways and next steps

If you’re running moderated tests on video conferencing platforms or other tools, make sure your report highlights:

  1. A brief reminder of the test goals
  2. Most repeated pain points or verbatims
  3. Relevant quantitative data, e.g. time spent completing a task
  4. Key takeaways
  5. A list of all the questions and answers for the ones that need that level of detail

8. Implement changes based on your insights

Once you’ve reviewed your test results, you can make decisions and implement changes on your website. Now’s the time to come up with a plan that acts upon the findings:

  1. List all the action steps
  2. Prioritize them—you can’t do everything, everywhere, all at once
  3. Determine what you’ll be testing before launch
  4. Define the type of test you’ll be running, e.g. A/B testing or usability testing
  5. Request changes from the design team
  6. Test them
  7. Have your product team make adjustments
  8. Test again
  9. Push to production
  10. Test one more time
  11. Keep iterating

9. Make a habit out of testing

Testing your ideas, designs, prototypes, or live websites isn’t a one-off effort. It’s something that should be ongoing.

We know it’s not always easy to adopt a continuous discovery mindset; you may have limited resources or still be on your journey to a user-centered culture. So, here are a few common reasons why companies disregard testing—and how you can combat them.

Reason not to test Workarounds
Testing makes the production cycle longer

1. Low budget/time-consuming: Get involved in online communities and build a pipeline of pre-screened users that you can contact at any time

2. High budget/not time-consuming: Hire a company to recruit users for you and contact them as you need 

3. Mid-budget/not time-consuming: Utilize a tool like Maze for building your tests, have a segmented list of previously recruited users, or send mazes to a handpicked-for-you user base

You think the designs look simple and easy-to-use Validate your hypothesis: While you may think the product is easy-to-use and intuitive, other people may not. Subconscious cognitive biases can make it easy to fall into the trap of assuming others will use a product, interpret language, and understand things in the same way as us—but this isn’t the case. Usability testing is the only definitive way to learn if your product is as intuitive to others as it is to you.
It’s too expensive Use free versions of user research tools and recruit users by hand. Try inviting your existing user pool or power users, and ask them to jump on a call with you using Google Meet, or Zoom. You might need to test fewer people, but you can still get some real feedback. Some discovery and research tools, like Maze, also offer free user plans.
It’s overwhelming Don’t overthink it. Usability testing your website can be as easy as reaching out to users and asking them to perform some tasks. Start with unmoderated tests and use a tool like Maze that guides you every step of the way to simplify the process. You can also use pre-built testing templates to minimize the work needed, and get an automatically generated report to share with stakeholders.

4 Best practices for testing your website usability

Testing your live website for usability is a best practice in itself, but there’s some other tips that are worth bearing in mind as well. To get the most accurate insights from your live website test, follow these recommendations:

1. Make sure your scenarios are realistic and easy to understand

Avoid using technical language in your test scenarios—make it easy for your participants to understand and follow what you’re asking. For example, instead of saying “Create a user account using any form of external authentication (not your email),” say “Sign up using your Facebook or LinkedIn account.”

Also, ensure you’re asking for something that’s actually possible. If you’re only inviting them to sign up using Facebook, but they don’t have a Facebook account, you’re just going to frustrate participants and limit your insights.

2. Encourage ‘thinking out loud’

Invite your testers to speak aloud as they’re going through the test. Then, you can listen back to the recording and spot exactly what their thought process was to get to the solution. For example, you may hear a user say: “I couldn’t find the sign-up button on the home page, but I’ve noticed that sometimes there’s a link to create a new account on the log-in page. So, I’m clicking on the log-in page to see if I can find it there.”

You’ll get richer insights from this, combined with quantitative data like clicks and heatmaps, than you would from the data alone.

Maze Clips functionality

Product tip ✨

Use Maze's Clips feature to record your tester's screen and video to collect qualitative insights about your product. Then, review these clips, along with the solutions to the tasks, asynchronously. Learn more about Clips.

3. Test early and often

A few years ago, Mariana Anjos, now Product Designer at Revelo, worked at a company where they had to launch an app with no time for usability tests.

“We launched the application with significant technical issues,” she shared. “The app presented problems in the interface and flow, and had copy inconsistencies. During launch week, the number of negative reviews was so high that we had to hold back and do a relaunch two weeks later. It cost the company several thousand dollars in addition to the loss of initial users who were completely dissatisfied.”

You never know what the real cost of pushing untested products to production is until you risk it. In these cases, testing early and often is the only way to ensure you’re safe rather than sorry.

4. Use a variety of testing methods

You should try a mix of test types. While both quantitative and qualitative insights offer value, the best value comes from a combination of the two. One type of data provides context for the other. For example, ask questions to humanize your quantitative data and truly understand your audience’s thinking process, or analyze test statistics to give numerical weight to qualitative feedback.

Final tips to excel at live website usability testing

You might think that skipping usability testing is a way to save time and money. However, it’s quite the opposite. When you push features to production, you need to be in a position to catch issues or inconsistencies early on, and later down the line as problems pop up. Testing regularly with a live site is the only way to ensure your website is always performing as needed, and serving a truly user-centered experience.

To simplify your live website usability tests, you should try an easy-to-use platform like Maze to help you run unmoderated sessions. Maze is a cost-effective solution that allows you to reach out to a pool of segmented users and get fast responses on your live website or browser-based product.

Check out the next chapter of this guide to keep learning about creating a usability testing plan to keep your research organized and insightful.

Test your live website effortlessly with Maze

Gather user feedback and improve your website’s usability in real time

Frequently asked questions about website usability testing

What is website usability testing?

Website usability testing is a way to assess how an audience is using and navigating through your web pages. You can test websites before launch using a Figma design or after launch by getting users to complete tasks on the live page.

How do you test a website for usability?

To test websites for usability, you need to:

  1. Define your goals
  2. Choose testing methods (moderated or unmoderated, remote or in-person, and qualitative or quantitative)
  3. Pick a testing tool or physical location
  4. Recruit users
  5. Write test scenarios
  6. Configure your tests
  7. Analyze test results
  8. Make changes based on insights
  9. Continue to test regularly after launch

What are tools for usability testing?

There are multiple tools for usability testing, the most popular include:

1. Maze: Maze is the best option for conducting unmoderated usability tests, including running usability tests on live websites or apps to gather post-launch insights. Plus, Maze offers pre-built testing templates to save time, and automatically processes your test results for a downloadable report you can share with stakeholders
2. UserTesting: UserTesting is one of the market’s largest tools for running moderated usability tests where you can see, hear, and interact with testers.
3. Google Meet or Zoom: All video conferencing tools can be tools for hosting qualitative moderated usability tests, and is a great free alternative to an official research tool, if you’re just getting started with usability testing.

Which tool is most useful when you want usability feedback?

The most useful tool to get usability feedback is Maze, thanks to its ability to collect all test insights and compile the data in a digestible and downloadable report. This unmoderated tool lets you record the audio and video of your testers so you can listen to the user's explanation later, and it integrates with all your favorite prototype and design tools, such as Adobe XD, Figma, InVision, and Sketch.

Why is it important to test your website after launch?

It’s important to test your website after launch because you get to catch performance and usability issues early. This helps you avoid design, copy, or development problems that turn into multiple bad reviews or sales losses.