How to conduct content testing like a pro: A step-by-step guide

Content is everywhere. It’s very often the first way your users will engage with your product.

You want your content to stand out for the right reasons, and for users to find (and understand) your content first-time round. If they’re re-reading, you’re doing it wrong.

So, what’s the key to ensuring the content on your site meets user needs? Content testing.

In this article, we’ll cover the what, why, and how of content testing—complete with a step-by-step guide to running your own content tests. Plus, we'll share some insights from industry experts: Vaida Pakulyte, UX Lead at Technigo, Steve Howe, Content Designer, and Nicole Michaelis, Content Design Lead at Wolt for their expert insights throughout.

Content testing is a research method that helps understand if users can find and understand your content. With content testing, you can check if the content you create resonates with users, speaks directly to their pain points, and provides enough context to help users complete tasks successfully.

Content testing is widely used during user research to test product, app, or website content, and can be conducted at any stage in product development.

Content testing is important as a preemptive tool to prevent bad content. Bad content can completely disrupt a user’s experience of a product and leave them confused or unengaged, which may drive them away from using your product altogether.

Content testing helps you:

Put simply, content testing helps ensure good content makes it into the product, and bad content never makes it out of the testing process.

Wherever there is content in your product, it’s worth doing content testing. We’re yet to see a product that uses no content at all (as unique as that would be).

Think about every piece of content on your product—your homepage, error messages, menu buttons, filters, product pages, and more. At first glance you may not think of this as content, but it is—and it all needs testing.

To decide which content to test, prioritize based on the urgency and importance the task holds for users. If it’s a critical message, such as an error message in payment flows, then it’s vital that the content is easily understood and helps the user achieve their goals.

Vaida Pakulyte, UX Lead at Technigo, emphasizes that when planning to test content, it’s important to give a reference for users to evaluate the content. For example, you can provide context such as a design mockup or a prototype.

Include visual elements, buttons, and everything else that will be on the final design. Your users will be seeing your content in context, so the way you test it should be the same (whenever possible).

So, what are you testing for? There are five main ways to evaluate content, each looking at a different key aspect for in-product content success:

Looking out for these five areas when testing your content will help ensure it’s exactly what your audience wants.

Determining the right stage of development to test your content is tricky. On the one hand, you want your content to reflect the real content your users will encounter. On the other, you don’t want to spend lots of time creating fully-formed and finalized content only to discover it’s not what your users need.

Ultimately, like most types of research and user testing, content testing works best when used iteratively at multiple stages in the design process. You can use content testing:

There are a number of methods for testing content, and each is particularly useful depending on your goals and use case.

You can run a usability test to test content, with slight modifications to the usual way the method works. In a typical usability test, users complete pre-scripted tasks that are closed-ended and shorter in form.

When testing content, keep the tasks open-ended and allow users to explore the product in their own time. The goal is to notice how users engage with a website or app, how they find the information they need, and whether they run into any confusion. You want to assess if the content is clear and enables people to use the product successfully.

Testing this way will provide you with qualitative information about what works and what doesn’t and give you insight into which content may need to be changed or improved. Apart from paying attention to what users say during the test, also notice their behavior: sometimes, the two may contradict each other.

Usability testing provides the perfect opportunity to make sure you and your users are on the same page in regards to the language used.

A Cloze test measures how well your readers can understand a piece of text through context and prior knowledge. It works by taking a 125- to 250-word piece of text and removing one every few words, typically every fifth or sixth.

You present this text to a test participant and see how many of the missing words they can guess correctly. An overall score of 60% or above is deemed comprehensible enough to meet user needs.

Here’s a short example of what a Cloze test looks like:

When Mr. Bilbo Baggins __________ Bag End announced that __________ would shortly be celebrating __________ eleventy-first birthday with a __________ of special magnificence, there __________ much talk and excitement __________ Hobbiton.

Card sorting enables you to learn how users understand and categorize information. It consists of having users group ideas or information in a way that makes sense to them using physical or virtual cards—revealing where users expect certain content to be found.

There are three different types of card sorting, all of which help at different stages of the content testing process. For example:

For more information on card sorting and it can help you better structure content, take a look at our guide to running a card sort.

A highlight test is particularly useful for seeing how people feel about your content, rather than testing for comprehension. As we pointed out before, your product’s voice is important, and this test is perfect for seeing how your copy resonates emotionally with users.

To do a highlight test, you’ll need a sample of your text and a red and green highlighter. You give participants the text and ask them to highlight in green what made them feel confident about the product, and in red what made them feel less confident.

If you’re testing for reactions and emotions other than confidence, you can just switch out “confident” for any other feeling.

In a five-second test, you present your users with a design and give them five seconds to look at it, then ask them questions. This method can be used to test landing page copy, UI content, and more. You’re testing to see if the information on your page can be quickly absorbed, perceived, or understood.

The questions you ask can range from broad, like “What do you think of the page?” or “What do you remember seeing?” to more specific ones, like “Which of the items are on sale?”.

It’s best to start the test with broad questions to get general feedback and thoughts from your participants, then asking more specific questions to see what information stuck with them.

Preference testing consists of showing research participants two to three versions of your design—in this case, with variations in the content—to get an idea of which they prefer and why. It can help you understand the user’s perception of your content, how it makes them think and feel, in order to optimize content in line with those user preferences.

Preference testing is best used during your initial testing process—not right when you begin, but early enough to understand what works and what doesn’t before you commit to a certain content or communication style. It’s similar to A/B testing—which we’re covering next—with the primary difference being when you should use it. Preference testing should be used on content drafts and wireframes, whereas A/B testing is better suited for live content. Let’s take a look.

An A/B test involves creating two variations of content and testing them with users on live websites. A/B tests can give you quantitative data on which content performs better.

This type of testing can provide useful metrics, such as the click-through rate (CTR) that allow you to determine the winning variation easily. However, to do well, you need to have a large sample size for statistical significance.

For example, Netflix is well-known for A/B testing artwork for titles to identify which images perform best at getting users to watch videos.

Readability formulas are a computer-run testing tool that provides an analysis of your content. They’re a simple way of testing how easy your copy is to read by prioritizing small words and short sentences.

However, if you’re truly aiming to test your content with real users, then readability formulas might not be the best option. While they may be an attractive solution, especially if you’re on a budget, they can often produce inconsistent results, and they don’t give any actionable feedback.

For better results, use readability tests in combination with the other methods described above.

So now you know why you should test content, and what you’re testing for, let’s cover how you can go about testing content.

We’ll break this process into six steps:

Before you even start planning your test, you need to consider what your objective is. In short, who and what are you testing, and what are your testing goals?

You should know by this point who your intended users are, and what their needs and expectations from your product are. If your product is used by people from a range of demographics, then the product should be tested with users from each group. If your test underrepresents a demographic, then you risk creating content that doesn’t speak to all your personas.

So choose the content that you want to test, create a concrete hypothesis, and you’re ready to start outlining the test.

With the who and why decided, you can then choose the most suitable method for testing and create the test.

For example:

Once you have created your initial test, you want to see if it makes sense. Before you start finding test participants, grab some colleagues or other stakeholders, and run the test with them first.

Testing the test will ensure the instructions make sense and that your test participants can successfully understand what is asked of them. This allows them to focus on your content, rather than trying to decipher the test itself.

With your test created, it’s time to find some test users.

If you’re running qualitative sessions, you need to find at least five participants. This number will give you a consensus to inform your content decisions. If you want to collect quantitative data, then it’s best to run the test with at least twenty people, more if possible. Ultimately, how many people you need to recruit depends on the type of test you’re doing, the method, and the criticality of the project.

The big focus here is to find representative users for the real test. Proxy users, such as the people you work with, may not be the people who use your product. You run the risk of getting biased feedback since they may already know information about the product or content, so the data isn’t reliable.

In our guide to finding and recruiting research participants, we break down seven ways to source participants for your user research. For example, you can find candidates online, use research participant recruitment tools, or even reach out to professional networks for feedback.

You now need to decide to run a moderated or unmoderated test. There are advantages to each method.

With a moderated test, you have greater control during the test. Having a facilitator present can provide an extra layer of feedback from the study. Moderating these studies allows you to ask follow-up questions specific to the user and the situation they find themselves in.

If you’re moderating the test, allow enough time for participants to read the content and give you feedback. Unlike traditional moderated usability testing where users complete tasks and talk through their thoughts, content testing may require significant stretches of time when both you and the participant are silent. Make sure to explain to the participants at the start that this is okay.

With unmoderated testing, you don’t need to find someone to act as a moderator for the test. This type of testing is usually done remotely and it allows you to run more tests at a faster rate, even simultaneously, which will give a greater amount of quantitative data for you to analyze. What’s more, it allows you to find users from different parts of the world, which can unlock insights about how people from different countries interact with your content.

Once this round of testing is done, you’re now able to analyze the test results. You should have an overall idea of how your piece of content works, as well as specifics to improve on—be it readability, voice, or any other important factor.

Note how we said ‘this round of testing’. That’s because these tests shouldn’t be a one-off—as Nicole puts it: “testing is never complete”. As your product and user base grow and evolve, you need to keep testing to see if your content works and speaks to them.

Several of the above content testing methods involve asking your participants follow-up or mid-test questions, but knowing the right questions to ask can be tricky. Let’s take a look at some example questions, and when to use them.

Questions to ask:

Questions to ask:

Questions to ask:

Questions to ask:

Your content is how you communicate with customers. It’s how you inform and engage users throughout their interactions with your product and business. Great content is crucial for intuitive and delightful user experiences, and content testing is how you get there.

Whether you’re working with a first product launch, or a new feature, it’s an imperative step for any of your in-product or on-site content—and key for making sure you meet user wants and needs.