Concept Testing in UX: Moving Forward with the Right Ideas

Concept testing is a type of research that evaluates the feasibility, appeal, and potential success of a new product before it’s built. It puts the user front and center in the ideation process, using research methods like A/B testing, surveys, and customer interviews.

The main difference between a good and bad product concept is whether it actually appeals to the people it’s made for. Thorough testing is critical to gauge your audience’s interest, understanding, and likelihood-to-purchase, before committing time and resources to a concept.

Concept testing can be done at multiple stages in the product discovery process—during ideation, when prototyping, when defining marketing messaging, or just before the launch of the product.

“Concept testing is the fine line between the phases of scoping the problem, and defining the solution, as a part of wider product discovery,” says Netali Jakubovitz, Director of Product Management at Maze.

Here are some specific points in the product development process when concept tests can be run:

There’s no one way to conduct a concept test, either. Some teams opt for concept testing surveys, while interviews and focus groups are also useful for gathering qualitative information. The method you go for will depend on your goals and resources—more on this in Chapter 2.

Concept testing proves valuable because it validates that your customers actually want the product. Idea validation is an essential part of UX research and a necessary step in creating successful products.

We could go on for days about the advantages of concept testing, but here are just a few of the reasons why this technique is an essential part of product success:

Approximately 95% of new products fail, according to research from Harvard Business School. A whopping amount of resources go down the drain when a product flops, from the money spent to create it, to the effort exerted by the product development team.

While concept testing itself requires a slice of your budget, in the long term that investment is far less costly than launching an idea that fails. For example, once a software product is launched, correcting an error can cost 100 times as much as it would have if it were fixed in the development phase. Studies also reveal that development teams waste 40 to 50% of their time on avoidable reworks, a reality that can be avoided with concept testing.

Concept testing also provides data to back up your idea and get buy-in from stakeholders or other members of your team. Bringing a new concept to life can be daunting, and management may understandably be cautious about taking that risk. However, once you have the research from concept testing on your side, even your most cautious team members will see the potential in your product concepts.

Andrea Ruggeri, Product Manager at Doodle, says that he finds concept testing useful for decreasing uncertainty. “The data allows product managers and decision-makers to gain confidence in favor or against an idea or a solution,” Andrea explains.

Concept testing can expose a new path for your company to take, whether that means optimizing the existing features of your product or pivoting entirely.

Your assumptions about what makes your concept great are just that—assumptions. So testing shows what aspects of the idea actually appeal to your customers, which you can then capitalize on when developing or highlighting the concept's best attributes in your marketing campaigns.

Even a concept test that appears to fail still has value. "Early failures are not only desirable but also necessary since experimenters can quickly eliminate unfavorable options and refocus their efforts on more promising alternatives (often building on ideas that were initially unsuccessful)," writes Stefan H. Thomke, Author of “Experimentation Works”.

Martín Burgener, Product Director at eDreams, explains how failure is a key element of building products at eDreams: “If we think about the products eDreams have been working on, roughly 70-85% of all tests we do result in a failure. And that’s for a company that is product-led and has been making digital products for 12 years.”

Once you evaluate what matters to your target audience through concept testing, you can use those learning opportunities to tweak or completely change your product to maximize its success.

To ensure that your concept testing efforts are effective and yield actionable insights, you need to follow some best practices that align with your research objectives and the development stage of your product idea.

Continuous product discovery and concept testing work effectively in tandem. As part of your continuous discovery habits, every time you make a change to your product, the new concept should be tested with your audience. Conducting iterative rounds of concept testing, even after your product launch, can help identify new opportunities for innovation and improve your product-market fit.

Just trying to gauge if customers ‘like’ your product concept is a bit too vague. You’ll get more value from your test if you have specific goals, for example: “We want to validate that most of our users (80% or more) find this feature useful.”

You can measure different aspects of the product concept, like perceived value, messaging, price sensitivity, and overall appeal. Do this by using a variety of testing formats to gather feedback on the different aspects of your product concept (more on this later).

The right research questions will reveal the insights necessary to validate your concept or ascertain what it’s lacking. For example, asking participants if they "like" your product concept may not provide enough detail to guide product development decisions.

On the other hand, targeted questions like "What do you like about Product A?" or "What features do you think Product B is missing?" can help you gain deeper insights into customer needs and preferences.

Questions should be open-ended rather than leading, allowing participants to express their opinions and feedback in their own words.

For example, if you ask, "How satisfied are you with our product?" and give options such as "very satisfied," "somewhat satisfied," or "not satisfied," participants may be more likely to choose the options that are provided, even if they don't accurately reflect their level of satisfaction.

There’s no point in conducting concept testing with participants who don't fit the demographics of your customer base, as different demographic groups may have different needs, preferences, and behaviors.

For example, if you’re testing a product concept designed for women aged 35-50, it wouldn’t be useful to gather feedback from male participants or participants aged 60+, as you’ll gather irrelevant insights.

Recruiting the right participants is key. With their feedback as guidance, you can develop and launch a successful product that meets customers’ needs. One way to help is by using a user testing platform with a recruitment panel that allows you to filter participants.

Different goals require different testing formats, each suited to unique objectives and gathering various research metrics.

Moving forward

The beauty of concept testing is that it's beneficial for everyone involved. Your customers get a product that's actually relevant to them, designers are reassured they're on the right track, and product teams save time and money moving forward with the right ideas.

In the following chapters, we’ll walk you through how to run a product concept test, the creation of a concept testing survey, and some real-life examples of concept tests. With these tools in your belt, you can move forward with confidence and help ensure your users find value in your products.

Here’s to making life easier for product owners and users 🎇