AI in user research: The battle of ethics and efficiency

Since the introduction of ChatGPT to the masses, AI-powered tools like Perplexity, Gemini and Claude have been flooding the product world one AI prompt at a time.

Some user researchers have been early-adopters, helming the latest wave of innovation. Others have been more skeptical, calling for consideration of the risks and ethical implications of using AI and LLMs (large language models) in user research.

Whichever side you fall on, AI is here to stay—and that means being informed on how to use it safely, ethically, and efficiently. (That’s where this article comes in.)

The undebatable truth is that AI helps product teams scale product research and increase efficiency. An AI co-pilot means faster research, more insights, and quicker decisions.

But while 58% of product teams already incorporate AI into their research practice, there are some reluctant to adopt AI. Our Future of Research Report 2025 found the top concerns of using AI were:

Other doubts range from AI’s tendency to present false information and experience ‘hallucinations’, to its propensity to churn out biased data—not to mention its massive contribution to the climate crisis.

While justified, these concerns don’t stop AI from evolving—and research workflows with it. As the demand for user insights continues to grow, product teams need to embrace AI-powered tools to accelerate research.

But can you do this while balancing ethics and efficiency?

With the right safeguards in place, AI can amplify the role and expertise of researchers, empowering them to take on more strategic, business-centric projects—without jeopardizing the integrity of user insights.

Just like negating cognitive biases in UX research, integrating AI means understanding the ethical risks, before learning how to counteract them.

Top challenges to address include:

Read on for the top ethical conundrums to consider—with best practices and guardrails to prevent them catching you out.

The potential for ingrained bias across generative AI output is a top concern for many researchers.

This isn’t unfounded—AI technology is known for displaying bias. This is because it’s only as good as the data it’s trained on, and since much of our society is fundamentally biased (e.g. eurocentrism, heteronormativity), AI also holds these assumptions.

Think about it like a child repeating their parents’ views even when they don’t understand: AI listens, absorbs, and imitates.

In practice, this can look like AI distorting data—for example, converging user feedback or experiences and skimming off any diverse or surprising perspectives, in order to create a homogenous narrative that ‘fits’ expectation.

The best way to minimize bias is to incorporate fully inclusive UX design and research practices throughout your wider research workflow. These changes will then trickle down to your AI usage.

This might look like:

One of the most frequent hesitations around AI in UX research is data privacy and storage—a concern intrinsically intertwined with participant consent.

Understanding the ins-and-outs of how user research tools process and retain your information is tricky at the best of times, but when you introduce GPTs, LLMs and third-party data-processing jargon… it all gets a bit overwhelming.
Especially if you’re a research participant just looking for the box that says ‘Accept Ts&Cs’.

This opacity of AI systems (called the ‘black box problem’) means research participants can’t clearly understand how their data is used—and therefore can’t fully consent.

Other concerns over how AI may use data include:

One new consideration for product teams is where consent falls when users are interacting with AI. After Reddit’s run-in with researchers deploying AI bots posing as people, product teams must make a conscious effort to over-communicate when users are speaking with AI—and clearly gain consent before doing so.

While AI has changed what users need to consent to—and the due diligence product teams must have with third-party tools—many standard user research best practices remain the same.

Informed consent means ensuring individuals understand risks, benefits, and implications of their consent before giving it.

To minimize the risk to your users, you should:

Making sure your users understand exactly what they’re agreeing to is critical for ethical user research. If all of the above isn’t communicated, consent becomes misleading at best—or, at worst, coercive.

When AI assists in creating research questions, or synthesizing user insights, the line between human and AI-generated content becomes blurred.

These concerns around authenticity and attribution raise questions about intellectual ownership, research integrity, and trustworthiness of takeaways.

We know AI is prone to hallucinations, so how can researchers rely on AI-generated insights?

Any good researcher is skeptical. They want traceability and reliability. So how can you support your team in assuring authentic, quality insights?

It’s impossible to talk about artificial intelligence without also acknowledging the impact AI is having on our environment.

While AI has potential to positively impact on climate change—scientists are already detecting patterns in extreme weather, improving fuel efficiency, and preventatively monitoring pollution—it is by and large having a detrimental effect on the climate crisis.

Just some of the ways AI impacts the environment are:

While the responsibility largely sits at a corporate and organizational level, product teams can make a difference. And it’s part of everyone’s responsibility to consider ways to minimize AI’s impact on the environment.

Maze’s CEO & Co-founder, Jo Widawski, explains that climate positive changes shouldn’t be made to counteract using AI—they should already be ingrained in the company.

If you’re just starting out and looking to implement everyday practices that guide your team’s AI usage, try these:

You know the risks, you know the best practices, but what are other UX professionals doing? As part of our research building AI moderator, we spoke to a number of product teams and industry experts, from AI enthusiastics to cynics alike.

Our research revealed a number of consistent guardrails teams have put into place to support ethical AI usage that scales research and protects against misinformation.

A common theme was emphasis on AI as a complementary resource, rather than a replacement.

As Jo explains: “The thing that makes research powerful is empathy. If you’re only doing AI-powered studies, it’s easy for the voice of the human to be flattened and lose nonverbal signals that are a defining part of the human experience within a product.”

Similarly, Matthieu Dixte, Senior Product Researcher at Maze, recommends combining AI-driven research with human-led research. For example, using AI to scale quantitative analysis and pattern recognition, while deploying human researchers for instances where empathy matters most.

Treat AI as an iteration tool, process support, or initial starting point, but don’t leave it to do everything unguided. Likewise, if you’re exploring synthetic users, use them to complement the findings from human research participants.

Another guardrail product teams are relying on is implementing human review processes.

While AI is exceptional at scale and speed, it can return false information or hallucinate. Having a human-in-the-loop is vital to uncover ethical, accurate AI findings.

This ranges from manually fact-checking and verifying each AI output, to incorporating random checks in ongoing AI-powered workflows. Ultimately, it’s up to your team to determine when and how you approach quality assurance.

One question product teams must answer is how datasets are passed on to train AI algorithms. Currently a murky area for participant consent, many product teams are firmly against user data being used to train LLMs.

Peter suggests many best practices for processing data don’t differ when it comes to AI usage:

“As long as the data is not being used for training the models, the same principles exist as with regular use of software for research purposes. You need to make sure you own the data, and that the data is not being used for unwanted purposes. Consent and clear communication is vital.”

When it comes to the perception of research participants, some may be put off by knowing AI is processing their data. While this attitude will likely evolve over time, it’s important to meet users where they’re at.

Pick and choose how you use AI, when you use it, and what it has access to. Then communicate all of that with your participants.

AI is only as good as the technology it’s built on—and the prompts you feed it.

Matthieu, who’s been instrumental in implementing behind-the-scenes guardrails for Maze’s AI features, explains his approach:

“I consider AI like a junior assistant researcher. It might lack critical thinking due to inexperience in the role, in the org, or to avoid offending me.”

Like Matthieu, many product teams are dedicating time to improving the AI products they work with. If you’ve created your own GPT or have your own account with Claude, ChatGPT, or Gemini, you can help the AI succeed by providing feedback and engineering clear, explicit prompts.

One primary guardrail researchers have embedded into their practice is trust.

Trust can mean many things, but when it comes to AI it broadly falls into three categories:

As Jo says, “research is a practice that has to constantly justify its own existence. If you're putting yourself in the shoes of a researcher who has been running studies for 10 years, they’ve spent so long building trust with their executives.”

Introducing AI tools can feel like handing over that hard-earned credibility to an unvetted counterpart, so it’s vital to align your team, and empower them to balance AI with their own judgement.

One thing you can do is opt for AI tools that equally prioritize trust.

Peter explains: “This is where traceability becomes very important. Your findings should have a direct connection to the source data, so you can easily review the relevancy and correctness of the AI’s insights.”

Work to cultivate that trust—within yourself, your tool, your team—then build out from that foundation and integrate ethical AI best practices to maintain stakeholder confidence and organizational trust.

The best practices and guardrails right for your team and research practice need to come from you, but that doesn’t mean we can’t help guide you to make those calls.

Here’s our downloadable checklist for assessing and implementing ethical AI research practices.

AI advancements bring exciting possibilities in user research, but they also muddy the ethical and legal responsibilities that researchers face.

As AI tools become more sophisticated and widely-adopted, it’s not just trust within your organization or participant boundaries that need considerations—teams must prepare for more complex, global ethical AI challenges.

Current regulations of machine learning are a compliance minefield for researchers, differing hugely between countries.

As strict regulation and enforcement on AI increases, research teams should stay on top of the latest governance.

Here’s the status quo:

Like any technology dealing with sensitive information, audits will likely arise for many forms of AI—be it on a company, industry, or national level.

Over time, researchers can expect to see best practices, like those outlined above, becoming mandatory through reviews that check AI systems for functionality, transparency, ethics, and legal compliance.

To get ahead, product teams should future-proof AI usage by establishing clear documentation, audit trails, and robust internal processes for any work relating to AI.

The emergence of synthetic users poses not just an ethical question, but a philosophical one.

While AI-generated personas offer value for large-scale research or early ‘scrappy’ testing, they also counteract the fundamentally human-centered nature of user research.

The majority of researchers see the potential in synthetic users, but endorse them as an addition to human users, rather than a substitute.

To prepare, teams should consider whether they would be comfortable using synthetic testers, how to validate AI-driven insights against authentic user behavior, and how to maintain that all-important human connection.

We can’t wrap up this deep dive into the ethics of AI in user research without turning the mirror inwards—so let’s take a look at Maze’s approach to AI.

You’re in control: Whether it’s AI Follow-Up questions or an AI-moderated interview, each functionality allows you to set the rules, preview the conversation and build with your preferred practices.

Everything is trackable: Each insight Maze AI generates is traceable, allowing you to dive into transcripts, AI-generated takeaways or highlights, and follow the AI’s decision path to get there.

Your data is never used to train AI models: The third-party providers Maze uses for AI will never use your study data (survey responses, test results, conversations, recordings) to train their models.

Your data is deleted after 30 days: Any data sent through third-party APIs is retained for a maximum of 30 days, then deleted permanently.

You can always opt out: Maze allows you to opt-out of specific AI features for yourself or your team, so you can control what you’re comfortable with.

We’re always evaluating: With strict human and AI-led review processes, Maze is always reviewing the quality, speed, and effectiveness of AI features.

Guardrails are built-in: Maze AI is meant to take the pressure off, not add to it—so rather than bombard you with instructions and tool tips, all the guardrails for quality and consistency are built into the product.

We cross-check between models: Maze uses OpenAI and Claude to power AI features, and cross-evalutes the outputs of both—meaning our reviews are never biased to one AI model.

Transparency is integral: We’ll never hide how or where AI is used—you can read all the nitty-gritty details right here.

We’re prioritizing value over hype: There’s a reason we waited before launching an AI moderator—getting it right was our priority, rather than shipping fast at the expense of your trust.

AI might be the new co-pilot for your research practice, but that doesn’t mean commitments to ethical and inclusive user research are changing.

With the right boundaries and best practices in place, research teams can harness AI’s efficiency while maintaining the integrity and empathy that sit at the heart of user research.

Above all, focus on building ethical foundations that center your users. Speed means nothing without trust, and insights lose all value if they compromise the very people they're meant to serve.