UX research with AI can save time: automatic transcription, interview synthesis, feedback tagging, pain point clustering. But it introduces ethical risks many teams ignore: consent, representativeness, confidentiality and truthfulness of data presented.
Basic rule: real people's data — interviews, recordings, surveys — must not be uploaded to AI tools without explicit consent and clear policy on what happens to data (training, storage, sharing). Many SaaS tools use user input to improve models. Informing research participants is mandatory.
Personas and user journeys generated by LLMs are not research — they are plausible fiction. Useful for creative warm-up or initial exploration, dangerous if presented in stakeholder meetings as "our users". Always label them: "Hypothesis to validate", not "Research insight".
"Synthetic users" — AI agents simulating user behaviour — are debated. They can stress-test hypotheses, but replicate model bias and do not capture cultural context, emotion, material constraints. They do not replace even 5 interviews with real people from the target group.
Automatic interview synthesis can homogenise diverse voices, lose nuance, emphasise what the model considers "normal". The researcher must read original transcripts, especially for underrepresented groups whose patterns the model might downplay.
Participatory and co-design remain the gold standard for AI products affecting vulnerable communities. Involving representatives of those who will be classified, excluded or surveilled by the system is not feel-good — it produces insights no dataset captures and prevents pre-launch harm.