How to do AI analysis you can actually trust

2026-02-17 · Source: Lenny's Newsletter · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Project & Product Management · Depth: Intermediate, long

Summary

Caitlin Sullivan, a user-research veteran, outlines four effective techniques for extracting trustworthy and actionable user insights from large language models (LLMs) like ChatGPT, Claude, and Gemini. She highlights that AI outputs often appear confident even when containing inaccuracies, such as invented quotes or false insights, which can lead to flawed product decisions. The article details common failure modes in AI-powered customer research, including invented evidence, generic insights, unhelpful "signal," and contradictory findings. Sullivan explains why LLMs struggle with unstructured interview data and complex survey responses, and provides specific prompting strategies to mitigate these issues. She also offers recommendations on which LLM is best suited for different analytical tasks, favoring Claude for in-depth analysis due to its thoroughness.

Key takeaway

For AI Product Managers evaluating LLMs for customer research, recognize that AI's confident but often flawed outputs necessitate rigorous validation. Your teams should implement explicit quote selection rules and a two-step verification process to ensure evidence accuracy. Prioritize comprehensive context loading in prompts, detailing project scope, business goals, product specifics, and participant overviews, to prevent generic insights and align AI analysis with critical product decisions.

Key insights

AI-generated customer insights require specific prompting and verification to overcome inherent biases and prevent critical decision errors.

Principles

• AI defaults to consensus, losing specificity and edge cases.
• LLMs generate text, not retrieve quotes like search engines.
• Context loading must be precise to guide AI interpretation.

Method

Define quote selection rules, verify quotes against source transcripts, and provide comprehensive context loading (project, business, product, participant) to guide AI interpretation and prevent generic or false insights.

In practice

• Use Claude for thorough, nuanced analysis.
• Employ quote verification prompts to detect fabrications.
• Specify "QUOTE SELECTION RULES" in prompts.

Topics

• AI Hallucinations
• User Research
• Prompt Engineering
• LLM Comparison
• Data Analysis Reliability

Best for: AI Product Manager, Product Manager, Research Scientist

Related on AIssential

• How to do AI analysis you can actually trust — AI analysis of customer data often yields confident but flawed insights, requiring specific prompting techniques for reliability.
• What We’ve Learned From A Year of Building with LLMs — Successful LLM product development requires a holistic approach beyond demos, focusing on robust systems, data, and strategic iteration.
• 4 Lines You Should Include in Your Claude Skill — LLMs can be "confidently wrong" without explicit prompt constraints on context, significance, and confidence.
• AI keeps getting smarter, so why does it still fail at obvious things? — AI's apparent intelligence stems from pattern matching, not true comprehension, leading to reliability gaps.
• How Spotify Taught an LLM to Think Like a Senior Data Analyst — LLMs fail from lack of meaningful context, not intelligence; expert-owned context is paramount.
• Google’s AI Can’t Spell Google. That’s Not a Joke. — LLMs fundamentally struggle with character-level tasks like spelling due to tokenization, impacting their reliability in factual search.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Lenny's Newsletter.