LLMs as Standardised Patients for Motivational Interviewing: How Faithful Are They?

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Science & Research — Mathematics & Computational Sciences, Health & Medical Research · Depth: Expert, medium

Summary

A study presented at the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026) investigates the faithfulness of large language model (LLM)-simulated patients in motivational interviewing contexts. Researchers Van Hoang, Eoin Rogers, and Robert Ross directly compared data properties generated by LLM simulations and human patients given identical profiles, avoiding subjective user experiences. Their findings, published on pages 258–270, indicate that while LLM-simulated patients produce semantically similar content and engage with comparable topics to human counterparts, their expressive modes differ significantly. Specifically, LLMs struggle to replicate the full complexity of human behaviors and attitudes, tending to skew towards uniformly positive responses, unlike human patients who exhibit a mix of positive and negative reactions.

Key takeaway

For research scientists developing LLM-based clinical training tools, you should prioritize enhancing the models' ability to generate diverse and nuanced emotional responses. Your current LLM patient simulations, while semantically accurate, may oversimplify human behavior by skewing uniformly positive, potentially hindering realistic training scenarios for motivational interviewing. Consider incorporating mechanisms to simulate a broader spectrum of human attitudes and mixed responses to improve fidelity.

Key insights

LLM-simulated patients in motivational interviewing contexts exhibit semantic similarity but lack the full emotional and behavioral complexity of human responses.

Principles

• LLM patient simulations lack behavioral complexity.
• Human patient responses are mixed.
• LLM patients skew uniformly positive.

Method

The study directly compared data properties from LLM-simulated and human patients using identical profiles, bypassing subjective user experience evaluations for faithfulness assessment.

In practice

• Enhance LLM emotional response diversity.
• Calibrate LLM patient positivity bias.
• Integrate human behavioral nuances.

Topics

• Large Language Models
• Motivational Interviewing
• Clinical Simulation
• Patient Simulation
• Computational Linguistics
• Behavioral Fidelity

Best for: AI Scientist, Research Scientist, Domain Expert

Related on AIssential

• Adaptive Interviewing for Persona Simulation in LLMs: Evidence-Grounded Reasoning Improves Decision Alignment — Adaptive interviewing improves LLM decision simulation by selectively grounding predictions in user-specific, follow-up derived evidence.
• Fake users generated by AI can't simulate humans — review of 182 research papers — LLMs fail to simulate human cognition and behavior due to their training on text, not diverse lived experience.
• Cross-Model Consistency of AI-Generated Exercise Prescriptions: A Repeated Generation Study Across Three Large Language Models — LLM consistency for clinical tasks varies significantly, demanding careful evaluation beyond single-output metrics.
• Artificial Intolerance: Stigmatizing Language in Clinical Documentation Skews Large Language Model Decision-Making — LLMs inherit and propagate human cognitive biases from stigmatizing language in clinical notes, leading to less aggressive patient management.
• An Infectious Disease Spread Simulation Based on Large Language Model Decision Making — LLMs can simulate nuanced human health behaviors in spatially-grounded agent-based models, revealing demographic disparities.
• When AI Says It Feels — LLMs can be trained to express human-like feelings and self-awareness via self-rewarded RL, with manageable performance trade-offs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.