The Reliability Illusion in Synthetic Patients: Psychometric Misalignment of Open-weight LLMs on PHQ-9 and GAD-7

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, AI in Mental Health Assessment · Depth: Expert, medium

Summary

A study by Qian Shen and Yu Han, presented at the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026), investigates the psychometric reliability of open-weight Large Language Models (LLMs) when generating responses for mental health assessment scales. Researchers used four open-weight LLMs to complete the GAD-7 and PHQ-9, manipulating prompts, sampling temperature, and dynamic contextual scenarios to mimic human response patterns. Employing multi-group confirmatory factor analysis and differential item functioning analyses, the study evaluated the factor structure of LLM-generated responses against human baselines. Findings reveal that despite exhibiting exceptionally high internal consistency, these LLMs demonstrate severe structural mismatch and fail to achieve scalar measurement invariance. This suggests LLMs rely on superficial, stereotype-driven semantic matching and prompt fragility rather than simulating stable latent psychological dynamics, raising concerns for their use in generating mental health assessment items.

Key takeaway

For research scientists developing AI for mental health applications, you must critically evaluate the psychometric validity of LLM-generated assessment items. Do not rely solely on internal consistency metrics, as LLMs can exhibit a "reliability illusion" without reflecting true psychological dynamics. Your validation process should include rigorous psychometric methods like differential item functioning and measurement invariance analyses against human data. This ensures your AI tools genuinely assess mental health, avoiding superficial, stereotype-driven outputs.

Key insights

Open-weight LLMs exhibit a "reliability illusion" in mental health assessments, showing high consistency but severe psychometric misalignment.

Principles

• High internal consistency does not imply psychometric validity.
• LLM responses are sensitive to prompt variations.
• LLMs simulate semantics, not latent psychological states.

Method

The study used multi-group confirmatory factor analysis and differential item functioning analyses on LLM-generated GAD-7 and PHQ-9 responses, varying prompts and sampling temperature.

In practice

• Validate LLM-generated psychometric scales with human baselines.
• Test LLM response stability across diverse prompting strategies.

Topics

• Large Language Models
• Mental Health Assessment
• Psychometrics
• GAD-7
• PHQ-9
• Measurement Invariance
• Prompt Engineering

Best for: AI Scientist, Research Scientist, AI Ethicist

Related on AIssential

• A better method for identifying overconfident large language models — Cross-model disagreement combined with self-consistency more reliably identifies LLM overconfidence and hallucinations.
• Can We Trust AI-Inferred User States. A Psychometric Framework for Validating the Reliability of Users States Classification by LLMs in Operational Environments — AI-inferred user state metrics often lack individual score reliability, limiting real-time adaptive system use.
• Diagnosing LLM Judge Reliability: Conformal Prediction Sets and Transitivity Violations — LLM judge reliability varies significantly by evaluation criterion and input difficulty, not just the judge itself.
• Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact — LLM psychological profiles are largely measurement artifacts driven by response bias, not genuine traits.
• Quantifying Consistency in LLM Logical Reasoning via Structural Uncertainty — Structural uncertainty quantifies LLM logical reasoning consistency by measuring the stability of self-preference-induced rankings over sampled solutions.
• Auditing Framing-Sensitive Behavioral Instability in Large Language Models for Mental Health Interactions — LLMs exhibit framing-sensitive behavioral instability in mental health contexts, rooted in internal representations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.