Self-Anchoring Calibration Drift in Large Language Models: How Multi-Turn Conversations Reshape Model Confidence

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A study published in the Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM) in July 2026 introduces Self-Anchoring Calibration Drift (SACD), a phenomenon where large language models (LLMs) systematically alter their expressed confidence when iteratively building on their own prior outputs in multi-turn conversations. Researchers conducted a controlled three-condition study, evaluating Claude Sonnet 4.6, Gemini 3.1 Pro, and GPT-5.2 across factual, technical, and open-ended domains. The findings confirm that SACD is a real and multiform issue, with models exhibiting unique self-anchoring signatures, including active confidence suppression and calibration improvement suppression. These effects were primarily observed in open-ended domains. This research challenges the sufficiency of traditional single-turn calibration evaluations for accurately assessing LLM reliability in realistic multi-turn deployment scenarios. Code and data are publicly available.

Key takeaway

For NLP Engineers and AI Scientists deploying LLMs in conversational agents, you must re-evaluate your model's calibration beyond single-turn assessments. Your current reliability metrics may be insufficient, especially for open-ended interactions where Self-Anchoring Calibration Drift (SACD) can systematically alter model confidence. Consider implementing multi-turn calibration evaluations to accurately characterize how your LLMs behave in realistic, iterative dialogue contexts. This will help you mitigate unexpected confidence shifts and improve user trust.

Key insights

Large language models exhibit Self-Anchoring Calibration Drift (SACD) in multi-turn conversations, altering confidence based on prior outputs.

Principles

• LLM calibration shifts in multi-turn interactions.
• SACD manifests as confidence suppression.
• Single-turn calibration is insufficient for multi-turn reliability.

Method

A controlled three-condition study compared Claude Sonnet 4.6, Gemini 3.1 Pro, and GPT-5.2 across factual, technical, and open-ended domains to identify SACD.

Topics

• Large Language Models
• LLM Calibration
• Multi-turn Conversations
• Self-Anchoring Calibration Drift
• Model Confidence
• Generative AI Evaluation

Code references

• hvardhan878/calibration-drift

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• The Hidden Reason LLMs Fail in Conversations: CCOPD — CCOPD re-anchors LLM responses in multi-turn conversations to a canonical context, mitigating "self-anchored drift" from piecemeal information.
• An Empirical Study of Security Calibration in Large Language Models for Code — LLMs for code are systematically overconfident in security, especially in complex repository contexts, making their self-assessments unreliable.
• Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models — Aligning multi-turn LLM behavior with full-context teacher responses mitigates self-anchored drift.
• SICI: A Semantic-Pragmatic Complexity Index Reveals Regime Shifts in LLM Stance Detection — LLM stance detection errors exhibit predictable regime shifts tied to semantic-pragmatic complexity, leading to distinct error patterns.
• Trained on Tokens, Calibrated on Concepts: The Emergence of Semantic Calibration in LLMs — Base LLMs exhibit emergent semantic calibration in Q&A, which instruction-tuning and chain-of-thought reasoning degrade.
• The Self-Correction Illusion: LLMs Correct Others but Not Themselves — LLMs' self-correction failure is a chat-template artifact, not a cognitive deficit, due to addressability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.