Do LLMs Know What They Know? Measuring Metacognitive Efficiency with Signal Detection Theory

2026-07-15 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A new evaluation framework, based on Type-2 Signal Detection Theory, measures Large Language Model (LLM) confidence by distinguishing between a model's knowledge (Type-1 sensitivity, d') and its awareness of that knowledge (Type-2 metacognitive sensitivity, meta-d'). This framework introduces the M-ratio (meta-d'/d') to quantify metacognitive efficiency. Applied to four LLMs (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Llama-3-8B-Base, Gemma-2-9B-Instruct) across 224,000 factual QA trials on TriviaQA and Natural Questions, the study found significant variations: Mistral achieved the highest d' but the lowest M-ratio (0.852), while Gemma-2 had the lowest d' but near-optimal M-ratio (1.048). Metacognitive efficiency was also domain-specific, and temperature manipulation dissociated confidence policy from metacognitive capacity for two models. Critically, AUROC2 and M-ratio produced inverted model rankings, highlighting their distinct evaluative questions.

Key takeaway

For AI Scientists and Machine Learning Engineers deploying LLMs in confidence-dependent systems, you should integrate meta-d' and M-ratio into your evaluation suite. These metrics reveal critical metacognitive deficits invisible to standard calibration scores like ECE or AUROC2. Prioritize models with higher M-ratio for applications requiring reliable uncertainty estimates, even if their raw accuracy (d') is slightly lower. Additionally, disaggregate these metrics by domain to identify and mitigate specific knowledge blind spots.

Key insights

Metacognitive efficiency, measured by M-ratio, reveals how well LLMs know what they know, distinct from their factual accuracy.

Principles

• LLM confidence evaluation requires distinguishing Type-1 from Type-2 sensitivity.
• Metacognitive efficiency is domain-specific and varies across models.
• Temperature tuning primarily shifts confidence criterion, not metacognitive capacity.

Method

The Type-2 SDT pipeline uses normalized log-probability (NLP) as an evidence variable, bins NLP values into 2 × K categories, and estimates meta-d' via maximum likelihood with metadpy 0.1.2.

In practice

• Use meta-d'/M-ratio to diagnose LLM confidence signal quality.
• Disaggregate confidence metrics by knowledge domain for granular insights.
• Prioritize M-ratio over ECE for selective prediction systems.

Topics

• LLM Confidence
• Metacognition
• Signal Detection Theory
• Model Evaluation
• Llama-3
• Mistral-7B
• Gemma-2

Code references

• embodied-computation-group/metadpy
• meta-llama/llama3

Best for: Research Scientist, AI Engineer, NLP Engineer, AI Scientist, Machine Learning Engineer, Director of AI/ML

Related on AIssential

• Making LLMs tell you how confident they really are through probe-targeted fine tuning.[R] — LLMs internally know their confidence but need fine-tuning to express it verbally, overcoming RLHF's bias.
• LLM as a Meta-Judge: Synthetic Data for NLP Evaluation Metric Validation — LLMs can generate synthetic data to validate NLP evaluation metrics, replacing costly human annotations.
• The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs — Quantization in LLMs creates behavioral divergence not captured by accuracy, necessitating new evaluation metrics.
• Wisdom Of The (AI) Crowd: Investigating Artificial Swarm Intelligence In Large Language Models — LLM swarms can reduce estimation errors and exhibit metacognitive awareness, mimicking human swarm intelligence.
• Making LLMs faster without sacrificing accuracy — Architectural choices significantly impact LLM inference efficiency without sacrificing accuracy, a gap addressed by new scaling laws.
• Latency-Response Theory Model: Evaluating Large Language Models via Response Accuracy and Chain-of-Thought Length — LLM evaluation improves by jointly modeling response accuracy and Chain-of-Thought length, revealing a negative correlation between ability and speed.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.