Traces of Social Competence in Large Language Models

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Research investigating social competence in Large Language Models (LLMs) addresses limitations of the False Belief Test (FBT), such as data contamination and inconsistent controls. This study tested 17 open-weight models using 192 FBT variants and Bayesian Logistic regression to analyze the impact of model size and post-training. Findings indicate that scaling model size generally improves performance, though not strictly. A significant "cross-over effect" was observed, where explicitly stating propositional attitudes like "X thinks" fundamentally changes response patterns. Instruction tuning partially reduces this effect, while reasoning-oriented fine-tuning amplifies it. A case study on OLMo 2 training revealed this cross-over effect emerges during pre-training, suggesting LLMs develop stereotypical responses to mental-state vocabulary that can override scenario semantics. Furthermore, vector steering isolated a specific "think" vector as the causal factor for observed FBT behaviors.

Key takeaway

For AI Scientists evaluating LLM social reasoning or Theory of Mind, be aware that False Belief Test results can be significantly influenced by explicit mental-state vocabulary like "X thinks". Your models might be exhibiting stereotypical linguistic responses rather than genuine socio-cognitive competence. Consider that instruction tuning can partially mitigate this effect, while reasoning-oriented fine-tuning amplifies it. Therefore, carefully scrutinize FBT methodologies and model training data to avoid misinterpreting superficial linguistic patterns as true understanding.

Key insights

LLMs' apparent social competence, particularly in False Belief Tests, is driven by specific mental-state vocabulary rather than deep understanding.

Principles

• Model scaling improves FBT performance.
• Explicit "X thinks" alters LLM responses.
• Mental-state vocabulary can override scenario semantics.

Method

Assess LLM social competence by testing 17 open-weight models on 192 False Belief Test variants using Bayesian Logistic regression, then isolate causal drivers via vector steering.

In practice

• Use instruction tuning to mitigate FBT biases.
• Avoid reasoning fine-tuning for FBT accuracy.
• Analyze pre-training for emergent ToM patterns.

Topics

• Large Language Models
• Theory of Mind
• False Belief Test
• Instruction Tuning
• Vector Steering
• Model Scaling

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• GPT-4o Lacks Core Features of Theory of Mind — LLMs like GPT-4o lack a consistent, domain-general causal model of mental states and behavior, despite social task success.
• Modeling Pathology-Like Behavioral Patterns in Language Models Through Behavioral Fine-Tuning — Behavioral fine-tuning can induce specific, stable, pathology-like biases in LLMs, shifting their fundamental interpretive priors.
• To Compare, or Not to Compare: On Methodological Practices in Evaluating Social Bias — LLM social bias activation differs significantly between isolated and comparative evaluation settings.
• The Riddle Riddle: Testing Flexible Reasoning in Large Language Models and Humans — LLMs struggle with flexible reasoning, often conflating pattern matching with genuine understanding, unlike humans who adapt reasoning strategies.
• NICE: A Theory-Grounded Diagnostic Benchmark for Social Intelligence of LLMs — NICE provides a theory-grounded diagnostic benchmark to identify specific social intelligence weaknesses in LLMs, crucial for their application in social contexts.
• Analysis of the Neglect-Zero Effect in Large Language Models — LLMs may not share the human "neglect-zero effect" cognitive bias regarding empty sets.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.