OSCToM: RL-Guided Adversarial Generation for High-Order Theory of Mind

2026-05-19 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

OSCToM (Observer-Self Conflict Theory of Mind) is a novel approach designed to model nested belief conflicts within Large Language Model (LLM)-based Theory of Mind (ToM) tasks. Published on 2026-05-19, this system specifically addresses complex social settings where an observer's view of another agent conflicts with their own belief state, demanding recursive, multi-layered reasoning beyond simple perspective-taking. OSCToM integrates reinforcement learning (RL), an extended domain-specific language, and compositional surrogate models to generate these challenging observer-self conflicts. In experiments, the OSCToM-8B model achieved 76% accuracy on the information-asymmetric FANToM benchmark, a substantial improvement over the 0.2% reported by ExploreToM. It also enhanced FANToM results and maintained competitiveness on Hi-ToM and BigToM. The data-synthesis process is 6x more efficient, indicating that targeted training data can significantly boost smaller models' capabilities in advanced cognitive reasoning. Project code is available at https://github.com/sharminsrishty/osct.

Key takeaway

For AI Scientists developing advanced LLMs, OSCToM demonstrates a critical method for improving Theory of Mind capabilities. You should consider integrating RL-guided adversarial generation and targeted data synthesis to address complex, recursive belief conflicts. This approach allows smaller models to achieve significant gains in information-asymmetric reasoning, potentially making your training processes 6x more efficient while enhancing model accuracy on challenging cognitive tasks.

Key insights

OSCToM uses RL-guided adversarial generation to create complex ToM benchmarks, significantly improving LLM performance on recursive belief conflicts.

Principles

• LLM ToM reasoning is uneven in complex social settings.
• Recursive beliefs and information asymmetries challenge existing benchmarks.
• Targeted training data improves smaller models' cognitive reasoning.

Method

OSCToM combines reinforcement learning, an extended domain-specific language, and compositional surrogate models to generate observer-self conflicts for LLM ToM tasks.

In practice

• Generate high-order ToM benchmarks for LLMs.
• Improve LLM performance on information-asymmetric tasks.
• Efficiently synthesize training data for cognitive reasoning.

Topics

• Large Language Models
• Theory of Mind
• Reinforcement Learning
• Adversarial Generation
• Benchmark Development
• Data Synthesis Efficiency

Code references

• sharminsrishty/osct

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Recursive Think-Answer Process to Stop "Thinking" Earlier — Integrating confidence-aware self-correction and RL-aware distillation improves LLM reasoning and efficiency.
• Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learning — Adversarial co-evolution of Code LLMs and Test LLMs improves code generation and test quality by separating objectives.
• Mind the Perspective: Let's Reason Recursively for Theory of Mind — RecToM improves LLM Theory of Mind by recursively constructing nested perspectives.
• Escaping the Verifier: Learning to Reason via Demonstrations — RARO enables LLMs to learn robust reasoning from expert demonstrations alone, even without task-specific verifiers or human preferences.
• Context-CoT: Forcing LLMs to Actually Think (No ICL) — LLMs struggle to genuinely learn and reason from novel, complex knowledge provided solely in the prompt, often defaulting to pre-trained biases.
• Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP — Programmatic state abstraction and clean task decomposition are more effective than deep per-agent reasoning in adversarial POMDPs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.