MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetings

2026-07-21 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

MeetingToM is a new benchmark designed to evaluate Multimodal Large Language Models (MLLMs) on Theory-of-Mind (ToM) reasoning within naturalistic multi-party meetings. Current MLLMs face significant challenges in inferring others' beliefs and intentions, particularly when social cues are distributed across speech and behavior. Unlike existing benchmarks that focus on overt, verifiable signals, MeetingToM specifically targets complex social behaviors such as pseudo-consensus, where social pressure leads to apparent agreement despite private dissent. The benchmark is hierarchically structured to assess ToM at three levels: subject-level mental state prediction, dyadic-level addressee understanding, and group-level consensus reasoning. Systematic analyses using a unified evaluation protocol revealed persistent limitations in representative MLLMs regarding non-verbal cue integration, hidden attitude inference, and distinguishing genuine from pseudo-consensus, establishing MeetingToM as a critical testbed for advancing meeting-grounded ToM capabilities.

Key takeaway

For Machine Learning Engineers developing MLLMs for social interaction or meeting analysis, you should recognize current models' significant limitations in Theory-of-Mind reasoning. Your development efforts must prioritize integrating non-verbal cues and inferring hidden attitudes to distinguish genuine consensus from pseudo-consensus. This benchmark highlights the need for advanced models capable of understanding complex group dynamics, guiding your focus towards more robust and socially intelligent MLLM architectures.

Key insights

MeetingToM evaluates MLLMs' Theory-of-Mind in multi-party settings, revealing limitations in complex social reasoning.

Principles

• ToM in MLLMs requires integrating distributed multimodal cues.
• Latent social states are crucial for robust ToM evaluation.
• Pseudo-consensus highlights MLLM social reasoning gaps.

Method

MeetingToM uses a hierarchical organization: subject-level mental state prediction, dyadic-level addressee understanding, and group-level consensus reasoning, with a unified evaluation protocol.

In practice

• Focus MLLM development on non-verbal cue integration.
• Prioritize models that infer hidden attitudes effectively.
• Develop MLLMs capable of distinguishing genuine from pseudo-consensus.

Topics

• Multimodal LLMs
• Theory of Mind
• Meeting Analysis
• Social Reasoning
• Pseudo-consensus
• Benchmark Evaluation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• PIVOTSBench: Evaluating Fine-Grained Interpersonal Relationship Reasoning in Multimodal Large Language Models — This benchmark assesses MLLMs' ability to reason about fine-grained, bidirectional interpersonal relationships using multimodal cues.
• What We are Missing in Multimodal LLM Evaluation? — Current MLLM evaluation benchmarks fail to assess cross-modal information integration, hindering true progress measurement.
• Beyond Sally-Anne: Evaluating Theory of Mind in LLMs using Epistemic Schelling Points — The Epistemic Asymmetry Schelling Task (EAST) reveals LLMs' significant functional social reasoning and epistemic tracking gaps.
• Reasoning for Mobile User Experience with Multimodal LLMs: Task, Benchmark, and Approach — MLLMs can diagnose complex UI/UX issues by combining task-adaptive rewards and reasoning efficiency penalties.
• LLM councils show groupthink — LLM councils, despite improving average answers, often suppress unique, high-value ideas due to a consensus bias.
• Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads — Multimodal LLMs exhibit functional sparsity, relying on a sparse subset of "CoRe heads" for efficient, precise cross-modal visual information retrieval.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.