Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?

2026-05-04 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

Multimodal Large Language Models (MLLMs) often fail to genuinely perceive personality, instead relying on superficial pattern matching, a phenomenon termed the "Prejudice Gap." A new task, Grounded Personality Reasoning (GPR), is introduced, requiring MLLMs to anchor Big Five personality ratings in observable behavioral evidence through a chain of rating, reasoning, and grounding. To evaluate this, the MM-OCEAN dataset was created, comprising 1,104 videos and 5,320 cue-grounding Multiple-Choice Questions (MCQs), developed via a multi-agent human-collaborative pipeline. Benchmarking 27 MLLMs revealed that 51% of correct ratings are ungrounded, with the Holistic-Grounding Rate spanning only 0-33.5%. This exposes a critical disconnect between achieving correct scores and providing evidence-based reasoning.

Key takeaway

For AI Scientists and Machine Learning Engineers developing human-centric MLLM applications, you must move beyond traditional numerical accuracy. Your models may achieve correct personality ratings without genuinely understanding the underlying behavioral evidence. Implement evaluation frameworks like Grounded Personality Reasoning (GPR) to assess whether your MLLMs can anchor judgments in observable cues. Prioritize improving fine-grained spatiotemporal grounding in model training to build truly trustworthy and explainable AI systems.

Key insights

MLLMs frequently "prejudge" personality without grounded behavioral evidence, revealing a critical gap in genuine understanding.

Principles

• Personality judgments require grounding in observable behavioral evidence.
• Trustworthy AI perception integrates specific behavioral micro-cues.
• Evaluating MLLMs needs a rating-reasoning-grounding chain.

Method

Evaluate MLLMs using Grounded Personality Reasoning (GPR) via a three-tier framework: ordinal rating, open-ended reasoning, and structured cue grounding, complemented by four sample-level failure-mode metrics.

In practice

• Prioritize fine-grained spatiotemporal grounding in MLLM post-training.
• Address visual-grounding bottlenecks like micro-expression localization.
• Develop MLLMs that genuinely understand, not merely judge, people.

Topics

• Multimodal Large Language Models
• Personality Perception
• Grounded Personality Reasoning
• MM-OCEAN Dataset
• AI Evaluation Benchmarks
• Spatiotemporal Grounding

Code references

• kkkcx/MM-OCEAN

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Ethicist

Related on AIssential

• ThinkDeception: A Progressive Reinforcement Learning Framework for Interpretable Multimodal Deception Detection — MLLMs and progressive reinforcement learning enable interpretable multimodal deception detection by modeling cognitive reasoning.
• Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning — Accuracy-only RLVR in medical VQA degrades visual grounding despite improving benchmark accuracy.
• Frozen Multimodal Embeddings for AI-Assisted Interview Assessment of Personality and Cognitive Ability — Frozen multimodal embeddings combined with trait-specific modeling significantly improve personality prediction in small-sample AVI assessments.
• MLLMs Get It Right, Then Get It Wrong: Tracing and Correcting Late-Layer Textual Bias — MLLMs often correctly process visual information in early layers, but a "late-layer textual override" bias shifts final predictions towards text.
• GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents — Multimodal perception must be a core component of agent reasoning, not merely an auxiliary interface.
• Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems — Effective abstention in multimodal systems requires abstention-aware training, not just better prompting or more agents.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.