Seeing Is Not Sharing: Some Vision-Language Models Overestimate Common Ground in Asymmetric Dialogue

2026-07-20 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A study on Vision-Language Models (VLMs) reveals that some models struggle to distinguish between shared perception and established common ground in asymmetric dialogue. Researchers investigated this by formulating an interpretation-matching task using 13,077 annotated reference expressions from HCRC MapTask dialogues. They evaluated VLMs under controlled conditions, manipulating dialogue context and map-information access. Results indicate that providing authentic map images improves overall VLM performance but causes models to over-predict alignment, a bias also observed with textual map descriptions. This suggests the bias is driven by task-relevant map content, not the visual channel itself. While performance improves, accuracy on non-aligned cases degrades. Calibration analysis further shows models rely on static map cues rather than tracking grounding through dialogue history. This pattern was most evident in Qwen3-VL-8B-Instruct and four other models across two architecture families.

Key takeaway

For NLP Engineers developing or deploying Vision-Language Models in collaborative dialogue systems, you must account for the models' tendency to over-predict common ground. Your systems should not assume mutual understanding based solely on shared visual or textual content. Instead, explicitly design mechanisms for grounding through interaction and track dialogue history to establish true common ground, especially when accuracy on non-aligned cases is critical.

Key insights

Vision-language models often conflate potential common ground with established mutual understanding in dialogue.

Principles

• Shared perception does not guarantee shared interpretation.
• VLMs can over-predict alignment from static content.
• Dialogue history is crucial for true common ground.

Method

The study formulated an interpretation-matching task on 13,077 HCRC MapTask reference expressions, evaluating VLMs under controlled dialogue context and map-information access manipulations.

In practice

• Test VLM common ground with non-aligned cases.
• Prioritize dialogue history over static visual cues.
• Evaluate VLM calibration for alignment bias.

Topics

• Vision-Language Models
• Common Ground
• Asymmetric Dialogue
• HCRC MapTask
• Qwen3-VL-8B-Instruct
• Dialogue Systems
• Model Calibration

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Seeing Is Not Sharing: Some Vision-Language Models Overestimate Common Ground in Asymmetric Dialogue — Vision-language models often conflate potential common ground with established mutual understanding in dialogue.
• LVLMs and Humans Ground Differently in Referential Communication — LVLMs fail to build common ground and adapt communication in multi-turn referential tasks, unlike humans.
• Conversational Grounding in Large Language Models: Evaluation Methods, Challenges and Future Directions — Evaluating conversational grounding in LLMs requires both explicit and implicit methods, facing significant methodological challenges.
• MT-PingEval: Evaluating Multi-Turn Collaboration with Private Information Games — Language models struggle with multi-turn collaboration, often failing to improve over non-interactive baselines despite potential.
• From Propositional to Perceptual Asymmetry: Extending FPO to Asymmetric Partial Information Dialogue — Understanding dialogue friction in asymmetric information requires evaluating FPO from each participant's unique perspective, as an informed single viewpoint outperforms omniscient access.
• Rethinking Evaluation in Retrieval-Augmented Personalized Dialogue: A Cognitive and Linguistic Perspective — Surface-level metrics fail to evaluate personalized dialogue; human and LLM judgments align on deeper cognitive and linguistic qualities.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.