MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction

2026-07-23 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Data Science & Analytics · Depth: Expert, extended

Summary

MV-Bench is introduced as the first benchmark for evaluating Multimodal Large Language Models (MLLMs) on coordinated multi-view interface construction, a task demanding joint management of data semantics, multi-view coordination, and interactive logic. Addressing a gap in existing single-chart evaluations, MV-Bench utilizes Tableau workbook files as ground-truth, converting their specifications into 1,048 verified benchmark instances with executable code, rendered interfaces, datasets, and interaction annotations via a multi-stage pipeline. Evaluating five MLLMs (GLM 4.6V, Kimi K2.5, Qwen3.5 Plus, Claude Sonnet 4.5, GPT 5.4) in a single-pass setting, results show the strongest model achieves 75.45% accuracy in visual layout, but only 21.71% on data binding and 11.68% on interaction completeness. This indicates MLLMs struggle with functional aspects despite visual fidelity.

Key takeaway

For AI Scientists and ML Engineers developing MLLM-driven visualization tools, recognize that current models, despite strong visual layout reproduction (up to 75.45%), severely underperform on data binding (21.71%) and interaction completeness (11.68%). You must prioritize training data and evaluation metrics that explicitly address data semantics and cross-view interaction logic. Relying solely on visual fidelity will lead to deploying non-functional, visually deceptive interfaces. Focus on robust functional verification.

Key insights

MLLMs reproduce visual layouts well but fundamentally struggle with data semantics and interactive logic in multi-view interfaces.

Principles

• Visual fidelity is an unreliable proxy for functional correctness.
• Multi-view interfaces demand reasoning about shared state and data flow.
• Data binding and interaction are latent properties, hard to infer visually.

Method

A multi-stage pipeline transforms Tableau workbooks into executable web interfaces via Tableau Structured Specification (TSS) and Code Translation Specification (CTS), with iterative generation and verification.

In practice

• Verify MLLM-generated dashboards via code execution, not just visuals.
• Train MLLMs with interaction-aware strategies for causal structures.
• Utilize MV-Bench for iterative generation feedback.

Topics

• Multimodal LLMs
• Visualization Benchmarking
• Coordinated Multi-View Interfaces
• Data Binding
• Interaction Logic
• Image-to-Code Generation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction — MLLMs reproduce multi-view interface visuals well but fail significantly at generating correct data binding and interactive logic.
• MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs — MultiView-Bench reveals VLMs struggle with world-centric 3D spatial reasoning and multi-view integration, despite 2D proficiency.
• Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages — WebIGBench is the first benchmark to evaluate MLLMs on interactive webpage code generation, addressing gaps in existing static-focused evaluations.
• What We are Missing in Multimodal LLM Evaluation? — Current MLLM evaluation benchmarks fail to assess cross-modal information integration, hindering true progress measurement.
• Can Agents Read the Room? Benchmarking Visual Social Intelligence in Multimodal Simulation — Multimodal agents struggle with complex visual social cues for interaction regulation and outcome achievement.
• VCIFBench: Evaluating Complex Instruction Following for Video Understanding — MLLMs struggle with complex, constraint-rich video instruction following, highlighting a critical evaluation gap.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.