VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing

2026-07-20 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Expert, extended

Summary

VCG-Bench is a new unified benchmark designed to evaluate Vision-Language Models' (VLMs) capabilities in structured diagram generation and editing. It addresses the limitations of pixel-based synthesis by introducing a "Diagram-as-Code" paradigm, leveraging mxGraph XML for precise control. The benchmark comprises a taxonomized dataset of 1,449 diverse diagrams spanning 6 domains and 15 sub-domains. It defines two core tasks: Vision-to-Code Generation (Task 1) and Instruction-based Code-to-Code Editing (Task 2). VCG-Bench employs a tailored evaluation protocol with multi-dimensional metrics, including mxGraph Execution Success Rate (ESR), Style Consistency Score (SCS), and XML Decomposed Requirements Following Ratio (XDRFR). Experimental results indicate that while current models struggle with Task 1's visual parsing and syntactic constraint satisfaction, they achieve high fidelity in Task 2's constrained XML editing, highlighting a significant vision-to-code vs. code-to-code gap.

Key takeaway

For AI Engineers evaluating Vision-Language Models for professional diagram workflows, recognize the significant gap between code-to-code editing and vision-to-code generation. While current models demonstrate high precision in modifying existing mxGraph XML, their ability to infer structured diagrams from raw images remains a major bottleneck. Prioritize VLM training and fine-tuning efforts on enhancing visual parsing and syntactic constraint satisfaction for structured outputs, rather than solely focusing on pixel-level fidelity. This will improve practical readiness for real-world engineering tasks.

Key insights

"Diagram-as-Code" with mxGraph XML enables precise, controllable generation and editing of structured visual content, surpassing pixel-based methods.

Principles

• Structured code enables precise diagram editing.
• Pixel-based generation limits editability.
• Visual parsing is a VLM bottleneck.

Method

VCG-Bench uses a "Data-Task-Evaluation" framework: curating 1,449 mxGraph XML diagrams, defining Vision-to-Code Generation and Code-to-Code Editing tasks, and employing multi-dimensional metrics like ESR, SCS, and XDRFR.

In practice

• Use mxGraph XML for structured diagrams.
• Evaluate VLMs on both generation and editing.
• Prioritize models with strong visual parsing.

Topics

• VCG-Bench
• Vision-Language Models
• mxGraph XML
• Structured Diagram Generation
• Code-to-Code Editing
• Multimodal Benchmarking

Code references

• OpenDCAI/Paper2Any

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• TurtleAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics — VLMs struggle with education-oriented visual programming, particularly spatial reasoning, but synthetic data fine-tuning can significantly improve performance.
• PairCoder++: Pair Programming as a Universal Paradigm for Verified Code-Driven Multimodal and Structured-Artifact Generation — Pair programming with agent-based review significantly improves code-driven artifact generation by leveraging toolchain feedback.
• 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code — 3DCodeBench benchmarks VLM agents for procedural 3D modeling, revealing API mismatch issues and the need for better data and iterative refinement.
• Symbolic Grounding Reveals Representational Bottlenecks in Abstract Visual Reasoning — Symbolic input dramatically improves abstract visual reasoning in LLMs, indicating representation, not reasoning, is the VLM bottleneck.
• MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction — MLLMs reproduce visual layouts well but fundamentally struggle with data semantics and interactive logic in multi-view interfaces.
• MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs — MultiView-Bench reveals VLMs struggle with world-centric 3D spatial reasoning and multi-view integration, despite 2D proficiency.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.