Expert-Guided Schema-Based Structured Extraction from CONSORT Diagrams Using Vision-Language Models

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Advanced, quick

Summary

A study investigates structured information extraction from CONSORT flow diagrams, which summarize participant screening and analysis in randomized controlled trials, using Vision-Language Models (VLMs). Researchers introduced a 200-example benchmark derived from PubMed Central diagrams, meticulously annotated by a biomedical team specializing in systematic literature reviews. The evaluation compared single-pass and stepwise extraction strategies for schema-constrained CONSORT data across various proprietary and open-weight VLM families. Expert-guided single-pass extraction demonstrated superior performance with proprietary frontier models, with Gemini 3 Pro achieving the strongest overall results. Conversely, stepwise prompting proved more effective for improving less capable open-weight models, particularly for challenging arm-level extraction tasks.

Key takeaway

For Machine Learning Engineers developing structured information extraction systems from scientific diagrams, consider that proprietary frontier models like Gemini 3 Pro offer superior performance with expert-guided single-pass extraction. If working with open-weight VLMs, implement stepwise prompting, especially for intricate tasks like arm-level extraction, to significantly improve their capabilities. This approach can enhance the accuracy of automated systematic literature reviews and clinical evidence extraction.

Key insights

VLMs can extract structured data from complex scientific diagrams, but challenges remain.

Principles

• Schema-constrained extraction is feasible.
• Expert guidance improves VLM performance.
• Stepwise prompting aids less capable models.

Method

The study evaluated schema-constrained CONSORT extraction using single-pass and stepwise strategies on a 200-example PubMed Central benchmark, comparing proprietary and open-weight VLMs.

In practice

• Use Gemini 3 Pro for high-quality extraction.
• Apply stepwise prompting for open-weight VLMs.
• Annotate diagrams for VLM training/evaluation.

Topics

• Vision-Language Models
• CONSORT Diagrams
• Structured Information Extraction
• Biomedical Literature Review
• Gemini 3 Pro
• Schema-Constrained Extraction

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Intelligent Healthcare Imaging Platform: A VLM-Based Framework for Automated Medical Image Analysis and Clinical Report Generation — A VLM-based framework automates medical image analysis and clinical report generation across multiple imaging modalities.
• OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models — Medical LVLMs require reasoning grounded in visual evidence and clinical knowledge, not just correct answers.
• Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding — MLLMs struggle with fine-grained medical video interpretation, but domain-specific methods significantly boost diagnostic accuracy for seizure semiology.
• Bridging the interpretability gap for medical artificial intelligence models using class-association manifold learning — CAML enhances medical AI explainability by decoupling decision patterns and visualizing global class-associated knowledge.
• ExtractBench: A Benchmark and Evaluation Methodology for Complex Structured Extraction — LLMs struggle with complex, enterprise-scale PDF-to-JSON extraction, especially as schema breadth increases.
• TAI #193: Gemini 3.1 Pro Takes the Benchmarks Crown, but Can it Catch Up in the Tools Race? — Raw model intelligence does not always equate to practical utility in real-world applications.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.