CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs

2026-06-25 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition, Health & Medical Research · Depth: Expert, quick

Summary

CORTEX (Clinically Organized Reasoning and sTructured EXplanation) is a new structured reasoning benchmark designed for 3D chest CT multimodal large language models (MLLMs). It addresses the challenge of interpreting and verifying free-form MLLM reasoning in medical imaging, where current datasets lack the diagnostic traces and patient history crucial for trustworthy diagnoses. CORTEX restores this missing reasoning by providing 76,177 validated traces, each following a four-stage radiologist workflow: task understanding, visual observation, diagnostic reasoning, and answer synthesis. These traces are generated using frontier large language models, then rigorously filtered and verified through automated rubric scoring and expert radiologist review, with the structure and rubrics developed in collaboration with clinicians. Built upon the CT-RATE dataset, CORTEX supports open-ended VQA, closed-ended VQA, and report generation, offering both structured supervision and a stage-level evaluation protocol for developing reliable 3D chest CT MLLMs. The dataset and evaluation code will be publicly available upon acceptance.

Key takeaway

For Machine Learning Engineers developing 3D chest CT MLLMs, CORTEX provides a critical resource for building more trustworthy models. You should integrate its 76,177 validated, four-stage diagnostic traces as structured supervision to train your models. Additionally, utilize the stage-level evaluation protocol and clinician-designed rubrics to rigorously verify your MLLM's reasoning, ensuring its diagnostic conclusions are traceable and clinically sound. This approach can significantly enhance model interpretability and reliability.

Key insights

CORTEX provides structured, verifiable reasoning traces for 3D chest CT MLLMs, mirroring clinical diagnostic workflows.

Principles

• Diagnostic reasoning benefits from a structured, multi-stage workflow.
• Trustworthy MLLMs require traceable evidence linking findings to conclusions.
• Clinician collaboration is vital for medical AI benchmark design.

Method

Generate four-stage diagnostic traces (task understanding, visual observation, diagnostic reasoning, answer synthesis) using frontier LLMs, then filter and verify via automated rubrics and expert review.

In practice

• Use CORTEX for training MLLMs on structured 3D chest CT reasoning.
• Evaluate MLLM diagnostic traces using stage-level rubrics.

Topics

• CORTEX Benchmark
• 3D Chest CT
• Medical MLLMs
• Structured Reasoning
• Diagnostic Workflow
• Medical Imaging

Best for: Computer Vision Engineer, AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Benchmarks and methods for 3D medical image retrieval — The 3D-MIR benchmark and methods advance medical image retrieval by providing standardized evaluation and multi-modal search strategies.
• Implementing trust in non-small cell lung cancer diagnosis with a conformalized uncertainty-aware AI framework — TRUECAM enhances pathology AI trustworthiness by integrating uncertainty quantification, data filtering, and error rate control.
• OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models — Medical LVLMs require reasoning grounded in visual evidence and clinical knowledge, not just correct answers.
• Enhancing Pathological VLMs with Cross-scale Reasoning — Explicit cross-scale reasoning training and a leakage-aware VQA benchmark enhance pathological VLM understanding across magnifications.
• When is 3D Worth It? A Resource-Performance Frontier for CNNs and Transformers in Lung CT — Input dimensionality significantly impacts model performance and stability in volumetric medical imaging.
• Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding — MLLMs struggle with fine-grained medical video interpretation, but domain-specific methods significantly boost diagnostic accuracy for seizure semiology.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.