Divide-then-Diagnose: Weaving Clinician-Inspired Contexts for Ultra-Long Capsule Endoscopy Videos

2026-04-23 · Source: Computer Vision and Pattern Recognition · Field: Health & Wellbeing — Medical Devices & Health Technology, Clinical Care & Medical Practice, Health & Medical Research · Depth: Expert, quick

Summary

A new task, diagnosis-driven capsule endoscopy (CE) video summarization, has been introduced to address the limitations of frame-level analysis in CE research. This task focuses on extracting key evidence frames that cover clinically meaningful findings and using them for accurate diagnoses from ultra-long CE videos. The challenge lies in the extreme sparsity of diagnostically relevant events amidst tens of thousands of redundant normal frames, compounded by issues like motion blur and debris. To support this, the VideoCAP dataset, the first CE dataset with diagnosis-driven annotations from real clinical reports, was created, comprising 240 full-length videos. The DiCE framework, inspired by clinical workflows, is proposed to tackle this task, outperforming existing methods by efficiently screening candidates, organizing them into diagnostic contexts, and aggregating multi-frame evidence.

Key takeaway

For AI Scientists developing medical imaging solutions, the introduction of diagnosis-driven CE video summarization and the VideoCAP dataset represents a significant shift. You should explore DiCE's clinician-inspired framework to improve diagnostic accuracy in ultra-long video analysis, moving beyond frame-level classification. Consider how contextual reasoning can enhance your models' ability to identify sparse, critical events in complex medical data.

Key insights

Diagnosis-driven video summarization extracts key evidence from ultra-long capsule endoscopy videos for accurate clinical diagnosis.

Principles

• Contextual reasoning improves diagnostic accuracy.
• Clinical workflow inspiration enhances AI framework design.

Method

DiCE screens video candidates, organizes them into diagnostic contexts using a Context Weaver, and aggregates multi-frame evidence with an Evidence Converger for robust clip-level judgments.

In practice

• Utilize VideoCAP for CE video summarization research.
• Implement DiCE's Context Weaver for sparse event detection.

Topics

• Capsule Endoscopy
• Video Summarization
• Diagnosis-Driven AI
• VideoCAP Dataset
• DiCE Framework

Best for: AI Scientist, Research Scientist, Computer Vision Engineer

Related on AIssential

• Computational Imaging Priors for Wireless Capsule Endoscopy: Monte Carlo-Guided Hemoglobin Mapping for Rare-Anomaly Detection — A Monte Carlo-inspired prior improves wireless capsule endoscopy classification by better distinguishing hemoglobin from confounding factors.
• From 'What' to 'How' and 'Why': Sharing LLM-Generated Retrospective Summaries of Older Adults' Passive Tracking Data with Remote Family Members — LLM-generated summaries for remote family members require context-aware narratives beyond raw data to be helpful and trusted.
• Training LLMs with Reinforcement Learning over Digital Twin Representations for Reasoning-Intensive Surgical VideoQA — Decoupling perception from reasoning in surgical video QA using RL-trained LLMs over digital twin representations improves multi-step reasoning.
• Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding — MLLMs struggle with fine-grained medical video interpretation, but domain-specific methods significantly boost diagnostic accuracy for seizure semiology.
• Active Evidence-Seeking and Diagnostic Reasoning in Large Language Models for Clinical Decision Support — LLMs exhibit a substantial performance gap in active, multi-turn clinical diagnosis compared to static, full-context evaluations.
• RARE disease detection from Capsule Endoscopic Videos based on Vision Transformers — Vision Transformers can be fine-tuned for multi-label classification of gastrointestinal diseases from capsule endoscopic videos.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.