Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding

2026-05-12 · Source: cs.CV updates on arXiv.org · Field: Science & Research — Health & Medical Research, Mathematics & Computational Sciences, Research Methodology & Innovation · Depth: Expert, extended

Summary

Seizure-Semiology-Suite (S³), a new clinically grounded dataset and benchmark, addresses the gap in multimodal large language models' (MLLMs) ability to interpret complex seizure semiology. It comprises 438 seizure videos with over 35,000 dense labels for 20 ILAE-defined semiological features. The suite includes a seven-task hierarchical benchmark, evaluating MLLMs from visual perception to narrative report generation and seizure diagnosis, and introduces the Report Quality Index for Seizure Semiology (Seizure-RQI). Baseline evaluations of 11 open-weight MLLMs revealed systematic weaknesses in laterality reasoning, temporal localization, and clinically faithful reporting. However, seizure-specific fine-tuning substantially improved performance, and a two-stage neuro-symbolic framework achieved an F1 score of 0.96 for epileptic versus non-epileptic seizure classification, guiding reliable domain-adaptive multimodal intelligence.

Key takeaway

For AI Scientists developing MLLM-based diagnostic tools for medical video, recognize that general-purpose models currently struggle with fine-grained spatial and temporal clinical reasoning. You should implement a two-stage neuro-symbolic framework, combining MLLM feature extraction with a statistical classifier, and apply domain-specific fine-tuning. This approach achieves an F1 score of 0.96 for seizure classification, significantly enhancing diagnostic accuracy and reliability in safety-critical applications, thereby reducing labor-intensive manual review.

Key insights

MLLMs struggle with fine-grained medical video interpretation, but domain-specific methods significantly boost diagnostic accuracy for seizure semiology.

Principles

• General-purpose MLLMs exhibit systematic weaknesses in spatial and temporal reasoning for pathological motion.
• Domain-specific fine-tuning substantially improves MLLM performance in safety-critical medical video understanding.
• Decoupling visual perception from diagnostic reasoning via a neuro-symbolic framework enhances classification accuracy and trust.

Method

The two-stage seizure classification strategy uses MLLMs to extract 20 ILAE-defined semiological features into a structured binary vector, which is then fed into a Random Forest classifier for diagnosis.

In practice

• Apply a two-stage neuro-symbolic approach for high-stakes medical video classification tasks.
• Prioritize domain-specific fine-tuning over model scaling for MLLMs interpreting pathological movements.
• Utilize the Seizure-RQI for clinically grounded evaluation of narrative reports generated by medical MLLMs.

Topics

• Multimodal Large Language Models
• Seizure Semiology
• Clinical Video Analysis
• Neuro-symbolic AI
• Medical AI Evaluation
• Epilepsy Diagnosis

Code references

• LinaZhangUCLA/SeizureSemiologySuite

Best for: Computer Vision Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Predicting Post-Traumatic Epilepsy from Clinical Records using Large Language Model Embeddings — LLM embeddings from clinical records can predict post-traumatic epilepsy with high accuracy.
• Intelligent Healthcare Imaging Platform: A VLM-Based Framework for Automated Medical Image Analysis and Clinical Report Generation — A VLM-based framework automates medical image analysis and clinical report generation across multiple imaging modalities.
• CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs — CORTEX provides structured, verifiable reasoning traces for 3D chest CT MLLMs, mirroring clinical diagnostic workflows.
• SMART: A Flexible, Interpretable, and Scalable Spatio-temporal Brain Atlas from High-Resolution Imaging Data — SMART creates flexible, interpretable brain atlases by decoupling global disease dynamics from patient-specific anatomy using differential equations and Neural Cellular Automata.
• How Good LLMs Are at Answering Bangla Medical Visual Questions? Dataset and Benchmarking — BanglaMedVQA dataset reveals current LLMs and LVLMs perform poorly on medical visual questions in Bangla.
• Bridging the interpretability gap for medical artificial intelligence models using class-association manifold learning — CAML enhances medical AI explainability by decoupling decision patterns and visualizing global class-associated knowledge.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.