NM at CRF Filling 2026: A Two-Stage LLM Pipeline for Clinical CRF Population

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Medical Devices & Health Technology · Depth: Expert, quick

Summary

A two-stage LLM pipeline, developed by Niccolò Morabito for the CRF Filling Shared Task 2026, automates the population of Case Report Forms (CRFs) from clinical notes describing dyspnea patients. The initial stage utilizes a few-shot prompted LLM to extract candidate CRF fields into a structured JSON format. A subsequent, separate LLM then verifies each extracted field against the original note, explicitly removing predictions lacking textual evidence to reduce false positives. Experiments on the development set confirmed this verification step significantly improves macro F1 by preserving correct extractions while reducing unsupported ones. The system achieved a 0.56 macro F1 score on the official test set for both English and Italian.

Key takeaway

For NLP Engineers developing clinical data extraction systems, this two-stage LLM pipeline offers a robust approach to improve accuracy. If you are struggling with high false positive rates in CRF population, consider implementing a separate LLM-based verification stage. This method, which achieved a 0.56 macro F1 on both English and Italian, can significantly enhance precision by ensuring extracted fields are explicitly supported by source text, balancing recall with validation.

Key insights

Separating LLM-based extraction and verification stages significantly improves precision in clinical CRF population by reducing false positives.

Principles

• Two-stage LLM pipelines can balance recall and precision.
• Verification stages reduce false positives effectively.
• Explicit textual evidence is crucial for validation.

Method

A two-stage LLM pipeline: first, a few-shot LLM extracts candidate CRF fields into JSON; second, a separate LLM verifies each field against the original note, removing unsupported predictions.

In practice

• Implement a verification LLM for structured data extraction.
• Use few-shot prompting for initial field extraction.
• Output extracted data in JSON format.

Topics

• Clinical NLP
• Large Language Models
• Case Report Form (CRF) Filling
• Information Extraction
• Two-stage Pipeline
• Dyspnea Patient Notes

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• sebis at CRF Filling 2026: A Two-Stage Local LLM Pipeline for Medical CRF Filling — A two-stage local LLM pipeline using MedGemma-27B achieves competitive, privacy-preserving medical CRF filling with strict evidence adherence.
• Source-Grounded Data Generation for Text-to-JSON Learning — STAGE generates high-quality, validated text-to-JSON training data using LLMs and spreadsheets, significantly improving extraction accuracy.
• Measuring the sensitivity of LLM-based structured extraction to prompt, model, and schema choices in clinical discharge summaries — LLM-based clinical data extraction sensitivity varies significantly with model and schema, especially regarding "absence-versus-silence" distinctions.
• EDEN: A Large-Scale Corpus of Clinical Notes for Italian — EDEN provides the largest Italian clinical note corpus and a CRF-filling benchmark for LLM-based information extraction.
• Possible or Definite? A Benchmark for Evaluating Diagnostic Uncertainty Preservation in Clinical Text — LLMs systematically distort diagnostic uncertainty in clinical text, often converting nuanced statements into definite assertions.
• Cross-Lingual Empirical Evaluation of Large Language Models for Arabic Medical Tasks — English-centric LLMs exhibit significant performance gaps and reliability issues in Arabic medical tasks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.