MedFact: Benchmarking the Fact-Checking Capabilities of Large Language Models on Chinese Medical Texts

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Medical Devices & Health Technology · Depth: Expert, quick

Summary

MedFact is a new Chinese medical fact-checking benchmark designed to rigorously evaluate Large Language Models (LLMs) for medical applications, crucial for patient safety and regulatory compliance. It comprises 2,116 expert-annotated instances derived from diverse real-world texts, covering 13 medical specialties, 8 error types, 4 writing styles, and 5 difficulty levels. The benchmark's construction utilized a hybrid AI-human framework, employing iterative expert feedback to refine AI-driven, multi-criteria filtering for high quality. Evaluation of 20 leading LLMs revealed that while models can often classify text veracity, they struggle significantly with precise error localization, with even top performers failing to match human accuracy. A notable "over-criticism" phenomenon was observed, where LLMs incorrectly flag correct information, a tendency exacerbated by advanced reasoning techniques like multi-agent collaboration and inference-time scaling.

Key takeaway

For ML Engineers deploying LLMs in medical applications, you must prioritize robust error localization over mere veracity classification. Your current advanced reasoning techniques, like multi-agent collaboration, might exacerbate "over-criticism," where correct information is misidentified as erroneous. Focus development on mitigating this specific issue to ensure patient safety and regulatory compliance. You should also rigorously test models across diverse medical specialties and error types using benchmarks like MedFact.

Key insights

LLMs struggle with precise error localization and exhibit "over-criticism" on medical fact-checking, even with advanced reasoning.

Principles

• Medical LLM deployment requires rigorous fact-checking.
• Veracity classification differs from error localization.
• Advanced reasoning can worsen "over-criticism."

Method

MedFact was built using a hybrid AI-human framework, combining AI-driven multi-criteria filtering with iterative expert feedback to ensure high-quality, difficult instances across diverse medical texts.

In practice

• Develop LLMs for precise error localization.
• Address "over-criticism" in medical AI.
• Test LLMs across diverse medical specialties.

Topics

• MedFact
• Large Language Models
• Medical Fact-Checking
• Error Localization
• Over-criticism Phenomenon
• AI Benchmarking

Best for: NLP Engineer, AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Please don’t trust your chatbot for medical advice — LLMs are unreliable for medical advice, frequently generating misinformation with overconfidence and lacking clinical reasoning.
• Cross-Lingual Empirical Evaluation of Large Language Models for Arabic Medical Tasks — English-centric LLMs exhibit significant performance gaps and reliability issues in Arabic medical tasks.
• CRAFT: A Unified Counterfactual Reasoning Framework for Tabular Question Answering and Fact Verification — Counterfactual reasoning significantly improves LLM performance and robustness in complex tabular QA and fact verification.
• From Benchmarking to Reasoning: A Dual-Aspect, Large-Scale Evaluation of LLMs on Vietnamese Legal Text — LLMs struggle with legal reasoning and application, often masking critical errors despite high surface-level accuracy.
• Active Evidence-Seeking and Diagnostic Reasoning in Large Language Models for Clinical Decision Support — LLMs exhibit a substantial performance gap in active, multi-turn clinical diagnosis compared to static, full-context evaluations.
• ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning — ChatHealthAI integrates EHR representations with LLMs for grounded, interpretable clinical reasoning and accurate patient prediction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.