Beyond Lexical Similarity: Evaluating Faithfulness in LLM-Based Medical Question Reformulation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Health & Wellbeing — Health & Medical Research, Medical Devices & Health Technology, Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

A new faithfulness metric, MedFaith-F1, has been introduced to evaluate large language models (LLMs) in medical query rewriting, a process crucial for health information retrieval. Standard metrics like ROUGE or BERTScore often fail to ensure the preservation of clinical content, leading to significant hallucinations. MedFaith-F1 assesses faithfulness across four clinically salient categories: diagnoses, medications, procedures, and follow-up intent. Researchers also proposed EKG-RAG, an Evidence and Knowledge-Grounded Retrieval-Augmented Generation framework. EKG-RAG combines hybrid retrieval from PubMed and MedlinePlus with UMLS-aligned ontology grounding. Evaluations on LLaMA-3 and Qwen2.5 using MeQSum and MQP datasets showed base models had category-level hallucination rates exceeding 40%. EKG-RAG, when combined with QLoRA, reduced this rate to 26.75% and achieved a MedFaith-F1 score of 0.73. These findings emphasize the need for faithfulness-aware evaluation in clinical query rewriting.

Key takeaway

For NLP Engineers developing medical information retrieval systems, relying solely on ROUGE or BERTScore for LLM evaluation is insufficient and risks high clinical hallucination rates. You should integrate faithfulness-aware metrics like MedFaith-F1 to accurately assess content preservation across critical medical categories. Consider implementing the EKG-RAG framework with QLoRA fine-tuning to significantly reduce hallucinations and improve the reliability of your medical query reformulation models. This approach ensures clinical accuracy, which is paramount in healthcare applications.

Key insights

Standard LLM metrics miss clinical hallucinations; new methods and metrics are needed for medical query faithfulness.

Principles

• Lexical similarity metrics are insufficient for clinical content.
• Faithfulness requires category-level evaluation.
• Knowledge grounding reduces medical LLM hallucinations.

Method

EKG-RAG is a hybrid framework combining retrieval from PubMed/MedlinePlus with UMLS-aligned ontology grounding. It enhances LLM faithfulness in medical query reformulation.

In practice

• Implement MedFaith-F1 for clinical LLM evaluation.
• Integrate EKG-RAG for medical query rewriting.
• Fine-tune LLMs with QLoRA for improved faithfulness.

Topics

• Medical Query Rewriting
• LLM Faithfulness
• Clinical Hallucinations
• MedFaith-F1 Metric
• EKG-RAG Framework
• UMLS Ontology

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Beyond Literal Summarization: Redefining Hallucination for Medical SOAP Note Evaluation — Lexical faithfulness in LLM evaluation misclassifies valid clinical reasoning as hallucination, inflating error rates.
• Beyond Semantic Similarity: A Component-Wise Evaluation Framework for Medical Question Answering Systems with Health Equity Implications — Current LLM medical QA evaluations overlook critical accuracy and health equity risks.
• FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG — FIDES resolves RAG retrieval-memory conflict by selectively applying contrastive decoding based on token-level conflict concentration.
• sebis at CRF Filling 2026: A Two-Stage Local LLM Pipeline for Medical CRF Filling — A two-stage local LLM pipeline using MedGemma-27B achieves competitive, privacy-preserving medical CRF filling with strict evidence adherence.
• Beyond the Blank Stare: Injecting Medical Expertise into LLMs Through RAG-Powered Knowledge — RAG systems enhance LLM reliability in medicine by grounding responses in verifiable, external knowledge bases.
• Cross-Lingual Empirical Evaluation of Large Language Models for Arabic Medical Tasks — English-centric LLMs exhibit significant performance gaps and reliability issues in Arabic medical tasks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.