DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA

2026-07-15 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

DS@GT ARC submitted a corrective pipeline to the CLEF 2026 LongEval Task 4, focusing on Retrieval-Augmented Generation (RAG) systems for scientific question answering. Their work investigates the discrepancy between standard natural language evaluation metrics and the crucial aspect of citation integrity in RAG QA. The team benchmarked their pipeline, which incorporates Corrective RAG (CRAG) and CiteFix, against both baseline and frontier RAG models. While frontier models achieved high scores for answer relevance and fluency, RAGAs LLM-as-judge diagnostics revealed they often identified relevant documents without actually utilizing the context in their generated answers. DS@GT ARC's pipeline, by implementing pre-generation chunk filtering and post-generation strict entailment checks, demonstrated marginal improvements in citation faithfulness and answer grounding. The authors advocate for evaluation metrics that specifically reward strict answer grounding to ensure trustworthy RAG QA.

Key takeaway

For NLP Engineers developing or evaluating RAG QA systems, relying solely on traditional relevance and fluency metrics is insufficient for ensuring trustworthiness. Your evaluation strategy must incorporate explicit checks for citation integrity and strict answer grounding, as frontier models can appear performant without genuinely utilizing source context. Consider implementing pre-generation filtering and post-generation entailment checks to improve factual grounding and build more reliable RAG applications.

Key insights

Traditional RAG evaluation metrics often fail to assess citation integrity, necessitating new metrics that reward strict answer grounding.

Principles

• RAG evaluation needs citation integrity metrics.
• High relevance/fluency doesn't guarantee context use.
• Strict answer grounding is key for trustworthy RAG.

Method

A corrective pipeline filters chunks pre-generation and enforces strict entailment of claims to cited material post-generation to improve citation faithfulness and answer grounding in RAG QA.

In practice

• Implement pre-generation chunk filtering.
• Apply post-generation strict entailment checks.
• Use RAGAs LLM-as-judge for context use.

Topics

• Retrieval-Augmented Generation
• Scientific QA
• Citation Integrity
• Factual Grounding
• RAG Evaluation Metrics
• Corrective RAG

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• ProvenAI: Provenance-Native Traces of Evidence in Generated Answers — Meaningful transparency in retrieval-grounded QA requires independently measuring correctness, citation fidelity, and source influence.
• Efficiency vs. Verifiability in Evidence-Aware RAG: Does Prompt Compression Preserve Citation Grounding? — Prompt compression in RAG severely degrades citation grounding despite minor impacts on answer correctness.
• UNH @ Rag4Reports: A Broad Exploration of LLM-Judges for RAG — Citation faithfulness and internal knowledge verification are key to effective LLM-as-a-Judge RAG evaluation.
• Retrieval vs. Generation: The 7 Metrics That Actually Diagnose RAG — Diagnosing RAG systems requires separate evaluation of retrieval and generation layers using specific metrics, not just final output.
• Explicit Evidence Grounding via Structured Inline Citation Generation — FullCite improves LLM attribution by jointly generating document and evidence-span citations, revealing LLMs struggle with precise span localization.
• Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation — RAG systems can be factually accurate yet clinically wrong due to undetected entity misattribution.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.