Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict

2026-05-15 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

A new diagnostic framework, Context-Driven Decomposition (CDD), has been introduced to analyze how Retrieval-Augmented Generation (RAG) models handle knowledge conflicts between retrieved context and their internal parametric knowledge. CDD is an inference-time probe that elicits separate contextual and parametric answers, compares them, isolates conflicting premises, and records the resolution trace. This framework was tested across Epi-Scale stress tests, TruthfulQA misconception injection, and cross-model reruns on Gemini-2.5-Flash and Claude Haiku/Sonnet/Opus. The study found that standard RAG can exhibit a "Context-Compliance Regime" where retrieved context dominates even when incorrect, achieving only 15.0% accuracy on TruthfulQA misconception injection. CDD significantly improved robustness, reaching 71.3% on temporal shifts and 69.9% on distractor evidence, and its adversarial accuracy gains transferred across model families, though the causal coupling mechanism varied.

Key takeaway

For AI Architects and Research Scientists evaluating RAG system robustness, this research indicates that standard RAG can be highly susceptible to misinformation in retrieved contexts. Implementing Context-Driven Decomposition (CDD) can significantly improve accuracy and robustness under knowledge conflict, particularly against explicit factual manipulations and temporal shifts. You should consider integrating CDD's diagnostic approach to understand and mitigate context-compliance issues in your RAG deployments, especially when dealing with potentially stale or adversarial external knowledge sources.

Key insights

CDD diagnoses RAG's handling of knowledge conflicts, revealing context compliance and improving robustness across models.

Principles

• Context compliance is a measurable RAG behavior.
• Explicit conflict decomposition improves robustness.
• Accuracy gains can decouple from causal coupling across models.

Method

CDD uses a five-step reasoning trace: contextual extraction, parametric extraction, divergence check, premise isolation, and conflict resolution to make implicit conflict-resolution observable.

In practice

• Use CDD to diagnose RAG behavior under conflicting information.
• Apply CDD for improved robustness against adversarial contexts.
• Consider CDD-$alpha$ for compute-accuracy tradeoffs.

Topics

• Retrieval-Augmented Generation
• Context-Compliance Regime
• Context-Driven Decomposition
• Knowledge Conflict
• Epi-Scale Benchmark

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict — Context-Driven Decomposition (CDD) measures and improves RAG's ability to resolve knowledge conflicts between retrieved context and parametric knowledge.
• Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict — Context-Driven Decomposition (CDD) diagnoses how RAG models handle conflicting information between retrieved context and parametric knowledge.
• Your RAG System Retrieves the Right Data — But Still Produces Wrong Answers. Here’s Why (and How to Fix It). — RAG systems can confidently provide wrong answers when conflicting information is silently resolved during context assembly.
• FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG — Retrieval-memory conflict in RAG is token-level, requiring dynamic, deep-signal-driven contrastive decoding for faithfulness.
• ConflictRAG: Detecting and Resolving Knowledge Conflicts in Retrieval Augmented Generation — ConflictRAG systematically detects, classifies, and resolves knowledge conflicts in RAG systems, significantly improving answer correctness and reliability.
• Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checking — LLM verifiers in RAG-based fact-checking show unreliable arbitration between parametric priors and contextual evidence, necessitating better evaluation and methods.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.