Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking

2026-03-03 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, extended

Summary

The Web-enhanced Knowledge Graph retrieval Fact-Checking agentic framework (WKGFC) addresses misinformation by integrating structured knowledge graphs (KGs) with unstructured web content for evidence retrieval. Developed by Shuzhi Gong, Richard Sinnott, and Jianzhong Qi in 2026, WKGFC utilizes an LLM-enabled agent to assess claims and retrieve relevant knowledge subgraphs, augmenting them with web content when KG evidence is insufficient. This process is formalized as a Markov Decision Process (MDP), where a reasoning LLM agent adaptively decides retrieval actions. The framework employs prompt optimization to fine-tune the agent's policy without modifying base LLM parameters. Extensive experiments across Wikipedia, web, and article summary datasets show WKGFC outperforms several advanced fact-checking methods by over 5% in balanced accuracy, achieving an overall 74.3% balanced accuracy, demonstrating the effectiveness of its knowledge-centric, multi-source approach.

Key takeaway

For research scientists developing robust fact-checking systems, WKGFC demonstrates that integrating knowledge graphs with web retrieval via an adaptive LLM agent significantly improves veracity prediction. You should consider adopting a Markov Decision Process framework for evidence acquisition and implement self-reflection with prompt optimization to enhance retrieval coordination and decision-making without costly model fine-tuning, especially for open-world scenarios with incomplete evidence.

Key insights

WKGFC unifies structured KGs and web evidence via an LLM agent for adaptive, self-improving fact-checking.

Principles

• Fact-checking is a sequential information retrieval process.
• Combine structured KGs with unstructured web evidence.
• Refine agent policy through self-reflection and prompt optimization.

Method

An LLM agent, operating as a POMDP, performs initial KG retrieval, then adaptively expands the KG or triggers web search, and finally makes a veracity verdict. Policy improvement occurs via self-reflection and prompt optimization using TextGrad.

In practice

• Use SpaCy for entity extraction from claims.
• Employ Wikidata APIs for entity mapping.
• Filter web passages with LLM for factual consistency.

Topics

• Fact-Checking
• Knowledge Graphs
• Retrieval-Augmented Generation
• LLM Agents
• Prompt Optimization

Best for: Research Scientist, AI Researcher, AI Scientist, NLP Engineer

Related on AIssential

• LLM-as-an-Investigator: Evidence-First Reasoning for Robust Interactive Problem Diagnosis — An evidence-first LLM agent improves problem diagnosis by iteratively questioning users to avoid premature conclusions and user-driven sycophancy.
• Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies — Formulating factuality checking as a true/false reading comprehension task with test-taking strategies improves LLM efficiency and accuracy.
• Agent Orchestration - LLM for Legal Metadata Extraction: A Comparative Analysis of Efficiency and Precision — Multi-agent LLM pipelines for legal metadata extraction offer conditional accuracy gains but increase complexity and cost.
• MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA — MAGE-RAG uses an adaptive evidence graph and query-time control for efficient multimodal RAG in long documents.
• ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models — Combining structured exploration, iterative refinement, explicit validation, and persistent memory significantly enhances LLM multi-step reasoning.
• MADRAG: Multi-Agent Debate with Retrieval-Augmented Generation for Training-Free Analytic Essay Scoring — Multi-agent debate with retrieval augmentation offers a training-free, reliable approach to analytic essay scoring, outperforming prompt-based LLMs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.