Disco-RAG: Discourse-Aware Retrieval-Augmented Generation

2024-04-09 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, extended

Summary

Disco-RAG is a novel framework that enhances Retrieval-Augmented Generation (RAG) by explicitly incorporating discourse knowledge into the generation pipeline. Traditional RAG systems often struggle with fragmented evidence and lack of logical connections between retrieved chunks. Disco-RAG addresses this by constructing intra-chunk Rhetorical Structure Theory (RST) trees for local coherence and inter-chunk rhetorical graphs for global discourse, along with a discourse-aware planning module. Evaluated on three benchmarks (Loong, ASQA, and SciNews), Disco-RAG consistently outperforms standard RAG and state-of-the-art methods, achieving up to a 10.0 point gain in LLM Score on Loong. Ablation studies confirm the complementary contributions of each structural component, with RST trees and rhetorical graphs being most critical. The system demonstrates robustness to varying chunk sizes, retrieval noise, and maintains performance even with structural perturbations, indicating that its benefits stem from leveraging structural signals rather than merely adding tokens.

Key takeaway

For AI Engineers and Research Scientists developing RAG systems, incorporating discourse-aware modeling like Disco-RAG can significantly improve the factual consistency and coherence of generated outputs. You should consider integrating explicit rhetorical structure theory (RST) trees and inter-chunk rhetorical graphs, coupled with a discourse-driven planning module, to move beyond fragmented evidence and achieve more robust, logically organized responses, especially in knowledge-intensive QA and summarization tasks.

Key insights

Explicitly modeling discourse structure and planning significantly improves RAG system performance, coherence, and factual consistency.

Principles

• Discourse structure enhances LLM reasoning.
• Rhetorical planning guides narrative flow.
• Structural modeling improves noise robustness.

Method

Disco-RAG constructs intra-chunk RST trees and inter-chunk rhetorical graphs, then uses a discourse-driven planning module to guide LLM generation, moving beyond simple chunk concatenation.

In practice

• Use RST trees for intra-chunk coherence.
• Build rhetorical graphs for inter-chunk relations.
• Implement discourse-aware planning for complex answers.

Topics

• Retrieval-Augmented Generation
• Discourse Structure Modeling
• Rhetorical Structure Theory
• Large Language Models
• Knowledge-Intensive QA

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Structured Summaries for Retrieval-Augmented Generation in Portuguese-Language Consumer Complaints — Structured summaries significantly improve dense retrieval effectiveness in RAG systems for noisy, lengthy texts.
• A Better, Cheaper RAG (Neuro-Sym Multi-hop Reasoning) — TGS RAG fuses vector and graph search to overcome semantic drift and broken reasoning paths, reducing costs and improving accuracy.
• Narrative Knowledge Weaver: Narrative-Centric Retrieval-Augmented Reasoning for Long-Form Text Understanding — Narrative Knowledge Weaver (NKW) enhances RAG by modeling evolving story worlds through structured narrative assets and multi-channel evidence assembly.
• Self-Augmenting Retrieval for Diffusion Language Models — Low-confidence tokens in discrete diffusion models provide a useful lookahead signal for dynamic retrieval-augmented generation.
• RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora — Existing RAG benchmarks fail in redundant, real-world corpora, necessitating a new evaluation framework.
• Why Naive Chunking Breaks RAG, and What to Build Instead — Structure-aware parsing and multimodal retrieval techniques are crucial for effective RAG on complex documents.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.