Decompose, Retrieve, Cite: A RAG Pipeline for Structured Report Generation from Technical Documentation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, medium

Summary

A new Retrieval-Augmented Generation (RAG) system, "Decompose, Retrieve, Cite," addresses challenges in processing dense technical documentation, specifically for OpenFOAM, an open-source computational fluid dynamics toolkit. This system operates in two distinct modes. The single-query mode employs a formula-preserving parser named Marker, adaptive header-aware chunking, two-stage dense-then-rerank retrieval, and a citation-enforcement prompt to generate grounded, source-attributed answers for a 20-question benchmark. The report mode decomposes user prompts into sub-questions using LLM planning, performs independent retrieval and cross-encoder re-ranking for each, and then uses a long-context generation call to produce structured, multi-section reports with inline citations. Both pipelines achieved overall scores exceeding 4.6/5.0 and perfect citation correctness (5.0/5.0) on their respective benchmarks. The decomposed pipeline demonstrated superior robustness with a 90% judge success rate compared to 70% for the single-query mode. Analysis identified retrieval breadth, with an absolute recall below 14%, as the primary bottleneck.

Key takeaway

For Machine Learning Engineers developing RAG systems for dense technical documentation, you should consider implementing a multi-stage pipeline that includes query decomposition and specialized parsing. Your system's robustness and citation accuracy will significantly improve by adopting techniques like formula-preserving parsing and two-stage retrieval. Focus on enhancing retrieval breadth, as this remains a critical bottleneck, to maximize the utility of your generated reports and answers.

Key insights

A RAG system for technical docs uses prompt decomposition and specialized parsing to generate structured, cited reports with high accuracy.

Principles

• Formula-preserving parsing is crucial for technical RAG.
• Decomposing complex queries enhances RAG robustness.
• Two-stage retrieval improves grounding in dense docs.

Method

The system uses a formula-preserving parser, adaptive chunking, and two-stage retrieval. For reports, it decomposes prompts, re-ranks retrieved chunks, and generates multi-section output with inline citations.

In practice

• Generate structured reports from engineering manuals.
• Answer specific queries from complex documentation.
• Attribute RAG outputs directly to source documents.

Topics

• Retrieval-Augmented Generation
• Technical Documentation
• LLM Planning
• Information Retrieval
• OpenFOAM
• Structured Report Generation

Best for: AI Architect, AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Baseline Enterprise RAG, From PDF to Highlighted Answer — A minimal RAG pipeline can provide verifiable, sourced answers by structuring outputs and linking directly to document evidence.
• Retrieval-Augmented Generation as a Bridge Between Classical NLP Pipelines and Modern LLM Systems — Retrieval-augmented generation (RAG) addresses the limitation of static knowledge in large language models (LLMs) by connecting them to external, up-to-date knowledge sources.
• Why Naive Chunking Breaks RAG, and What to Build Instead — Structure-aware parsing and multimodal retrieval techniques are crucial for effective RAG on complex documents.
• Stop Returning Flat Text from a PDF: The Relational Tables RAG Needs — PDF parsing for RAG should produce a relational set of linked tables, not flat text, to preserve document structure.
• Your RAG Pipeline Is Probably Useless. Here’s a Better Alternative — RAG pipelines fail predictably; alternatives like long-context, summarization, structured, or graph-based retrieval offer better solutions.
• RAG in Production: Designing Retrieval Pipelines That Stay Accurate as Your Data Changes — Production RAG systems require dynamic, continuous retrieval pipelines to maintain accuracy with evolving data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.