The Untaught Lessons of RAG Question Parsing: Structure Before You Search

2026-07-02 · Source: Towards Data Science · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, long

Summary

This article, a companion to the Enterprise Document Intelligence series, details "question parsing" as a critical "brick 2" in a four-brick RAG architecture, arguing against treating user questions as simple search queries. It introduces a structured approach where user input is transformed into a question_df row with five typed columns: keywords, scope, shape, decomposition, and clarification. This structured data then generates two distinct briefs, RetrievalQuery and GenerationBrief, tailored for downstream retrieval and generation bricks. The method aims to eliminate silent partial answers and enhance auditability by replacing prompt-based logic with a relational schema and a deterministic dispatcher. Key lessons include using expert dictionaries for synonyms, handling four compound-question patterns (independent, sequential, unified, conditional), and sizing context windows in lines based on answer shape. Runnable companion notebooks are available on GitHub at doc-intel/notebooks-vol1.

Key takeaway

For AI Engineers building enterprise RAG systems, relying on verbatim user strings for retrieval often leads to silent partial answers and un-auditable behavior. You should implement a structured question parsing brick, transforming user input into a typed relational schema (question_df) to explicitly define question scope, shape, and decomposition. This approach enables precise context windowing and deterministic dispatching, significantly improving answer accuracy and system auditability for compliance.

Key insights

Structuring RAG user questions into a relational schema with typed columns prevents silent partial answers and enhances system auditability.

Principles

• Treat user questions as structured data, not raw strings.
• Design RAG question handling with schemas, not branching code.
• Use expert dictionaries for precise, auditable synonym mapping.

Method

Transform user questions into a question_df row with five typed columns (keywords, scope, shape, decomposition, clarification). Generate two briefs: RetrievalQuery and GenerationBrief. Size context windows in lines based on answer shape.

In practice

• Implement question_df with five typed columns.
• Maintain concept_keywords_df for expert-curated synonyms.
• Define four compound-question decomposition patterns.

Topics

• RAG Question Parsing
• Enterprise Document Intelligence
• Relational Schema Design
• Deterministic Dispatcher
• Context Windowing
• Expert Dictionaries

Code references

• doc-intel/notebooks-vol1

Best for: AI Engineer, Machine Learning Engineer, AI Architect

Related on AIssential

• Dispatching the Parsed RAG Question: Chunk Strategy, Model Tier, Activations, Audit — Effective RAG dispatch dynamically adapts chunking, model choice, and activations based on parsed questions and document profiles for auditability and cost efficiency.
• Retrieval Is Filtering, Not Search: A Mental Model for Enterprise RAG — RAG retrieval is filtering structured document tables, not free-text search, separating anchor detection from context expansion.
• How Fine-Grained Should a RAG Benchmark Be? A Hierarchical Framework for Synthetic Question Generation — Optimal RAG benchmark granularity varies by dimension, maximizing discriminative power for effective evaluation.
• Building a PDF Question-Answering Chatbot with Spring AI: From PDF Upload to RAG-Powered Answers — RAG systems enhance LLM accuracy by grounding answers in retrieved, relevant private data, outperforming larger models without context.
• Your RAG Pipeline Is Probably Useless. Here’s a Better Alternative — RAG pipelines fail predictably; alternatives like long-context, summarization, structured, or graph-based retrieval offer better solutions.
• RIP Chunking? Meet Reasoning-Based, Vectorless RAG. — PageIndex offers a vectorless RAG alternative that navigates document structure like a human expert, avoiding fixed-chunking issues.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards Data Science.