Context Engineering for RAG Question Parsing: From a Raw Question to Typed Fields That Steer Retrieval and Generation

2026-07-16 · Source: Towards Data Science · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Data Science & Analytics · Depth: Advanced, long

Summary

This article re-examines question parsing within Retrieval Augmented Generation (RAG) through the lens of "context engineering," a term popularized by Tobi Lütke and Andrej Karpathy in mid-2025. It posits that the user's raw question is critical context for the Large Language Model (LLM) and requires structured treatment, similar to retrieved documents. The question parser acts as both a consumer and writer of typed context, generating a "ParsedQuestion" row with fields such as "keywords", "intent", "pages_hint", "section_hint", and "layout_hint". These structured outputs directly steer downstream RAG components like retrieval and generation. The article maps this process to LangChain's four canonical context-engineering strategies: "write" (the "ParsedQuestion" contract), "compress" (the retrieval brief), "select" (the generation brief), and "isolate" (the clarification request). This approach ensures precise control over how question signals influence the RAG pipeline, preventing accidental couplings and improving reproducibility.

Key takeaway

For AI Engineers building robust RAG pipelines, prioritize implementing a dedicated question parsing brick. Your system should transform raw user questions into structured, typed fields like "keywords", "intent", and "section_hint" to precisely steer downstream retrieval and generation. This approach, aligned with "context engineering," prevents common RAG failures by ensuring each component receives only the relevant, pre-processed context it needs, improving accuracy and auditability. Avoid merging these distinct pieces to maintain modularity and efficient caching.

Key insights

Structured question parsing transforms raw user queries into typed context, precisely steering RAG retrieval and generation.

Principles

• Treat user questions as typed context.
• Each parsed field must have a downstream receiver.
• Separate payloads prevent accidental coupling.

Method

A question parser uses a fixed system prompt, "DocContext" (100 tokens), few-shot examples, and the raw question to produce a "ParsedQuestion" row, which then generates specialized briefs or a "ClarificationRequest".

In practice

• Map "keywords" to retrieval detectors.
• Use "intent" for model tier dispatch.
• Filter retrieval with "section_hint" or "pages_hint".

Topics

• Context Engineering
• Retrieval-Augmented Generation
• Question Parsing
• LLM Pipelines
• LangChain
• Typed Schemas

Code references

• doc-intel/notebooks-vol1

Best for: AI Engineer, Machine Learning Engineer, AI Architect

Related on AIssential

• Context Engineering for RAG : The Four Typed Inputs Behind Every RAG Answer — Context engineering systematically assembles typed inputs for LLMs, enhancing RAG auditability and cost-efficiency.
• The Untaught Lessons of RAG Question Parsing: Structure Before You Search — Structuring RAG user questions into a relational schema with typed columns prevents silent partial answers and enhances system auditability.
• Exploring Retrieval Augmented Generation Approaches for Natural Language Question Understanding — RAG with LLMs can effectively interpret natural language questions for adaptive AI explanations.
• Your RAG Pipeline Is Probably Useless. Here’s a Better Alternative — RAG pipelines fail predictably; alternatives like long-context, summarization, structured, or graph-based retrieval offer better solutions.
• Understanding Retrieval Augmented Generation (RAG): End-to-End Explained — RAG combines LLM reasoning with real-time data retrieval to produce accurate, context-aware, and trustworthy AI responses.
• Long Context Is Not Replacing RAG - It Is Forcing Us to Rethink Context Engineering — Long context models enhance, rather than replace, RAG, demanding advanced context engineering.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards Data Science.