Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead

2026-06-24 · Source: Towards Data Science · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, long

Summary

A single-agent LLM architecture for text-to-SQL applications often fails with complex queries requiring multiple operations and schema scans, as the agent struggles to parse intent, map schema, generate valid SQL, and validate its own output simultaneously. This leads to context bloat and self-contradiction after retries. A multi-agent pipeline, exemplified by a text-to-SQL system, addresses this by employing specialized agents for distinct tasks: Intent Parser, Schema Agent, Query Builder, Critic Agent, and Response Agent. These agents are coordinated by an orchestrator, often deterministic routing logic, and manage state explicitly using frameworks like LangGraph. The system incorporates a conditional retry loop with a "retry_count" ceiling of 3 attempts to prevent indefinite failures, though it introduces overhead, higher token costs, and more complex debugging compared to simpler single-agent solutions.

Key takeaway

For AI Engineers building complex LLM applications like text-to-SQL, consider adopting a multi-agent architecture to enhance reliability and performance. Your single-agent system will likely fail on intricate queries due to context overload; instead, specialize agents for distinct tasks and use a framework like LangGraph for robust orchestration and state management. Implement a dedicated critic agent and a retry loop with a defined ceiling to improve output quality and debug production issues effectively.

Key insights

Single LLM agents struggle with complex, multi-faceted tasks by attempting too many competing mental models in one context.

Principles

• Specialized agents improve system reliability.
• Fresh context prevents self-rationalization.
• Deterministic routing is often superior.

Method

Design specialized agents for distinct tasks (e.g., intent parsing, schema mapping, query building, critiquing), coordinate them with an orchestrator (like LangGraph), and implement conditional routing with retry logic.

In practice

• Use LangGraph for explicit state and routing.
• Implement a Critic Agent for independent validation.
• Add "failure_source" to pipeline state.

Topics

• Multi-Agent Systems
• Text-to-SQL
• LLM Architecture
• LangGraph
• Agent Orchestration
• Prompt Engineering
• Context Management

Best for: AI Engineer, Machine Learning Engineer, AI Architect

Related on AIssential

• Single Agent vs Multi-Agent: When to Build a Multi-Agent System — AI agents leverage LLMs, tools, and memory, often using the ReAct pattern, to perform complex tasks through reasoning and action.
• Multi-Agent Architecture with RAG and Dynamic Context Windows for Text-to-SQL Optimization — A multi-agent Text-to-SQL system uses RAG and dynamic context to reduce token consumption.
• AI Agents Don't Fail Because of the LLM. They Fail Because of the System Around It. — AI agent failures are primarily due to system design, not LLM limitations.
• AgentNLQ: A General-Purpose Agent for Natural Language to SQL — Multi-agent orchestration with enriched schema and execution feedback significantly improves NL2SQL accuracy and efficiency.
• LangChain's CEO argues that better models alone won't get your AI agent to production — Harness engineering extends context engineering to enable autonomous, long-running AI agents by giving LLMs more control.
• LLM Consortium for Software Design Refinement: A Controlled Experiment on Multi-Agent Collaboration Topologies — Structural adversarial and cross-model review topologies significantly enhance multi-agent LLM software design.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards Data Science.