Natural Language to SQL Over an Unknown Schema

2026-07-10 · Source: NLP on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, long

Summary

An architecture for Natural Language to SQL (NL2SQL) over unknown schemas is presented, designed for operational admin tools where data correctness and safety are paramount. This system addresses challenges like resolving ambiguous natural language queries, preventing unsafe SQL execution against live production databases, and dynamically adapting to unique, evolving schemas without full context dumps. The proposed six-step pipeline involves routing and classifying queries, retrieving relevant schema slices via vector embeddings, generating constrained SQL with an escape hatch for unstructured questions, performing static validation, executing queries within read-only, rolled-back transactions with a single repair attempt, and synthesizing plain-language answers with token budget limits. This multi-layered approach prioritizes defense-in-depth and honest failure over confident fabrication, ensuring system safety and adaptability.

Key takeaway

For AI Architects or MLOps Engineers building natural language interfaces for critical operational databases, you must prioritize a defense-in-depth strategy. Implement a multi-stage pipeline that includes dynamic schema retrieval, constrained SQL generation, static validation, and read-only transaction execution. This approach ensures safety and honest failure, preventing dangerous queries or confident but incorrect answers from impacting live data. Focus on visible refusals and bounded retries rather than unbounded agentic loops.

Key insights

A multi-stage NL2SQL pipeline ensures safety and accuracy for operational databases by constraining model output and validating execution.

Principles

• Prioritize defense-in-depth over single-layer perfection.
• Opt for honest failure over confident fabrication.
• Adapt to schema changes via dynamic retrieval, not static prompts.

Method

The six-step pipeline involves routing, schema retrieval, constrained SQL generation with an escape hatch, static validation, read-only execution with repair, and token-budgeted answer synthesis.

In practice

• Implement static SQL checks before database interaction.
• Use read-only transactions for all generated queries.
• Embed schema views for dynamic, relevant context retrieval.

Topics

• Natural Language to SQL
• Database Security
• Retrieval-Augmented Generation
• Operational Databases
• LLM Safety
• Schema Management

Best for: AI Engineer, AI Architect, MLOps Engineer

Related on AIssential

• AgentNLQ: A General-Purpose Agent for Natural Language to SQL — Multi-agent orchestration with enriched schema and execution feedback significantly improves NL2SQL accuracy and efficiency.
• Routing End User Queries to Enterprise Databases — Routing natural language queries to enterprise databases requires structured reasoning beyond embeddings or direct LLM prompting.
• Querying an astronomical database using large language models: the ALeRCE text-to-SQL system — The ALeRCE text-to-SQL system uses LLMs and a step-by-step framework to query astronomical databases via natural language.
• Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrieval — The system automatically derives an executable schema from multi-source data to unify knowledge graph construction and multi-tool query retrieval.
• Benchmarking Local LLMs for Natural-Language-to-SQL Querying in Biopharmaceutical Manufacturing: An Empirical Benchmark on Consumer-Grade Hardware — Code-tuned general-purpose LLMs outperform domain-specific medical LLMs for T-SQL generation on manufacturing schemas.
• Multi-Agent Architecture with RAG and Dynamic Context Windows for Text-to-SQL Optimization — A multi-agent Text-to-SQL system uses RAG and dynamic context to reduce token consumption.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NLP on Medium.