AI for Systems: Using LLMs to Optimize Database Query Execution

2026-04-03 · Source: Together AI | The AI Native Cloud - Together.ai · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, medium

Summary

A collaboration involving Stanford University, the University of Wisconsin–Madison, and Bauplan demonstrates that LLM-guided plan rewrites can significantly optimize database query execution without modifying the database engine. Traditional cost-based optimizers often struggle with attribute independence assumptions, leading to cardinality misestimates and suboptimal physical plans. The team introduced DBPlanBench, a harness for Apache DataFusion, exposing the physical operator graph to an LLM. This system employs a token-efficient JSON schema, reducing plan payload size by approximately 10x. The LLM generates targeted JSON Patches (RFC 6902) for localized edits like swapping join sides. This method achieved a 4.78x speedup in a TPC-DS derived query, cutting hash-table build time from 10.16 seconds to 0.41 seconds and memory from 3.3 GB to 411 MB. Median speedups on TPC-H and TPC-DS workloads were 1.1x to 1.2x, with 60.8% of queries optimized over 5%. An "optimize small, deploy large" workflow was validated, transferring SF3 optimizations to SF10 databases.

Key takeaway

For AI Engineers or Data Engineers optimizing database performance, consider integrating LLM-guided query plan optimization. This approach significantly improves query latency and resource consumption, especially for complex OLAP workloads where traditional optimizers struggle with semantic correlations. You should explore using JSON Patches for safe, localized plan modifications. Also, adopt an "optimize small, deploy large" strategy to efficiently transfer optimizations from development to production environments. This can yield substantial speedups and reduce manual tuning efforts.

Key insights

LLMs can semantically optimize database query plans by correcting cardinality misestimates without engine modification.

Principles

• Traditional optimizers fail on semantic correlations.
• Token-efficient serialization is crucial for LLM context.
• Iterative patching improves optimization over single-shot.

Method

DBPlanBench serializes physical plans into a token-efficient JSON schema, then an LLM generates JSON Patches (RFC 6902) for targeted edits, iteratively refining plans.

In practice

• Use JSON Patches for safe, localized plan edits.
• Optimize on smaller datasets, then transfer to production.
• Expose physical operator graphs to LLMs for semantic reasoning.

Topics

• LLM Optimization
• Database Query Optimization
• Apache DataFusion
• JSON Patches
• OLAP Workloads
• Performance Engineering
• Evolutionary Algorithms

Best for: Research Scientist, AI Scientist, AI Engineer, Data Engineer

Related on AIssential

• Towards Automated Knowledge Transfer in Evolutionary Multitasking via Large Language Models — LLMs can autonomously design efficient and effective knowledge transfer models for multi-task optimization.
• Are LLM agents good at join order optimization? — LLM agents can autonomously optimize database join orders offline, significantly improving query performance.
• Semantic Caching for LLMs: FastAPI, Redis, and Embeddings — Semantic caching reduces LLM costs and latency by reusing responses for semantically similar queries via a layered approach.
• Turing Award Winner: Postgres, Disagreeing with Google, Future Problems | Mike Stonebraker — Specialized database architectures and strong data integrity are crucial for performance and correctness, especially as applications become complex.
• Adaptive Test-Time Compute Allocation for Reasoning LLMs via Constrained Policy Optimization — Optimizing LLM test-time compute involves balancing accuracy and cost via a constrained policy.
• TAHOE: Text-to-SQL with Automated Hint Optimization from Experience — TAHOE optimizes Text-to-SQL LLM prompts by dynamically learning and applying error-driven hints from development and deployment experiences.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Together AI | The AI Native Cloud - Together.ai.