Residual Skill Optimization for Text-to-SQL Ensembles

2026-05-22 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

DivSkill-SQL is a novel residual skill optimization framework designed to enhance Text-to-SQL ensembles by generating complementary SQL candidates. Unlike existing methods that rely on stochastic decoding or prompt variants, DivSkill-SQL optimizes each new "skill" (a high-level instruction prompt) on examples that the current ensemble fails to solve, directly targeting its marginal contribution to Pass@K. This approach avoids model fine-tuning and significantly improves selected accuracy on benchmarks like Spider2-Lite, achieving gains of up to +11.1 points on Snowflake and +8.3 points on BigQuery over the strongest ensemble baseline, CHASE-SQL, using models like Opus-4.6 and GPT-5.4. The framework also demonstrates strong transferability, with skills optimized on one SQL dialect (Snowflake) performing effectively across others (SQLite, BigQuery) and even generalizing to different task formulations such as BIRD-Critic, where it improved accuracy by +2.6 points. Error diagnostics reveal DivSkill-SQL reduces hallucinated schema references and function calls by up to 3x, indicating genuinely reliable complementary skills.

Key takeaway

For Machine Learning Engineers developing robust Text-to-SQL systems, DivSkill-SQL offers a method to significantly boost ensemble performance and reduce correlated failures. You should consider implementing residual skill optimization to create genuinely diverse agent behaviors, especially when working with complex SQL dialects or debugging tasks. This approach improves accuracy and reduces hallucinated outputs without requiring expensive model fine-tuning, making your systems more reliable and cost-efficient.

Key insights

DivSkill-SQL optimizes agent skills on residual failures to build complementary Text-to-SQL ensembles, improving Pass@K without model fine-tuning.

Principles

• Optimize new skills on prior failures.
• Target marginal Pass@K contribution.
• Complementary skills reduce correlated errors.

Method

DivSkill-SQL uses a batch-sequential process with LLM-assisted reflective prompt optimization (GEPA) to refine diverse seed skills. Each skill is optimized against the residual set of unsolved examples, then candidates are selected via pairwise comparison.

In practice

• Use GEPA for prompt optimization.
• Employ proxy models for cost-effective skill learning.
• Prioritize shorter prompts as a tiebreaker.

Topics

• Text-to-SQL
• Ensemble Learning
• Prompt Optimization
• Agentic AI
• SQL Dialects
• Pass@K Optimization

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• SIRIUS-SQL: Anchoring Multi-Candidate Text-to-SQL in Execution Feedback — Multi-candidate Text-to-SQL reliability improves by diversifying generation, targeting repairs, and hybrid selection.
• LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQL — Modular curriculum learning with tier-specific adapters improves LLM performance on complex Text-to-SQL tasks.
• Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards — Multi-turn reinforcement learning with progressive rewards and ODT feedback significantly improves Text-to-SQL generation.
• ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL — ReSkill integrates skill creation directly into policy optimization to develop reusable, generalizable strategies in agentic RL.
• HyperDFlash: MHC-Aligned Block Speculative Decoding with Gated Residual Reduction — HyperDFlash improves speculative decoding for MHC architectures by aligning drafting with the target model's native pathways.
• Textual Autograd Mechanics: Computation Graphs in Language Optimization — Textual Autograd Mechanics applies computation graphs to optimize language models by backpropagating text feedback.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.