Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization

2026-05-23 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

Text2Opt-Bench, a scalable benchmark of 12 solver-verified optimization problem categories, reveals that large language models (LLMs) are primarily bottlenecked by "binding"—grounding coefficients and parameters in concrete problem data—rather than "modeling" the optimization structure. The benchmark, spanning linear programs to multi-objective formulations with up to thousands of variables, shows accuracy collapses as instance data grows. For example, GPT-5-Nano's accuracy drops from 72% to 11% with increased data. A simple inference-time approach called BIND, which externalizes numeric data to structured files for programmatic binding, significantly improves performance: GPT-5-Nano from 59.1% to 82.4% and GPT-5 from 86.2% to 95.8%. Furthermore, training binding-specific models outperforms end-to-end supervised finetuning (SFT) and reinforcement learning (RL), with a 1.5B binding specialist matching a 7B end-to-end baseline.

Key takeaway

For AI Scientists and Machine Learning Engineers developing text-to-optimization solutions, you must address the "binding" bottleneck directly. Implement data externalization strategies, such as BIND, to offload numerical data transcription from your LLMs to structured files. This approach significantly boosts accuracy and efficiency, allowing your models to focus on complex modeling. Consider fine-tuning smaller, specialized binding models, as they demonstrate superior performance and parameter efficiency compared to larger, end-to-end systems for data-intensive optimization problems.

Key insights

LLMs' text-to-optimization performance is bottlenecked by accurately binding numerical data, not just modeling structure.

Principles

• Data externalization significantly improves LLM accuracy for numerical grounding.
• Specialized training for binding outperforms end-to-end methods.
• LLM context windows are effectively smaller for dense numerical tasks.

Method

BIND externalizes numeric data to structured JSON files, enabling programmatic binding rather than transcription from the prompt.

In practice

• Implement data externalization for LLM inputs with dense numerical parameters.
• Fine-tune specialized binding models for optimization tasks.

Topics

• Text-to-Optimization
• Large Language Models
• Operations Research
• Data Binding
• LLM Benchmarks
• Supervised Fine-tuning
• Gurobi

Best for: Research Scientist, AI Engineer, NLP Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data — Structured pedagogical formats and diverse outputs are key to high-quality synthetic pretraining data, not larger generator models.
• Inference Optimization in LLMs: A Systems View — Inference optimization for LLMs is a layered systems problem, not a single technique, where memory bandwidth is the primary constraint.
• Building an LLM from Scratch: The Foundational Layer and Structural Stability — LLM foundational layers convert text to contextualized numerical representations via embeddings, attention, and normalization.
• Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit — Latent Context Language Models compress LLM input context before decoding, significantly boosting speed and memory efficiency without major accuracy loss.
• How Large Language Models Are Actually Built, Everything I Learned From Stanford’s CS229 LLM… — Data, evaluation, and systems are the true differentiators in LLM quality, not just architecture.
• From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment — Efficient LLM alignment stems from synergistically identifying and training task-specific attention heads with high-affinity data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.