optimize_anything: A Universal API for Optimizing any Text Parameter

2026-05-20 · Source: cs.NE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

optimize_anything is a new LLM-based optimization system that unifies diverse problem-solving tasks under a single declarative API. It can optimize text artifacts across six fundamentally different domains, including code, prompts, agent architectures, numerical configurations, and images. The system achieves state-of-the-art results, such as nearly tripling Gemini Flash's ARC-AGI accuracy from 32.5% to 89.5%, cutting cloud costs by 40% with discovered scheduling algorithms, and generating CUDA kernels where 87% match or beat PyTorch baselines. It supports single-task, multi-task, and generalization modes, with multi-task search demonstrating cross-problem transfer benefits. A key feature is the use of actionable "Side Information" (SI) from evaluators, which provides diagnostic feedback to the LLM proposer, leading to 4-6x faster convergence and higher final performance compared to score-only feedback. The system is open-sourced as part of the GEPA project.

Key takeaway

For research scientists developing optimization solutions, optimize_anything demonstrates that a single LLM-based framework can outperform specialized tools across diverse domains. You should consider formulating your optimization problems as text artifact improvements, leveraging rich diagnostic feedback (Side Information) to guide LLM proposers, and exploring multi-task search for related problems to achieve faster convergence and superior results compared to single-task approaches.

Key insights

A unified LLM-based system optimizes diverse text artifacts across domains using diagnostic feedback and multi-task search.

Principles

• Text optimization is a general-purpose problem-solving paradigm.
• Actionable side information accelerates convergence and improves performance.
• Multi-task search enables cross-problem transfer of optimization patterns.

Method

The system takes a seed artifact and an evaluator returning a score and optional diagnostic feedback. An LLM proposer iteratively refines the artifact based on this feedback, using Pareto-based search across three optimization modes.

In practice

• Formulate optimization problems as improving text artifacts.
• Design evaluators to return rich diagnostic "Side Information" (SI).
• Utilize multi-task search for related problems to leverage cross-transfer.

Topics

• optimize_anything API
• LLM-based Text Optimization
• Side Information
• Multi-task Search
• Agent Architecture Search

Code references

• gepa-ai/gepa
• sigopt/evalset
• algorithmicsuperintelligence/openevolve
• gepa-ai/optimize-anything-artifact

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• AI for Systems: Using LLMs to Optimize Database Query Execution — LLMs can semantically optimize database query plans by correcting cardinality misestimates without engine modification.
• Textual Autograd Mechanics: Computation Graphs in Language Optimization — Textual Autograd Mechanics applies computation graphs to optimize language models by backpropagating text feedback.
• Controlled Self-Evolution for Algorithmic Code Optimization — Controlled Self-Evolution significantly boosts code optimization efficiency by guiding LLM-based iterative refinement.
• DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Models — DualOptim+ adaptively balances shared and decoupled optimizer states for superior LLM unlearning.
• Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning — AdaNAGED unifies gradient-free, adaptive, and geometry-aware optimization for memory-efficient LLM fine-tuning.
• Towards Automated Knowledge Transfer in Evolutionary Multitasking via Large Language Models — LLMs can autonomously design efficient and effective knowledge transfer models for multi-task optimization.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.NE updates on arXiv.org.