OptSkills: Learning Generalizable Optimization Skills from Problem Archetypes via Cluster-Based Distillation

2026-05-28 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

OptSkills is an archetype-centric skill learning and reasoning agent system designed to enhance the generalization capabilities of Large Language Models (LLMs) in automated optimization. Addressing limitations of current methods, which struggle with narrative variations and adapting to new problem types, OptSkills employs a novel approach. It clusters optimization problems by their underlying archetypes rather than surface narratives to improve robust generalization. For in-distribution generalization, the system explores diverse modeling paradigms and solver configurations within each cluster, distilling successful trajectories into reusable workflow-level skills. Out-of-distribution generalization is achieved by refining existing skills or expanding the skill library with newly obtained trajectories. OptSkills achieves a state-of-the-art micro-averaged accuracy of 68.27% on diverse datasets. Furthermore, it reaches 26.91% accuracy on the challenging MIPLIB-NL benchmark, surpassing DeepSeek-V3.2-Thinking by 4.53%, and 72.79% on the OOD NLCO benchmark after learning on Nano-CO.

Key takeaway

For Machine Learning Engineers developing automated optimization systems with Large Language Models, OptSkills presents a critical advancement. Its archetype-centric skill learning significantly improves generalization, moving beyond superficial narrative variations. You should explore integrating similar cluster-based distillation and skill refinement techniques to enhance your system's robustness and adaptability to new problem types. This approach can lead to higher accuracy and more reliable performance across diverse optimization challenges.

Key insights

OptSkills learns generalizable optimization skills by clustering problem archetypes and distilling successful workflow trajectories.

Principles

• Cluster problems by underlying archetypes for robust generalization.
• Distill successful problem-solving trajectories into reusable workflow skills.
• Refine and expand skill libraries with newly obtained trajectories.

Method

OptSkills clusters problems by archetypes, explores diverse modeling and solver configurations, then distills successful trajectories into reusable workflow-level skills, refining them with new data.

In practice

• Apply archetype-based clustering for problem generalization.
• Distill successful solution workflows into reusable skills.
• Update skill libraries with new problem-solving data.

Topics

• Large Language Models
• Automated Optimization
• Skill Learning
• Cluster-Based Distillation
• Problem Archetypes
• Generalization

Code references

• fujiwaranoM0kou/OptSkills

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Automated Optimization Modeling via a Localizable Error-Driven Perspective — Localized error patterns in optimization modeling enable targeted data synthesis and refined post-training for LLMs.
• Dynamic Dual-Granularity Skill Bank for Agentic RL — D2Skill enhances agentic RL by dynamically managing dual-granularity skills for improved policy learning and error correction.
• Subject to: Susanne Heipcke — Combining diverse skills like languages and mathematics can lead to unique contributions in technical fields.
• DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models — DiffusionOPD unifies multi-task diffusion model training via online policy distillation, improving efficiency and performance.
• Can your AI agent actually learn from its mistakes or just keep repeating them? — SkillOpt systematically optimizes AI agent skills by treating them as trainable textual parameters, validated against held-out data.
• DiPRL: Learning Discrete Programmatic Policies via Architecture Entropy Regularization — DiPRL learns discrete programmatic policies directly, avoiding performance drops from post-hoc continuous-to-discrete conversion.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.