A Survey of Features Used for Representing Black-box Single-objective Continuous Optimization

2026-07-21 · Source: cs.NE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences, Data Science & Analytics · Depth: Expert, extended

Summary

This survey by Cenikj et al. provides an overview of meta-features used for automated algorithm selection in single-objective continuous black-box optimization. It categorizes features into problem landscape features, algorithm features, and trajectory-based features. Problem landscape features include high-level properties and low-level techniques like Exploratory Landscape Analysis (ELA), Topological Landscape Analysis (TLA), and deep learning-based methods (CNNs, Transformers, VAEs). Algorithm features are derived from source code, performance, or graph embeddings. Trajectory-based features capture problem-algorithm interaction through internal parameters, adaptive ELA, DynamoRep, Opt2Vec, iterative ELA, Local Optima Networks (LONs), and probing. The survey identifies limitations such as ELA's sensitivity to sampling and lack of invariance, computational demands for high-dimensional problems, and the poor generalizability of current ML models across diverse benchmark suites like BBOB and CEC. It suggests incorporating more diverse training data and continual learning.

Key takeaway

For AI Scientists and Machine Learning Engineers selecting algorithms for black-box optimization, you should prioritize developing robust meta-feature representations. Focus on features invariant to problem transformations, like TLA or Deep-ELA, to improve model generalizability beyond specific benchmarks. Consider integrating trajectory-based features for dynamic algorithm selection and explore continual learning frameworks to adapt models to evolving problem landscapes. This approach will enhance predictive accuracy and real-world applicability.

Key insights

Automated algorithm selection for black-box optimization relies on diverse meta-features, but current ML models struggle with generalization across varied problem instances.

Principles

• Algorithm selection benefits from complementary algorithm performance.
• Meta-features can represent problem landscapes, algorithms, or their interactions.
• Generalizability of ML models is crucial for real-world applicability.

Method

The article surveys various methods for constructing meta-features, including statistical analysis of problem landscapes (ELA, TLA), deep learning models (CNNs, Transformers), and extracting data from algorithm execution trajectories.

In practice

• Use ELA features for initial problem characterization.
• Explore deep learning features for high-dimensional problems.
• Incorporate trajectory-based features for dynamic algorithm selection.

Topics

• Algorithm Selection
• Black-box Optimization
• Meta-feature Engineering
• Exploratory Landscape Analysis
• Deep Learning Representations
• Continual Learning

Code references

• Nevergrad

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Stein Variational Black-Box Combinatorial Optimization — SVGD-EDA uses kernel-induced repulsion to prevent premature convergence in black-box combinatorial optimization.
• The Feature Selection Trap: Why ‘More Data’ Can Actively Hurt Your Machine Learning Model — Adding more features does not always improve ML model performance and can actively degrade it due to redundancy.
• Evolving Systems, Volume 17, Issue 2, June 2026 — The issue highlights diverse applications of evolving systems, metaheuristics, and deep learning across engineering, healthcare, and environmental domains.
• Meta-Learning Transformers to Improve In-Context Generalization — Training transformers on diverse small datasets improves in-context generalization and modularity over large, uncurated corpora.
• RASP-Tuner: Retrieval-Augmented Soft Prompts for Context-Aware Black-Box Optimization in Non-Stationary Environments — RASP-Tuner optimizes black-box systems in non-stationary environments using retrieval-augmented soft prompts for efficient context adaptation.
• Subject to: Philippe Toint — Optimization is a dynamic field, offering continuous opportunities for impactful research and application.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.NE updates on arXiv.org.