LEAF: A Learning-Enabled ADMM Framework for Accelerated Convex Optimization

2026-06-08 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

LEAF is a novel learning-enabled ADMM framework for accelerated convex optimization. It approximates the objective function's Moreau envelope using an Input Convex Neural Network (ICNN), ensuring the learned model preserves convexity and smoothness. This yields Moreau Envelope Learning ADMM (MEL-ADMM) and its splitting variant, sMEL-ADMM. Unlike existing methods that learn high-dimensional operators, LEAF learns a scalar-valued Moreau envelope. This significantly reduces model complexity and improves data efficiency. The framework accommodates a broad class of convex problems, including those with smooth and non-smooth objectives. Explicitly embedding convexity through the ICNN architecture maintains high approximation accuracy and preserves structural properties. Both MEL-ADMM and sMEL-ADMM provide theoretical convergence guarantees and feasibility. They achieve convergence rates comparable to classical ADMM. Numerical experiments demonstrate up to an order-of-magnitude speedup over state-of-the-art solvers with low optimality gaps.

Key takeaway

For Machine Learning Engineers optimizing complex convex problems, LEAF offers a significant performance improvement. You should consider integrating MEL-ADMM or sMEL-ADMM into your workflows to achieve up to an order-of-magnitude speedup. This approach, which uses Input Convex Neural Networks, reduces model complexity and ensures theoretical convergence. It provides a robust alternative to traditional ADMM solvers. Evaluate its applicability for your specific smooth and non-smooth objective functions to enhance computational efficiency.

Key insights

LEAF uses ICNNs to learn a scalar Moreau envelope, accelerating convex optimization with reduced complexity.

Principles

• Learning a scalar Moreau envelope reduces model complexity.
• ICNNs preserve convexity and smoothness in learned models.
• Theoretical guarantees ensure convergence and feasibility.

Method

LEAF approximates the Moreau envelope of the objective function using an Input Convex Neural Network (ICNN). This results in Moreau Envelope Learning ADMM (MEL-ADMM) and its splitting variant, sMEL-ADMM, which maintain convexity and smoothness.

In practice

• Apply ICNNs to preserve convexity in learned models.
• Use MEL-ADMM for faster convex optimization.
• Reduce computational cost in ADMM-based solvers.

Topics

• LEAF Framework
• ADMM
• Convex Optimization
• Input Convex Neural Networks
• Moreau Envelope
• Machine Learning

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Learning Over-Relaxation Policies for ADMM with Convergence Guarantees — Learning ADMM relaxation parameters online improves performance without costly matrix refactorizations.
• A frequency analysis of filterbank initialisation and noise augmentation for LEAF — LEAF filterbanks show high inertia, resisting adaptation and converging to a fixed S-curve regardless of input frequency content.
• Efficient Test-Time Finetuning of LLMs via Convex Reconstruction and Gradient Caching — HullFT uses convex reconstruction and gradient caching to make test-time finetuning of LLMs faster and more efficient.
• Toward Reliable Tea Leaf Disease Diagnosis Using Deep Learning Model: Enhancing Robustness With Explainable AI and Adversarial Training — Deep learning with adversarial training and XAI enhances tea leaf disease detection robustness and interpretability.
• Stein Variational Black-Box Combinatorial Optimization — SVGD-EDA uses kernel-induced repulsion to prevent premature convergence in black-box combinatorial optimization.
• Gradient Regularized Newton Boosting Trees with Global Convergence — A gradient-regularized Newton boosting scheme achieves global convergence for GBDTs, addressing prior theoretical gaps.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.