Hyper Input Convex Neural Networks for Shape Constrained Learning and Optimal Transport

2026-04-30 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Life Sciences & Biology, Mathematics & Computational Sciences · Depth: Expert, short

Summary

Hyper Input Convex Neural Networks (HyCNNs) are a new neural network architecture introduced for learning convex functions, combining Maxout networks with Input Convex Neural Networks (ICNNs). Submitted on April 29, 2026, HyCNNs are designed to be inherently convex in their input, leverage depth effectively, and offer reliable performance at scale compared to traditional ICNNs. The authors prove that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions to a specified precision. Through synthetic experiments, HyCNNs demonstrated superior predictive performance over existing ICNNs and MLPs in convex regression and interpolation tasks. Furthermore, HyCNNs were successfully applied to learn high-dimensional optimal transport maps for synthetic data and single-cell RNA sequencing data, frequently outperforming ICNN-based neural optimal transport methods and other baselines across various settings.

Key takeaway

For machine learning engineers working with shape-constrained learning or optimal transport, HyCNNs offer a compelling alternative to traditional ICNNs. Your models could achieve better predictive performance and require significantly fewer parameters, especially for approximating quadratic functions. Consider integrating HyCNNs into your next project to improve efficiency and accuracy in tasks requiring convex function approximation, such as analyzing single-cell RNA sequencing data.

Key insights

HyCNNs offer a parameter-efficient, depth-leveraging architecture for learning convex functions, outperforming ICNNs.

Principles

• Combine Maxout with ICNNs for convexity.
• Depth can be leveraged reliably at scale.
• Exponential parameter reduction for quadratic approximation.

Method

HyCNNs integrate Maxout network principles with Input Convex Neural Networks (ICNNs) to ensure input convexity, enabling efficient learning of convex functions and high-dimensional optimal transport maps.

In practice

• Apply HyCNNs for convex regression tasks.
• Use HyCNNs for high-dimensional optimal transport.
• Consider HyCNNs for single-cell RNA sequencing data.

Topics

• Hyper Input Convex Neural Networks
• Input Convex Neural Networks
• Optimal Transport
• Convex Regression
• Shape Constrained Learning

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Hyper Input Convex Neural Networks for Shape Constrained Learning and Optimal Transport — HyCNNs offer a parameter-efficient, depth-leveraging architecture for learning convex functions, outperforming ICNNs.
• Generative Modeling on Metric Graphs via Neural Optimal Transport — A neural optimal transport framework enables continuous generative modeling on metric graphs by embedding, solving, and projecting.
• Your GFlowNet Secretly Learns an Optimal Transport Plan — Non-acyclic GFlowNets implicitly learn optimal transport plans with graph-induced shortest path costs.
• Neural-Symbolic Logic Query Answering in Non-Euclidean Space — Hyperbolic GNNs with learned curvature effectively capture hierarchical logic query structures for knowledge graph reasoning.
• The y=x Problem: Rewiring Transformers with Hyper Connections — Hyper Connections and mHC offer new connectivity schemes to stabilize and optimize deep neural network training.
• Convex training of Lipschitz-regularized shallow neural networks — A convex restriction efficiently solves non-convex Lipschitz-regularized shallow neural network training, enhancing adversarial robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.