Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks

2026-04-20 · Source: cs.AI updates on arXiv.org · Field: Science & Research — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

A new lightweight curvature-aware optimization framework is proposed to enhance Physics-Informed Neural Networks (PINNs) training, which often suffers from slow convergence and instability due to anisotropic loss landscapes. This framework augments existing first-order optimizers by incorporating an adaptive predictive correction based on secant information, using consecutive gradient differences as a proxy for local geometric change. A step-normalized secant curvature indicator controls the correction strength. The method is plug-and-play, computationally efficient, and compatible with optimizers like AdamW, SOAP, and Muon, without requiring explicit second-order matrix formation. Experiments on diverse PDE benchmarks, including the 10D heat equation, Gray–Scott system, Belousov–Zhabotinsky system, and 2D Kuramoto–Sivashinsky system, demonstrate consistent improvements in convergence speed, training stability, and solution accuracy.

Key takeaway

For AI Scientists and Machine Learning Engineers developing or deploying PINNs, integrating this curvature-aware optimization framework can substantially improve model training. Your PINNs will achieve faster convergence, enhanced stability, and higher solution accuracy, especially for challenging, stiff partial differential equations. Consider implementing the CA framework with your preferred first-order optimizer to mitigate common training pathologies and achieve more robust physical simulations.

Key insights

Curvature-aware optimization using secant information significantly improves PINN training stability and accuracy.

Principles

• PINN training difficulties stem from anisotropic loss landscapes.
• Consecutive gradient differences indicate local geometric change.
• Adaptive correction strength is crucial for varying local stiffness.

Method

The framework computes a secant curvature indicator from consecutive gradient differences and parameter displacements, then uses it to dynamically modulate a predictive correction applied to the base optimizer's gradient input.

In practice

• Augment AdamW, SOAP, or Muon with curvature-aware gating.
• Use EMA for gradient differences to improve robustness.
• Apply time-marching for stiff PDE systems like Gray-Scott.

Topics

• Physics-Informed Neural Networks
• Curvature-Aware Optimization
• Loss Landscape Geometry
• Secant-based Gating
• Gradient Differences

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks — A curvature-aware optimization framework improves PINN training by adaptively correcting first-order optimizers using secant information.
• Physics-Informed Neural Networks: A Didactic Derivation of the Complete Training Cycle — This guide demystifies PINN training by exposing the full algebraic cycle, from forward pass to gradient descent.
• Evolutionary Two-Stage Hyperparameter Optimization Strategies for Physics-Informed Neural Networks — Evolutionary two-stage hyperparameter optimization significantly enhances PINN accuracy and robustness under fixed computational budgets.
• Auxiliary Finite-Difference Residual-Gradient Regularization for PINNs — An auxiliary finite-difference term can effectively regularize PINN residual fields, improving accuracy in specific regions.
• Canonical Regularisation of Wide Feature-Learning Neural Networks — Traditional ridge regularization biases feature-learning networks due to curved parameter-space geometry, necessitating a geodesic-based approach.
• Robust Parameter and State Estimation in Multiscale Neuronal Systems Using Physics-Informed Neural Networks — PINNs offer robust parameter and state estimation in multiscale neuronal systems, overcoming limitations of traditional methods.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.