A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equation

2025-12-31 · Source: JMLR · Field: Science & Research — Mathematics & Computational Sciences, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Shu Liu, Stanley Osher, and Wuchen Li propose a scalable preconditioned primal-dual hybrid gradient (NPDG) algorithm for solving partial differential equations (PDEs) using adversarial neural network training. This method transforms PDEs into an inf-sup problem, which is then optimized via a Primal-Dual Hybrid Gradient (PDHG) algorithm. By introducing precondition operators, the algorithm achieves a natural gradient ascent-descent scheme, with natural gradients efficiently evaluated using the Krylov subspace method (MINRES). The paper establishes a posteriori convergence for time-continuous linear PDEs and a refined a priori convergence for elliptic equations with boundary loss. Tested on PDEs ranging from 1 to 50 dimensions, including linear and nonlinear elliptic, reaction-diffusion, and Monge-Ampere equations, the NPDG method demonstrates superior efficiency, robustness, stability, and accuracy compared to existing deep learning algorithms like PINNs, DeepRitz, and WANs using Adam or L-BFGS optimizers.

Key takeaway

For research scientists developing neural network-based PDE solvers, the Natural Primal-Dual Hybrid Gradient (NPDG) method offers a robust and accurate alternative. If you are struggling with stability or accuracy in high-dimensional or nonlinear PDE problems, consider integrating this preconditioned PDHG approach. Its demonstrated efficiency and superior convergence over PINNs, DeepRitz, and WANs suggest it can significantly improve your model's performance and reliability. Explore the provided GitHub code to implement and test this method.

Key insights

The NPDG method solves PDEs via an inf-sup problem and preconditioned PDHG, yielding robust, accurate neural network training.

Principles

• PDEs can be reformulated as inf-sup problems.
• Preconditioning proximal steps yields natural gradients.
• Lower-order operators simplify loss functionals.

Method

Reformulate PDEs as an inf-sup problem using a dual test function. Apply a preconditioned Primal-Dual Hybrid Gradient (PDHG) algorithm to derive a natural gradient ascent-descent scheme. Evaluate natural gradients efficiently via the Krylov subspace method (MINRES).

In practice

• Solves linear and nonlinear PDEs up to 50D.
• Outperforms PINNs, DeepRitz, and WANs.
• Code is available for implementation.

Topics

• Partial Differential Equations
• Neural Networks
• Primal-Dual Hybrid Gradient
• Adversarial Training
• Natural Gradient
• Scientific Machine Learning

Code references

• LSLSliushu/NPDG

Best for: AI Scientist, Research Scientist

Related on AIssential

• A Novel Tensor Product-Based Neural Network for Solving Partial Differential Equations — TPNet solves PDEs and approximates functions via a tensor-product basis and direct least-squares, bypassing gradient training.
• Communication-Efficient, 2D Parallel Stochastic Gradient Descent for Distributed-Memory Optimization — HybridSGD combines 1D s-step SGD and FedAvg via a 2D processor grid for communication-efficient distributed optimization.
• Natural gradient descent with momentum — Natural gradient descent with momentum improves optimization for nonlinear models by overcoming local minima and suboptimal directions.
• Functional Gradient Descent with Adaptive Representations — Adaptive representation in functional gradient descent ensures convergence to global minimizers by dynamically refining gradient approximations.
• PC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-Training — Polynomial preconditioning stabilizes LLM training by controlling weight singular values, incurring no inference overhead.
• Auxiliary Finite-Difference Residual-Gradient Regularization for PINNs — An auxiliary finite-difference term can effectively regularize PINN residual fields, improving accuracy in specific regions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.