Auxiliary Finite-Difference Residual-Gradient Regularization for PINNs

2026-04-15 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, quick

Summary

A new hybrid design for Physics-informed neural networks (PINNs) introduces an auxiliary finite-difference (FD) residual-gradient regularizer. This method maintains the governing PDE residual as automatic-differentiation (AD) based, while the FD term weakly penalizes gradients of the sampled residual field, regularizing it without replacing the core PDE residual. The approach was evaluated in two stages: first, a Poisson benchmark compared it against a baseline PINN and an AD residual-gradient baseline, showing the FD regularizer reproduces the main effect of residual-gradient control with a trade-off between field accuracy and residual cleanliness. Second, a three-dimensional annular heat-conduction benchmark (PINN3D) used a body-fitted shell auxiliary grid near a wavy outer wall. In this application, the shell regularizer significantly improved outer-wall flux and boundary-condition behavior, reducing mean outer-wall BC RMSE from 1.22e-2 to 9.29e-4 and mean wall-flux RMSE from 9.21e-3 to 9.63e-4 over 100k epochs with a fixed shell weight of 5e-4 under the Kourkoutas-beta optimizer.

Key takeaway

For AI Scientists developing PINN models, consider integrating an auxiliary finite-difference residual-gradient regularizer, especially when specific physical quantities or boundary conditions are critical. Your models can achieve significantly improved accuracy in targeted regions, such as outer-wall flux, by aligning the regularizer with the quantity of interest. Experiment with a fixed shell weight of 5e-4 and the Kourkoutas-beta optimizer for reliable performance gains.

Key insights

An auxiliary finite-difference term can effectively regularize PINN residual fields, improving accuracy in specific regions.

Principles

• Hybrid PINNs can combine AD and FD.
• Targeted regularization improves specific quantities.

Method

The method uses an AD-based PDE residual with an auxiliary finite-difference term that penalizes gradients of the sampled residual field, applied as a body-fitted shell in 3D problems.

In practice

• Apply auxiliary FD regularization near critical boundaries.
• Use Kourkoutas-beta optimizer for robust benefits.

Topics

• Physics-informed Neural Networks
• Finite-Difference Regularization
• Residual-Gradient Regularization
• Automatic Differentiation
• Heat Conduction Modeling

Best for: AI Scientist, Research Scientist

Related on AIssential

• Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks — Curvature-aware optimization using secant information significantly improves PINN training stability and accuracy.
• Regularisation in Neural Networks — Regularization techniques are crucial for neural networks to prevent overfitting and improve generalization on unseen data.
• Statistical Learning Analysis of Physics-Informed Neural Networks — PINNs can be understood as a statistical learning problem minimizing KL divergence with physics as indirect data.
• Smoothness-Based Derandomization of PAC-Bayes Bounds — PAC-Bayes derandomization for smooth loss functions quantifies the cost of deterministic predictors using flatness metrics.
• Canonical Regularisation of Wide Feature-Learning Neural Networks — Traditional ridge regularization biases feature-learning networks due to curved parameter-space geometry, necessitating a geodesic-based approach.
• Physics-Guided Dimension Reduction for Simulation-Free Operator Learning of Stiff Differential--Algebraic Systems — An extended Newton implicit layer enables simulation-free, accurate operator learning for stiff DAEs by enforcing constraints and reducing dimensions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.