PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates

2026-06-15 · Source: Machine Learning · Field: Science & Research — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences, Engineering & Applied Sciences · Depth: Expert, quick

Summary

PhysGuard is a novel physics-preserving framework designed to enhance the sim-to-real adaptation of neural operator models, which often lose accuracy when applied to experimental measurements. Unlike standard fine-tuning, which can damage core physics-relevant representations learned during pretraining, PhysGuard identifies physics-critical parameter directions using the empirical Fisher Information Matrix computed on simulation data. It then restricts fine-tuning updates to avoid interfering with these essential directions. The framework employs a layer-wise Gram-matrix formulation for efficiency with large models and an adaptive threshold to determine the protected subspace size. Experiments across four neural operator architectures and various physical systems demonstrate PhysGuard's strong performance, particularly under severe domain shift, where it reduces low-frequency error by up to 32% compared to standard fine-tuning while maintaining adaptability. The code is available at https://github.com/ZhouChaunge/PhysGuard.

Key takeaway

For Machine Learning Engineers adapting neural operator models from simulation to real-world data, standard fine-tuning risks damaging crucial physics-relevant representations. You should consider implementing PhysGuard's Fisher-guided gradient projection to preserve core physical structures during adaptation. This approach significantly reduces low-frequency error, especially under severe domain shift, ensuring your models maintain accuracy and physical consistency when deployed with experimental measurements.

Key insights

PhysGuard protects physics-critical neural operator parameters during sim-to-real adaptation by restricting fine-tuning updates based on Fisher Information.

Principles

• Physics-critical parameters are identifiable via Fisher Information.
• Preserving low-frequency structures is key for sim-to-real accuracy.
• Restricting fine-tuning updates can prevent knowledge damage.

Method

Compute the empirical Fisher Information Matrix on simulation data to identify physics-critical parameter directions, then restrict fine-tuning updates to non-interfering directions using a layer-wise Gram-matrix and adaptive threshold.

In practice

• Apply Fisher-guided projection for neural operator fine-tuning.
• Prioritize preserving low-frequency output structures.
• Use PhysGuard's Gram-matrix formulation for large models.

Topics

• Neural Operators
• Sim-to-Real Transfer
• Fisher Information Matrix
• Physics-Informed Machine Learning
• Gradient Projection
• Domain Adaptation

Code references

• ZhouChaunge/PhysGuard

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Physics-Guided Dimension Reduction for Simulation-Free Operator Learning of Stiff Differential--Algebraic Systems — An extended Newton implicit layer enables simulation-free, accurate operator learning for stiff DAEs by enforcing constraints and reducing dimensions.
• Fisher-Geometric Diffusion in Stochastic Gradient Descent: Optimal Rates, Oracle Complexity, and Information-Theoretic Limits — Mini-batch noise in SGD possesses an intrinsic, loss-induced matrix geometry, not scalar variance.
• Retrievable Gradients: Continual Post-Training Without Cumulative Weight Drift — ReGrad enables drift-free continual post-training by treating gradients as retrievable, temporary knowledge units.
• A Conservation Law for Equilibrium Propagation and Coupled Learning — Physical learning methods like coupled learning and equilibrium propagation conserve a mass-like quantity, ensuring reliable convergence.
• AdaGrad-Diff: A New Version of the Adaptive Gradient Algorithm — AdaGrad-Diff improves optimization robustness by adapting stepsizes based on gradient differences, not just gradient magnitudes.
• Physics-Informed Neural Networks: A Didactic Derivation of the Complete Training Cycle — This guide demystifies PINN training by exposing the full algebraic cycle, from forward pass to gradient descent.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.