Deep Neural Networks as Discrete Dynamical Systems: Implications for Physics-Informed Learning

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Science & Research — Physical Sciences & Chemistry, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

A study inspects the analogy between transformer architectures without self-attention, termed "neural chains," and discrete dynamical systems derived from discretized neural integral and partial differential equations. Comparing Physics-Informed Neural Networks (PINNs) with standard finite-difference (FD) numerical discretization for solving 1D viscid and inviscid Burgers and Eikonal equations, the research finds both methods acquire essentially identical knowledge about system dynamics. PINNs achieve this through random matrices, which are far more numerous and easier to find than the unique, highly structured tridiagonal matrices of FD procedures. This approach, however, leads to a significantly larger number of parameters, reducing physical transparency and increasing training costs. For instance, the viscous Burgers equation was solved using a PINN with 7 hidden layers and 100 neurons each. While PINNs demonstrate robustness, they struggle with discontinuous initial conditions, as seen with the inviscid Burgers equation. The findings suggest a tradeoff between PINN's robustness and FD's explainability, though PINNs may offer advantages for high-dimensional problems.

Key takeaway

For research scientists evaluating numerical methods for solving PDEs, you should recognize that Physics-Informed Neural Networks (PINNs) and traditional finite-difference methods can yield equivalent solutions, but with different tradeoffs. While PINNs offer robustness and may be advantageous for high-dimensional problems, their reliance on numerous random parameters reduces explainability and increases computational costs. Conversely, finite-difference methods provide transparent, structured solutions. Consider the problem's dimensionality and the importance of interpretability when selecting your approach, especially for systems with discontinuities where PINNs may struggle.

Key insights

PINNs and finite-difference methods acquire similar knowledge via distinct matrix structures, revealing a robustness-explainability tradeoff.

Principles

• Neural chains map to discrete dynamical systems.
• PINNs and FD acquire similar knowledge differently.
• Random matrices offer robustness, structured matrices explainability.

Method

The paper compares numerical solutions of Burgers and Eikonal equations using standard FD discretization (cast as neural chains) and PINN learning, inspecting weight matrices and activation heatmaps.

In practice

• PINNs increase parameters, reducing explainability.
• Avoid PINNs for problems with sharp discontinuities.
• Explore PINNs for high-dimensional physics problems.

Topics

• Physics-Informed Neural Networks
• Discrete Dynamical Systems
• Neural Chains
• Finite Difference Methods
• PDE Solvers
• Model Explainability

Code references

• abhishekganguly808/neural-chains

Best for: AI Scientist, Research Scientist

Related on AIssential

• Differential Equations V — Physics Informed Neural Networks (PINNs) — Physics Informed Neural Networks embed physical laws into neural networks for fast, accurate, and physically consistent simulations.
• Overcoming the Limits of Finite Difference Method; Physics-Informed Neural Network for Noisy High-Dimensional Heat Diffusion — PINNs overcome classical numerical method limitations in noisy, high-dimensional heat diffusion, maintaining accuracy and efficiency.
• Physics-Informed Neural Embeddings of PDE Solution Families — A multihead PINN framework learns compact, robust embeddings of PDE solution families, revealing inherent dimensional reduction and reproducible spectral profiles.
• PINN loss functions: why physics-informed networks often fail to train — PINNs struggle to train due to severe gradient imbalance and scale differences among their multi-term loss functions.
• Transferable Physics-Informed Representations via Closed-Form Head Adaptation — Pi-PINN uses transferable representations and closed-form head adaptation for fast, accurate PDE solutions without extensive training data.
• A Machine Learning Approach to the Nirenberg Problem — A physics-informed neural network can effectively solve the Nirenberg problem and distinguish between realisable and non-realisable curvatures.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.