SirenFNO: Efficient and Full Frequency Learning of Fourier Neural Operators

2026-06-09 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

SirenFNO is a new framework designed to overcome the spectral bias of Fourier neural operators (FNOs) towards low-frequency information, a limitation arising from frequency truncation used to maintain FNO learning efficiency. This novel approach leverages sinusoidal representation networks (SIRENs) to learn implicit neural representations and employs mode-wise kernel parameterization. SirenFNO learns a full-grid spectrum with a constant, discretization-independent parameter count, thereby eliminating the need for frequency truncation. The framework further incorporates functional tensor decompositions to enhance both parameter and learning efficiency. Empirical studies demonstrate that SirenFNO consistently outperforms FNO, achieving approximately 4 to 15 times parameter reductions while preserving discretization invariance. Its functional decomposition variants show even greater performance improvements, utilizing a maximum of 73 times fewer parameters across multiple PDE benchmarks.

Key takeaway

For Machine Learning Engineers developing neural operators for PDE solutions, SirenFNO offers a compelling alternative to traditional FNOs. You should consider adopting SirenFNO to overcome spectral bias and achieve significant parameter reductions, potentially up to 73 times with functional decompositions, without sacrificing discretization invariance. This allows for more efficient model deployment and training, especially for PDEs with high-frequency oscillations.

Key insights

SirenFNO uses SIRENs and mode-wise kernel parameterization to enable full-frequency learning in FNOs, significantly reducing parameters.

Principles

• FNOs suffer from spectral bias due to frequency truncation.
• SIRENs can learn full-grid spectra without truncation.
• Functional tensor decompositions enhance parameter efficiency.

Method

SirenFNO integrates SIRENs for implicit neural representations and mode-wise kernel parameterization to learn a full-grid spectrum, then extends with functional tensor decompositions.

In practice

• Achieve 4-15x parameter reduction over FNOs.
• Maintain discretization invariance in PDE solutions.
• Reduce parameters by up to 73x with decompositions.

Topics

• Fourier Neural Operators
• Sinusoidal Representation Networks
• Partial Differential Equations
• Neural Operators
• Spectral Bias
• Parameter Efficiency

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• JA-SIREN: Deterministic Initialization for Sinusoidal Networks via Spectral Matching — Deterministic initialization via spectral matching for sinusoidal networks significantly enhances INR reproducibility and performance.
• Isotropic Fourier Neural Operators — Isotropic Fourier Neural Operators improve FNOs by enforcing spatial symmetries, boosting performance and reducing parameters.
• Sequential Fitting: A Different Perspective on the Spectral Bias of Neural Networks — The spectral bias in MLPs stems from "sequential fitting" and "boundary effects," not solely frequency content.
• Fourier Neural Operators with rank-1 lattice points and hyperbolic cross — Replacing FNO's tensor product grids with rank-1 lattices and hyperbolic cross improves efficiency and accuracy using fewer resources.
• A Practitioner's Guide to Kolmogorov-Arnold Networks — KANs offer enhanced expressivity and interpretability over MLPs by using learnable univariate basis functions on network edges.
• A data-driven Fourier-mixture neural-network method for density estimation — A Fourier-trained neural network using Gaussian-Laplace mixtures estimates probability densities from empirical characteristic functions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.