Adversarial Robustness of NTK Neural Networks

2026-04-30 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

This 2026 paper, "Adversarial Robustness of NTK Neural Networks," theoretically analyzes the adversarial robustness of Neural Tangent Kernel (NTK) neural networks in nonparametric regression. The authors establish minimax optimal rates for adversarial regression in Sobolev spaces, demonstrating that NTK neural networks trained via gradient flow with early stopping can achieve these optimal rates. Conversely, the study proves that in the overfitting regime, the minimum norm interpolant becomes highly vulnerable to adversarial perturbations, with its adversarial risk diverging. Experimental results on 1D synthetic, real-world Diabetes, and high-dimensional synthetic datasets consistently show a U-shaped adversarial risk curve against training time, confirming the necessity of early stopping. The paper also explores randomized smoothing as a mitigation strategy, finding it effective in low-dimensional synthetic cases but less so in higher dimensions.

Key takeaway

For AI Scientists and Research Scientists developing or deploying deep learning models in safety-critical domains, this research underscores a critical trade-off: while overparameterized NTK networks can achieve optimal standard generalization, prolonged training to perfect data interpolation severely degrades adversarial robustness. You should prioritize early stopping or equivalent spectral regularization techniques to maintain adversarial stability, especially when dealing with potential adversarial attacks. Be cautious with full interpolation, as it can lead to diverging adversarial risk, even if $L^2$ consistency is achieved.

Key insights

Early stopping is crucial for adversarial robustness in NTK neural networks, as overfitting leads to vulnerability.

Principles

• NTK networks with early stopping achieve minimax optimal adversarial rates.
• Overfitting causes adversarial risk to diverge in minimum norm interpolants.
• Sobolev spaces are suitable for analyzing NTK kernel RKHS properties.

Method

The study establishes minimax optimal rates for adversarial regression in Sobolev spaces and designs an algorithm based on Lepski's method adaptive to unknown Sobolev index. It constructs NTK estimators and analyzes their gradient flow dynamics.

In practice

• Implement early stopping in NTK-based models for safety-critical applications.
• Avoid training NTK models to full interpolation to maintain robustness.
• Consider randomized smoothing for low-dimensional adversarial defense.

Topics

• Adversarial Robustness
• Neural Tangent Kernel
• Nonparametric Regression
• Sobolev Spaces
• Minimax Optimal Rates

Best for: AI Scientist, Research Scientist

Related on AIssential

• The Matching Principle: A Geometric Theory of Loss Functions for Nuisance-Robust Representation Learning — Robustness techniques converge on estimating and regularizing encoder Jacobians against label-preserving deployment nuisance covariance.
• Attacking the Spike: On the Transferability and Security of Spiking Neural Networks to Adversarial Examples — Adversarial attacks on SNNs are highly dependent on surrogate gradient estimators, and a new method, MDSE, exploits this for cross-model effectiveness.
• S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLP — S-GBT utilizes second-order sensitivity (curvature) to achieve tighter certified robustness against NLP word substitution attacks.
• Double descent for least-squares interpolation on contaminated data: A simulation study — The least-squares interpolator exhibits double descent and can outperform robust methods on contaminated data with high overparameterization.
• Optimal Stability of KL Divergence under Gaussian Perturbations — KL divergence stability under Gaussian perturbations extends to arbitrary distributions with optimal $\sqrt{\epsilon}$ degradation.
• GRAPE: Guided Parameter-Space Evolution for Compact Adversarial Robustness — Guided evolution of a neural network's parameter space during adversarial training can yield more compact and robust models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.