Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds

2026-06-18 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

Fisher width is introduced as a novel geometric complexity measure for statistical manifolds, addressing the limitations of the intrinsically Euclidean Gaussian width. This new measure incorporates the local Fisher information metric G(θ)¹/², making it sensitive to statistical curvature and invariant under smooth reparameterizations. The work develops its foundational theory, including concentration inequalities, metric perturbation stability, and spectral comparison bounds. A key application is a generalization bound for Fisher-Lipschitz hypothesis classes, scaling as ¹/√n. The paper also proposes computable estimators, which were empirically evaluated on MNIST across logistic, softmax, and ridge regression models. Experiments show Fisher width is significantly below the Euclidean baseline (e.g., 0.001 to 0.295 for binary logistic regression) and increases with regularization.

Key takeaway

For AI Scientists and Machine Learning Engineers quantifying model complexity or predicting generalization, Fisher width provides a statistically informed alternative to Euclidean Gaussian width. This metric accounts for the local statistical distinguishability of parameters, offering a more precise measure of effective model size on curved manifolds. You should consider using Fisher width to refine generalization bounds and evaluate model confidence, particularly in models like neural networks or exponential families where Euclidean assumptions fall short.

Key insights

Fisher width extends Gaussian width to statistical manifolds by incorporating the local Fisher information metric.

Principles

• Fisher width is invariant to smooth reparameterizations.
• Metric perturbation errors translate linearly to Fisher width.
• Generalization bounds are controlled by Fisher width.

Method

Fisher width is computed as the Gaussian width of the Fisher-rescaled set G(θ₀)¹/²T, estimable via Monte Carlo sampling with empirical Fisher matrices.

In practice

• Estimate Fisher width using empirical Fisher matrices.
• Employ low-rank approximations for large models.
• Use score-norm estimator for Euclidean ball parameter sets.

Topics

• Fisher Width
• Statistical Manifolds
• Fisher Information Metric
• Generalization Bounds
• Model Complexity
• Empirical Fisher

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Fisher-Geometric Diffusion in Stochastic Gradient Descent: Optimal Rates, Oracle Complexity, and Information-Theoretic Limits — Mini-batch noise in SGD possesses an intrinsic, loss-induced matrix geometry, not scalar variance.
• The Data Manifold under the Microscope — A new framework provides controlled benchmarks and accurate estimators for deep learning data manifold geometry.
• Separating Geometry from Probability in the Analysis of Generalization — Generalization can be analyzed deterministically via sensitivity of optimization solutions to data perturbations.
• Gating Enables Curvature: A Geometric Expressivity Gap in Attention — Multiplicative gating in attention enables non-flat geometric representations, enhancing expressivity for nonlinear tasks.
• Pointwise Generalization in Deep Neural Networks — Pointwise Riemannian Dimension explains deep network generalization by characterizing feature representation complexity.
• Fisher Information is Just Curvature — Fisher information measures log-likelihood curvature, directly correlating with estimator precision.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.