Batch Normalization for Neural Networks on Complex Domains

2026-05-04 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

This paper introduces a novel Batch Normalization (BN) layer designed for neural networks operating on complex domains, specifically the Siegel disk and the complex unit ball. The proposed BN layer extends existing Riemannian BN concepts by leveraging automorphisms and the Kobayashi pseudodistance to define essential operations like batch centering, biasing, and updating the running mean, particularly when closed-form geometric quantities (e.g., exponential/logarithmic maps) are unavailable. The method's efficacy is demonstrated through experiments on radar clutter classification, node classification, and action recognition tasks. Results show that the new BN layer, named SiegelNetBN and CBallNetBN for respective domains, consistently improves mean accuracy and training stability compared to baselines, including existing FC layers and other Riemannian BN layers, with improvements ranging from 1.15% to over 9% in various scenarios.

Key takeaway

For AI Scientists and Machine Learning Engineers developing neural networks on non-Euclidean data, particularly complex domains like Siegel spaces or the complex unit ball, integrating this novel Batch Normalization layer is crucial. Your models will likely achieve higher accuracy and improved training stability, especially in applications like radar clutter or node classification, by leveraging the proposed automorphism-based centering and Kobayashi pseudodistance-derived almost geodesics. Consider replacing traditional BN layers with this approach to enhance performance on complex-valued inputs.

Key insights

A novel Batch Normalization layer for complex domains enhances neural network stability and accuracy using automorphisms and Kobayashi pseudodistance.

Principles

• Automorphisms enable BN operations in complex domains.
• Kobayashi pseudodistance constructs effective "almost geodesics."
• BN layers improve accuracy and stability in complex domain NNs.

Method

The method computes a Fréchet mean, then uses domain-specific automorphisms for batch centering and biasing. Running mean updates rely on "almost geodesics" derived from the Kobayashi pseudodistance, particularly for Siegel disk and complex unit ball domains.

In practice

• Apply this BN layer to Siegel neural networks for radar clutter classification.
• Integrate this BN layer into hyperbolic neural networks for node classification.
• Use this BN layer to improve action recognition on complex data embeddings.

Topics

• Batch Normalization
• Complex Domains
• Siegel Disk
• Complex Unit Ball
• Kobayashi Pseudodistance

Code references

• GitZH-Chen/GyroBN
• zhiwu-huang/SPDNet
• GitZH-Chen/LieBN
• jjscc/GBWBN

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Neural Networks on Symmetric Spaces of Noncompact Type — A unified point-to-hyperplane distance enables novel neural network architectures on noncompact symmetric spaces.
• The Most Underrated Layer Inside Every AI Model — Normalization stabilizes neural network training by bounding activations and balancing gradients, preventing instability.
• Complex Layout Classification in the Wild: A Low-Resource Approach with Layout-Preserving Augmentations — Domain-aware augmentations significantly improve low-resource complex layout classification by focusing on global geometric arrangements.
• IEEE Transactions on Neural Networks and Learning Systems, Volume 37, Issue 3, March 2026 — The issue highlights advancements across neural networks, computer vision, and machine learning applications.
• Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration — Classical momentum acceleration in mini-batch SGD enables perfect parallelization proportional to mini-batch size.
• Certified and accurate computation of function space norms of deep neural networks — Certified function space norms for neural networks are achieved via interval arithmetic and adaptive quadrature.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.