Distribution Alignment for One-Shot Federated Learning via Optimal Transport

2026-06-15 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

SLOT-Align (Single-round, Learning-free Optimal Transport Alignment) is a novel geometry-aware feature harmonization framework for One-Shot Federated Learning (OSFL). OSFL addresses extreme communication regimes where clients interact with a server only once, amplifying issues from heterogeneous client data distributions, specifically joint domain and label shift. These shifts cause misaligned feature representations that iterative optimization cannot correct. Existing OSFL methods, relying on distillation, server-side generation, or ensemble-based aggregation, typically assume aligned representations or handle shifts separately. SLOT-Align uses a shared frozen encoder to extract compact feature statistics, constructs a global reference via Bures-Wasserstein barycenters, and aligns local representations using closed-form geodesic optimal transport maps. This computationally efficient method integrates with existing OSFL pipelines using frozen encoders without modifying their training. Experiments show consistent accuracy and robustness improvements under joint domain and label shift across multiple benchmarks, backbones, and OSFL methods.

Key takeaway

For Machine Learning Engineers deploying One-Shot Federated Learning in environments with significant client data heterogeneity, particularly joint domain and label shift, you should consider integrating SLOT-Align. This framework directly addresses misaligned feature representations without iterative optimization, offering a computationally efficient way to improve model accuracy and robustness. Its compatibility with existing frozen encoder OSFL pipelines means you can enhance performance without modifying core training procedures.

Key insights

SLOT-Align harmonizes heterogeneous client feature distributions in One-Shot Federated Learning using optimal transport for improved accuracy.

Principles

• Heterogeneous client data causes misaligned feature representations.
• Optimal transport can align feature distributions geometrically.
• A global reference improves local representation harmonization.

Method

SLOT-Align extracts feature statistics via a shared frozen encoder, builds a global reference using Bures-Wasserstein barycenters, then aligns local representations with closed-form geodesic optimal transport maps.

In practice

• Integrate with existing OSFL pipelines using frozen encoders.
• Apply to scenarios with joint domain and label shift.
• Improve robustness in single-round federated learning.

Topics

• One-Shot Federated Learning
• Optimal Transport
• Distribution Alignment
• Feature Harmonization
• Domain Shift
• Label Shift

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• CRAFT: Conflict-Resolved Aggregation for Federated Training — CRAFT resolves federated learning update conflicts via geometric projection, improving global accuracy and client fairness.
• Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization — Inverse entropic optimal transport provides a unified likelihood maximization framework for semi-supervised conditional distribution learning using mixed data.
• A Zero-shot Generalized Graph Anomaly Detection Framework via Node Reconstruction — AlignGAD enables zero-shot graph anomaly detection across diverse domains by unifying features and leveraging reconstruction discrepancy.
• Faster Rates For Federated Variational Inequalities — Improved algorithms and analysis yield faster convergence rates for federated stochastic variational inequalities.
• Social-JEPA: Emergent Geometric Isomorphism — Independently trained JEPA world models spontaneously develop linearly alignable latent spaces, enabling efficient knowledge transfer.
• FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment — FedMTFI improves heterogeneous federated learning by combining multi-teacher knowledge distillation with SHAP-based feature importance for enhanced accuracy and interpretability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.