Geometric Gradient Rectification for Safe Open-Set Semi-Supervised Learning

2026-06-25 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition · Depth: Expert, quick

Summary

Geometric Gradient Rectification (GGR) is a novel plug-in framework designed for open-set semi-supervised learning (OSSL), addressing the challenge of leveraging unlabeled data that may contain out-of-distribution outliers. Traditional OSSL methods often struggle with a trade-off: aggressive filtering of suspicious samples can discard valuable in-distribution data, while incorporating potentially incorrect pseudo-labels can introduce auxiliary gradients that conflict with supervised learning. GGR shifts focus to gradient-level control, using the supervised gradient as an anchor. It projects conflicting auxiliary gradients onto an admissible region in gradient space, ensuring the applied auxiliary update is first-order non-opposing within the rectified coordinate block while preserving useful orthogonal components. The framework is further enhanced with subspace-aware rectification to stabilize the anchor against noisy mini-batch gradients. Experiments on CIFAR and ImageNet benchmarks demonstrate that GGR consistently improves representative OSSL baselines, yielding gains in both closed-set generalization and open-set robustness. Code will be available at https://github.com/JiaheChen2002/GGR.

Key takeaway

For Machine Learning Engineers developing open-set semi-supervised learning models, you should consider integrating Geometric Gradient Rectification (GGR) to enhance both closed-set generalization and open-set robustness. GGR's gradient-level control approach directly addresses the challenge of conflicting auxiliary gradients from unlabeled data, offering a more stable and effective way to utilize noisy datasets. Implement GGR as a plug-in to your existing OSSL baselines, especially when dealing with potential out-of-distribution outliers.

Key insights

GGR rectifies conflicting auxiliary gradients in OSSL by projecting them onto an admissible region, improving robustness and generalization.

Principles

• Gradient-level control mitigates OSSL trade-offs.
• Supervised gradients can anchor auxiliary updates.
• Preserve orthogonal components for representation signals.

Method

GGR uses supervised gradients as an anchor, projecting conflicting auxiliary gradients onto an admissible region in gradient space. It extends with subspace-aware rectification for stability.

In practice

• Apply GGR as a plug-in to existing OSSL baselines.
• Use subspace-aware rectification for noisy gradients.
• Evaluate GGR on CIFAR and ImageNet datasets.

Topics

• Open-Set Semi-Supervised Learning
• Gradient Rectification
• Machine Learning Robustness
• Out-of-Distribution Detection
• Computer Vision

Code references

• JiaheChen2002/GGR

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Gradient Regularized Newton Boosting Trees with Global Convergence — A gradient-regularized Newton boosting scheme achieves global convergence for GBDTs, addressing prior theoretical gaps.
• ILDR: Geometric Early Detection of Grokking — ILDR provides an early, robust geometric signal for grokking, enabling earlier intervention and reduced training.
• Optimal Stability of KL Divergence under Gaussian Perturbations — KL divergence stability under Gaussian perturbations extends to arbitrary distributions with optimal $\sqrt{\epsilon}$ degradation.
• The Double Dilemma in Multi-Task Radiology Report Generation: A Gradient Dynamics Analysis and Solution — Linear scalarization in multi-task RRG fails due to gradient dynamics' "Double Dilemma," solvable by CAME-Grad.
• Separating Geometry from Probability in the Analysis of Generalization — Generalization can be analyzed deterministically via sensitivity of optimization solutions to data perturbations.
• One-Step Generalization Ratio Guided Optimization for Domain Generalization — GENIE optimizer uses One-Step Generalization Ratio (OSGR) to balance parameter contributions for robust domain generalization.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.