ALCL: An Adaptive Log-Correntropy Loss for Robust Learning under Non-Gaussian Noise

2026-06-14 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

An Adaptive Log-Correntropy Loss (ALCL) is proposed to enhance robust deep learning, particularly in environments with heavy-tailed and impulsive non-Gaussian noise. Conventional losses like mean squared error (MSE) are highly sensitive to outliers, and existing correntropy-based objectives require empirical tuning of fixed kernel parameters. ALCL overcomes these issues by adaptively learning its robustness geometry during optimization. It employs a logarithmic residual model where shape and scale parameters are learned alongside network weights through differentiable reparameterization. This approach results in a maximum likelihood formulation with a formally bounded and redescending influence function, allowing dynamic adaptation to residual statistics and effective outlier suppression. Comparative experiments across four benchmark datasets, including grayscale and RGB image data, demonstrate ALCL's superior performance. Under high-noise conditions, ALCL improves median accuracy by up to 4.75% on grayscale benchmarks and 4.51% on RGB datasets, exhibiting reduced variance compared to MSE and optimally tuned generalized correntropy losses.

Key takeaway

For Machine Learning Engineers developing deep learning models in environments with significant heavy-tailed or impulsive noise, you should consider integrating the Adaptive Log-Correntropy Loss (ALCL). This method provides a computationally efficient and superior alternative to conventional MSE or fixed-parameter correntropy losses, dynamically adapting to evolving noise statistics. Implementing ALCL can significantly improve reconstruction fidelity and classification accuracy, especially under high-noise conditions, reducing performance variance across runs.

Key insights

ALCL adaptively learns loss parameters for robust deep learning against non-Gaussian noise, outperforming static methods.

Principles

• Conventional losses are sensitive to outliers.
• Fixed kernel parameters limit correntropy robustness.
• Adaptive loss parameters improve noise resilience.

Method

ALCL learns logarithmic residual model shape and scale parameters jointly with network weights via differentiable reparameterization, yielding a maximum likelihood formulation with a bounded, redescending influence function.

In practice

• Apply ALCL for robust image reconstruction.
• Use ALCL in deep learning with noisy data.
• Improve classification accuracy in high-noise.

Topics

• Adaptive Loss Functions
• Non-Gaussian Noise
• Deep Learning Robustness
• Correntropy
• Image Classification
• Outlier Suppression

Best for: Research Scientist, AI Engineer, Computer Vision Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• Preserving Plasticity in Continual Learning via Dynamical Isometry — Dynamical isometry is a key mechanism for preserving plasticity in continual learning.
• Architecture-Adaptive Uncertainty Fusion for Deepfake Detection — Correlation-Optimized Fusion (COF) adaptively combines multiple uncertainty sources to predict deepfake detection errors, improving cross-domain reliability.
• Distributional Loss for Robust Classification — Distributional Loss uses a bimodal Gaussian distribution as a softer target for classifier outputs, enhancing robustness and mitigating overfitting without extra labels.
• Realistic noise synthesis reduces bias and improves tissue microstructure estimation with supervised machine learning — Realistic noise synthesis in simulated data is crucial for unbiased dMRI microstructure estimation, especially at low SNR.
• Residual Modeling for High-Fidelity Learned Compression of Scientific Data — Tailored residual representations preserve learned compression advantages in high-fidelity scientific data scenarios.
• Asymmetric-Loss-Guided Hybrid CNN-BiLSTM-Attention Model for Industrial RUL Prediction with Interpretable Failure Heatmaps — A hybrid CNN-BiLSTM-Attention model with asymmetric loss improves turbofan RUL prediction and interpretability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.