Infinite-dimensional Mahalanobis Distance with Applications to Kernelized Novelty Detection

2024-12-31 · Source: JMLR · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

A new framework extends the classical Mahalanobis distance to separable Banach spaces, reinterpreting it as a Cameron-Martin norm associated with a probability measure. This leads to a basis-free, data-driven anomaly distance called the variance norm, which can be estimated using empirical measures. The framework generalizes existing $\mathbb{R}^d$, functional $(L^2[0,1])^d$, and kernelized settings, notably incorporating non-injective covariance operators. The variance norm is proven invariant under invertible bounded linear transformations, extending prior results limited to unitary operators. In Hilbert spaces, the variance norm connects to the RKHS of the covariance operator, with consistency and convergence established for empirical estimation using Tikhonov regularization. The authors introduce a kernelized nearest-neighbour Mahalanobis distance, demonstrating its superior performance over traditional kernelized Mahalanobis distance for multivariate time series novelty detection on 12 real-world datasets, utilizing advanced time series kernels like signature, global alignment, and Volterra reservoir kernels.

Key takeaway

For research scientists developing anomaly detection systems, this work suggests that adopting the new infinite-dimensional Mahalanobis distance, specifically the kernelized nearest-neighbour variant, can significantly improve performance for multivariate time series novelty detection. You should consider implementing this approach, especially when dealing with complex time series data and non-injective covariance operators, to achieve more robust and accurate anomaly identification.

Key insights

The Mahalanobis distance is extended to infinite-dimensional spaces via a variance norm for enhanced novelty detection.

Principles

• Variance norm is invariant under invertible linear transformations.
• Cameron-Martin norm reinterprets Mahalanobis distance.
• Empirical measures can estimate variance norm consistently.

Method

The method reinterprets Mahalanobis distance as a Cameron-Martin norm, estimates a variance norm using empirical measures with Tikhonov regularization, and applies a kernelized nearest-neighbour approach for novelty detection.

In practice

• Apply kernelized nearest-neighbour for time series anomaly.
• Use signature or global alignment kernels.
• Leverage Tikhonov regularization for estimation.

Topics

• Mahalanobis Distance
• Novelty Detection
• Kernel Methods
• Banach Spaces
• Time Series Analysis

Code references

• nikitazozoulenko/kernel-timeseries-anomaly-detection

Best for: Research Scientist, AI Researcher, AI Scientist

Related on AIssential

• Bayesian Nonparametric Detection of Anomalies in Multivariate Functional Data — WICMAD uses infinite Gaussian process mixtures and adaptive kernel selection for semi-supervised functional anomaly detection.
• PaAno+: Multiscale Encoding and Cross-Variable Attention for Time Series Anomaly Detection — PaAno+ uses multiscale encoding and cross-variable attention for efficient, accurate time series anomaly detection, outperforming larger models.
• Robust State-Conditional Feature-Weighted Jump Models for Temporal Clustering — Robust Feature-Weighted Jump Models accurately cluster time series and identify state-specific feature importance, even with outliers.
• Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins — Overly compact feature clusters, like those from CenterLoss, harm OOD detection by compressing inter-class margins.
• Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence — A new theory for RKHS M-estimation provides sharp convergence rates and explains how these estimators avoid the curse of dimensionality.
• Rethinking Structural Anomaly Detection: From Decision Boundaries to Projection Operators — Structural anomaly detection improves by projecting data onto a normal manifold, identifying anomalies via projection residuals.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.