Structural interpretability in SVMs with truncated orthogonal polynomial kernels

2026-04-17 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mathematics & Computational Sciences · Depth: Expert, extended

Summary

This paper introduces Orthogonal Representation Contribution Analysis (ORCA), a post-training interpretability framework for Support Vector Machines (SVMs) that utilize truncated orthogonal polynomial kernels. The method leverages the finite-dimensional, tensor-product orthonormal basis of the associated reproducing kernel Hilbert space (RKHS) to exactly expand the SVM's decision function. This expansion allows for the quantification of how the squared RKHS norm of the classifier is distributed across various structural components, including interaction orders, total polynomial degrees, and marginal coordinate effects, through normalized Orthogonal Kernel Contribution (OKC) indices. ORCA is a post-training diagnostic tool, requiring no surrogate models or retraining. Its utility is demonstrated on a synthetic double-spiral dataset and a real five-dimensional echocardiogram dataset, revealing structural aspects of model complexity not captured by predictive accuracy alone.

Key takeaway

For AI Scientists and Research Scientists developing or deploying SVMs with orthogonal polynomial kernels, ORCA offers a powerful post-training diagnostic. You can use OKC indices to understand the internal organization of your classifier, identifying whether it relies more on marginal effects or high-order interactions. This framework helps in selecting between models with similar predictive accuracy by preferring those with simpler structural profiles, and can also signal structural overfitting by tracking shifts in RKHS norm distribution towards higher complexity.

Key insights

ORCA provides exact, post-training interpretability for SVMs using truncated orthogonal polynomial kernels by decomposing their RKHS norm.

Principles

• Truncated orthogonal polynomial kernels induce finite-dimensional RKHS with explicit orthonormal bases.
• SVM decision functions can be exactly expanded in these orthogonal coordinates.
• RKHS norm decomposition reveals structural contributions of interaction orders and polynomial degrees.

Method

ORCA computes Orthogonal Kernel Contribution (OKC) indices by summing squared coefficients of the SVM's decision function, expanded in a tensor-product orthonormal basis, grouped by interaction order and total polynomial degree.

In practice

• Quantify SVM model complexity beyond predictive accuracy.
• Identify dominant interaction orders and polynomial degrees.
• Compare structural differences between models with similar performance.

Topics

• Support Vector Machines
• Orthogonal Polynomial Kernels
• Reproducing Kernel Hilbert Spaces
• Orthogonal Representation Contribution Analysis
• SVM Interpretability

Best for: AI Scientist, Research Scientist

Related on AIssential

• Structural interpretability in SVMs with truncated orthogonal polynomial kernels — ORCA provides structural interpretability for SVMs using truncated orthogonal polynomial kernels via OKC indices.
• Structural interpretability in SVMs with truncated orthogonal polynomial kernels — ORCA provides structural interpretability for SVMs using truncated orthogonal polynomial kernels via OKC indices.
• OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems — OrthoReg prevents neural networks in hybrid models from relearning symbolic components by directly penalizing overlap.
• Decomposing one-class support vector machine into an ensemble of one-data support vector machines — Decomposing OCSVM into an ensemble of one-data models significantly accelerates processing for large datasets.
• A Smooth Alternative to the Boosted Tree — MS-SKM provides a smooth, interpretable kernel-based alternative to boosted trees for tabular data with smooth underlying structures.
• A Kernel Nonconformity Score for Multivariate Conformal Prediction — The MKS unifies Bayesian uncertainty with frequentist coverage for multivariate conformal prediction via adaptive kernel-based scoring.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.