The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks

2026-07-22 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

The Quadrilateral Loss is introduced as a differentiable penalty for dense neural networks, designed to measure additivity as a behavioral characteristic rather than an architectural constraint. This loss functions as a second-order mixed difference on training point pairs, vanishing when a coordinate lacks interaction and providing insights for piecewise-linear networks. It also equates, in expectation, to the per-coordinate interaction mass of the interventional Shapley-GAM. The loss enables additivity to be adjusted, demonstrating that most learned interactions can be removed with minimal impact, and on smaller datasets, a moderate penalty enhances both accuracy and additivity. Furthermore, its per-feature surrender curves act as an online observable, revealing that initial interaction magnitudes poorly predict what a regularized model retains, thereby challenging conventional post-hoc interaction rankings. The authors compare this approach against other exact additivity methods, noting that behavioral constraints often outperform weight-space constraints.

Key takeaway

For Machine Learning Engineers developing interpretable models, the Quadrilateral Loss offers a novel approach to managing feature interactions. You can apply this differentiable penalty to dense neural networks to dial in additivity, potentially improving accuracy on small datasets while enhancing model interpretability. This method provides an online observable for interaction changes, challenging the reliability of traditional post-hoc interaction rankings and guiding your regularization strategies more effectively.

Key insights

The Quadrilateral Loss measures and controls feature additivity in neural networks behaviorally, improving interpretability and accuracy.

Principles

• Additivity is a measurable behavior, not just an architectural constraint.
• Behavioral additivity constraints often outperform weight-space methods.
• Pre-regularization interaction magnitude poorly predicts retained interactions.

Method

The quadrilateral loss is a differentiable penalty based on a second-order mixed difference on training points, which quantifies per-coordinate interaction mass and allows additivity to be tuned.

In practice

• Apply quadrilateral loss to tune interaction levels in dense neural networks.
• Use per-feature surrender curves for online observation of interaction changes.
• Re-evaluate post-hoc interaction rankings given their unreliability.

Topics

• Quadrilateral Loss
• Neural Network Interpretability
• Feature Interactions
• Additive Models
• Regularization Techniques
• Shapley-GAM

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks — The quadrilateral loss quantifies feature interaction in dense neural networks, enabling tunable additivity and improved interpretability.
• Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity — A unified function space theory reveals depth's limited functional expressivity when complexity is controlled by true function norms, not parameter counts.
• Flatness and Generalization: Learning Multi-Index Models with Homogeneous Neural Networks — Flattest interpolators in homogeneous neural networks generalize well for multi-index models despite symmetries.
• When Are Neural Interaction Discoveries Real? Identifiability, Recoverability, and a Pre-Fit Diagnostic — Neural interaction discoveries are identifiable based on input data geometry, not model architecture, with pre-fit diagnostics available.
• Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic — Neural networks on modular arithmetic tasks form task-intrinsic cyclic geometries, deviating from neural collapse due to a structured trade-off.
• Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision — Directional confusions and Rate-Distortion geometry reveal distinct inductive biases in human and machine vision.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.