Neural Additive and Basis Models with Feature Selection and Interactions

2026-06-18 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new method enhances Neural Additive Models (NAM) and Neural Basis Models (NBM) by integrating a feature selection mechanism to address computational bottlenecks. While NAM and NBM offer high interpretability and strong performance by using neural networks as nonlinear shape functions in generalized additive models (GAMs), they become intractable when applied to high-dimensional datasets or when incorporating feature interactions via two-input neural networks. The proposed approach introduces a feature selection layer within both models, updating selection weights during training. This simple modification significantly reduces computational costs and model sizes compared to vanilla NAM and NBM. Furthermore, it enables the effective use of two-input NNs even in high-dimensional contexts, thereby facilitating the capture of crucial feature interactions. The enhanced models demonstrate improved computational efficiency and achieve performance comparable to or better than leading GAMs.

Key takeaway

For Machine Learning Engineers developing interpretable models with high-dimensional datasets, you should consider integrating feature selection into Neural Additive or Basis Models. This approach allows you to efficiently capture complex feature interactions using two-input neural networks without incurring prohibitive computational costs or excessive model sizes. You can achieve comparable or superior performance to existing GAMs while maintaining model interpretability.

Key insights

Feature selection enhances Neural Additive and Basis Models, enabling efficient interpretability and interaction capture in high-dimensional datasets.

Principles

• GAM-based NNs offer high interpretability.
• Feature selection improves model scalability.
• Two-input NNs capture feature interactions.

Method

Integrate a feature selection layer into Neural Additive and Basis Models, dynamically updating selection weights during training to manage computational resources and enable feature interactions.

In practice

• Apply feature selection to scale NAM/NBM.
• Capture feature interactions in high-dimensional data.
• Reduce NAM/NBM model size and cost.

Topics

• Neural Additive Models
• Neural Basis Models
• Feature Selection
• Model Interpretability
• Generalized Additive Models
• High-Dimensional Data

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• The Feature Selection Trap: Why ‘More Data’ Can Actively Hurt Your Machine Learning Model — Adding more features does not always improve ML model performance and can actively degrade it due to redundancy.
• Bag of Dims: Training-Free Mechanistic Interpretability via Dimension-Level Sign Patterns — Transformer hidden states' standard basis provides a training-free, dimension-level feature basis encoding semantics via signs.
• When Are Neural Interaction Discoveries Real? Identifiability, Recoverability, and a Pre-Fit Diagnostic — Neural interaction discoveries are identifiable based on input data geometry, not model architecture, with pre-fit diagnostics available.
• Robust Representation Learning through Explicit Environment Modeling — Explicitly modeling and marginalizing environmental variation yields more robust OOD generalization than invariance-seeking methods.
• Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selection — Finite neural networks can be approximated by Gaussian process mixtures with provable error bounds.
• Issue #130 - The Regression Playbook Part 2 — Different regression algorithms vary in function shape, error measurement, and training methods.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.