Approximation-Free Differentiable Oblique Decision Trees

2025-12-31 · Source: JMLR · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

Subrat Prasad Panda, Blaise Genest, and Arvind Easwaran introduce DTSemNet, a novel approach for training approximation-free differentiable oblique Decision Trees (DTs). Published in 2026, DTSemNet addresses the challenges of complex optimization landscapes and overfitting in traditional oblique DTs, particularly in regression tasks. Unlike existing differentiable DTs that rely on probabilistic softening or quantized gradients like the Straight-Through Estimator (STE), DTSemNet represents hard oblique DTs as semantically equivalent, invertible neural networks. This enables end-to-end training using standard gradient descent for both classification and regression. For regression, DTSemNet incorporates an annealed Top-$k$ method to generate accurate, approximation-free gradient signals, mitigating STE's limitations. Extensive experiments demonstrate that DTSemNet-trained oblique DTs surpass state-of-the-art differentiable DTs on various classification and regression benchmarks. Furthermore, DTSemNet expands DT applicability by serving as programmatic DT policies in reinforcement learning environments.

Key takeaway

For Machine Learning Engineers developing interpretable models for safety-critical domains or tabular data, DTSemNet offers a robust alternative to approximation-reliant differentiable Decision Trees. You can achieve superior accuracy in both classification and regression by leveraging its approximation-free, end-to-end gradient training. Consider integrating DTSemNet to implement precise, programmatic DT policies within your reinforcement learning environments, enhancing model transparency and performance.

Key insights

DTSemNet enables approximation-free, end-to-end gradient-based training of hard oblique Decision Trees by representing them as neural networks.

Principles

• Oblique DTs offer interpretability for tabular data.
• Approximations hinder differentiable DT accuracy.
• Hard DTs can be semantically mapped to NNs.

Method

DTSemNet represents hard oblique DTs as neural networks for end-to-end gradient descent. It uses an annealed Top-$k$ method for approximation-free gradient signals in regression, overcoming STE limitations.

In practice

• Train interpretable DTs for safety-critical domains.
• Improve regression accuracy with oblique DTs.
• Implement DT policies in reinforcement learning.

Topics

• Oblique Decision Trees
• DTSemNet
• Gradient-based Training
• Reinforcement Learning
• Tabular Data
• Interpretable AI

Code references

• CPS-research-group/dtsemnet

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Decision Trees Demystified: The Algorithm That Thinks Like You Do — Decision trees mimic human decision-making by optimally reducing uncertainty through recursive data splits, offering high interpretability.
• Discrete Tilt Matching — DTM fine-tunes dLLMs by matching state-level unmasking posteriors under reward tilting, bypassing intractable sequence-level likelihoods.
• ARC Prize 2025 Paper Award 1st Place TRM — TRM demonstrates that small, recursive models can achieve strong generalization on tiny datasets by learning to iteratively improve answers.
• Ternary Decision Trees with Locally-Adaptive Uncertainty Zones — Ternary decision trees improve accuracy by explicitly modeling and blending predictions for boundary-uncertain instances.
• A Smooth Alternative to the Boosted Tree — MS-SKM provides a smooth, interpretable kernel-based alternative to boosted trees for tabular data with smooth underlying structures.
• Hinge Regression Tree: A Newton Method for Oblique Regression Tree Splitting — Hinge Regression Trees use a damped Newton method to learn powerful oblique splits, achieving universal approximation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by JMLR.