Twoblock clustering trees with coskewness-based dimension reduction: recovering piecewise multivariate linear regimes

2026-07-24 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

The twoblock clustering tree (tbtree) is a novel, highly interpretable regression tree designed for multivariate responses, featuring local multivariate linear models in its leaves. It employs dense or sparse twoblock dimension reduction and introduces a new estimator that maximizes coskewness to identify non-normal data clusters, enhancing regime discovery. This deterministic decision tree is computationally efficient and capable of recovering piecewise linear regimes, performing comparably to black-box techniques like random forests on complex nonlinear dependencies. A simulation demonstrated the coskewness objective's superior regime recovery, particularly with a k-means split. On the UCI energy-efficiency benchmark, a combined-split tbtree achieved an R^2 of 0.975 with 11 leaves, matching random forests (R^2=0.980) using significantly fewer parameters. For gas-turbine emissions, a decoupled coskewness-split tree yielded a scale-weighted RMSEP of 0.830, outperforming several black-box models on out-of-distribution data while providing 16 interpretable local emission models.

Key takeaway

For Machine Learning Engineers developing interpretable multivariate regression models, you should consider twoblock clustering trees (tbtree). This approach allows you to recover piecewise linear regimes and achieve predictive accuracy comparable to random forests or neural networks, but with orders of magnitude fewer parameters. Use coskewness-based splits for regime discovery and covariance-based leaves for optimal prediction. Be aware that coskewness requires reasonable sample sizes to be effective.

Key insights

Coskewness-driven twoblock clustering trees offer interpretable, accurate multivariate regression by identifying distinct data regimes.

Principles

• Non-normal mixtures reveal regime separation in third-order moments.
• Decoupling split and leaf objectives balances interpretability and accuracy.
• Higher-order moments are more variable, requiring sufficient sample size.

Method

The tbtree algorithm recursively induces splits using a coskewness-based twoblock model's first score, then fits covariance-based twoblock models in the leaves for prediction, pruned by cross-validation.

In practice

• Use coskewness splits for regime discovery.
• Employ covariance leaves for predictive accuracy.
• Combine objectives for smooth, non-linear data.

Topics

• Twoblock Clustering Trees
• Coskewness
• Interpretable Machine Learning
• Multivariate Regression
• Regime Discovery
• Dimension Reduction

Code references

• SvenSerneels/twoblock

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• CLARITree: Cholesky and Lookahead Accelerations for Regression with Interpretable Piecewise Linear Trees — CLARITree combines lookahead search and Cholesky updates for efficient, near-optimal, sparse piecewise linear regression trees.
• Graph-based Clustering Revisited: A Relaxation of Kernel k-Means Perspective — LoRD and B-LoRD improve graph-based clustering by selectively relaxing kernel k-means constraints and using block diagonal regularization.
• Learning Order Forest for Qualitative-Attribute Data Clustering — This paper introduces "Clustering with Order Forest learning" (COForest), a novel paradigm addressing the challenge of clustering qualitative (categorical/nominal) attribute data where traditional Euc…
• Simultaneous Latent Budget Trees for Stratified Classification — Simultaneous Latent Budget Trees enable interpretable classification by optimizing conditional splits using a probabilistic mixture model for stratified data.
• Hinge Regression Tree: A Newton Method for Oblique Regression Tree Splitting — Hinge Regression Trees use a damped Newton method to learn powerful oblique splits, achieving universal approximation.
• CEDAR: Causal Edge Discovery for Autoregressive Processes — CEDAR discovers lagged causal edges in sparse autoregressive time series, optimized for scarce data and lag-1 self-dynamics.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.