Theta-regularized Kriging: Modelling and Algorithms

2026-04-17 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

Researchers from Wuhan University and Hubei Key Laboratory of Computational Science have introduced Theta-regularized Kriging (TRK), a novel surrogate model designed to enhance prediction accuracy and stability by penalizing the hyperparameter theta in the Gaussian stochastic process. The TRK model, derived from a maximum likelihood perspective, incorporates three distinct penalty methods: Lasso ($l_{1}$ regularization), Ridge ($l_{2}$ regularization), and Elastic-net. The authors developed a regularized optimization algorithm and a geometric search cross-validation (GSCV) tuning algorithm for implementation. Tested on nine numerical functions and two practical engineering examples, including Borehole simulation and steel column design optimization, TRK consistently outperformed existing penalized Kriging models (Universal Kriging, Penalized Blind Kriging, Modified Penalized Blind Kriging) in accuracy and stability. The source code for the proposed method and the OptRP MATLAB package for parameter optimization are publicly available on GitHub.

Key takeaway

Research scientists developing or applying Kriging models for computationally expensive simulations should consider integrating Theta-regularized Kriging (TRK). This approach, particularly the Ridge-penalized TR-RK variant, offers superior accuracy and stability compared to traditional Kriging and other penalized methods by specifically targeting the Gaussian stochastic process parameter $\bm{\theta}$. You can leverage the provided GSCV algorithm and OptRP MATLAB package to efficiently tune regularization parameters, potentially leading to more robust and precise surrogate models in engineering design and scientific computing.

Key insights

Theta-regularized Kriging improves prediction accuracy and stability by penalizing the Gaussian stochastic process's theta hyperparameter.

Principles

• Penalizing stochastic process parameters enhances local prediction.
• Regularization can prevent overfitting and improve generalization.
• Geometric search cross-validation optimizes regularization coefficients.

Method

The TRK model optimizes a modified log-likelihood function with Lasso, Ridge, or Elastic-net penalties on the Gaussian stochastic process parameter $\bm{\theta}$, using an iterative algorithm and a GSCV tuning method.

In practice

• Use TR-RK for general problems due to its balance of parameters and accuracy.
• Consider TR-LK for high-dimensional data with strong feature correlations.
• Employ the GSCV method to automate optimal regularization parameter selection.

Topics

• Theta-regularized Kriging
• Gaussian Process Regression
• Regularization Penalties
• Hyperparameter Optimization
• Geometric Search Cross-Validation

Code references

• xuelin-xie/Regularized-Kriging

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Data Scientist

Related on AIssential

• Canonical Regularisation of Wide Feature-Learning Neural Networks — Traditional ridge regularization biases feature-learning networks due to curved parameter-space geometry, necessitating a geodesic-based approach.
• Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation — SGLRW improves SG-MCMC robustness by localizing stochastic noise to off-diagonal covariance elements via lattice-based updates.
• Closed-Form Spectral Regularization for Multi-Task Model Merging — Iterative model merging implicitly regularizes an ill-posed inverse problem, which can be replaced by explicit closed-form spectral filtering.
• Ridge Regression from Poisson Resetting: A Renewal Perspective on Spectral Regularization — Connecting stochastic resetting to ridge regression offers a renewal perspective on spectral regularization.
• Guided Trajectory Optimization with Sparse Scaling for Test-Time Diffusion — Reward-guided Trajectory Scaling (RTS) optimizes diffusion model noise paths via sparse, PCA-driven key-step selection and a coarse-to-fine search.
• Low-Rank Tensor Completion Based on Fractional Regularization with Ky Fan p-k Norm — A novel nonconvex surrogate, TNPK, improves low-rank tensor completion by accurately approximating tubal rank with flexible parameters and proven convergence.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.