Guiding Multi-Objective Genetic Programming with Description Length Improves Symbolic Regression Solutions

2026-05-22 · Source: cs.NE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

A study evaluates Description Length (DL) and Fractional Bayes Factor (FBF) as model selection criteria for Symbolic Regression with Genetic Programming (GPSR), addressing overfitting and structural bloat. Researchers implemented DL using a Fisher-information-based parameter encoding with SVD for correlated parameters, comparing it against AIC, BIC, and a modified BIC (BIC_SR) across synthetic and real-world datasets. Three strategies were tested: multi-objective search for accuracy and program length (MO-Length) with post-selection, multi-objective search with DL as an objective (MO-DL), and single-objective optimization (SO) using DL/FBF as fitness. Findings indicate that DL/FBF post-selection in MO-Length significantly improves test performance over AIC/BIC, with BIC_SR yielding comparable results. However, direct single-objective optimization with DL/FBF often leads to premature convergence and underfitting. DL also adaptively adjusts model size based on noise and dataset size.

Key takeaway

For AI Scientists and Machine Learning Engineers developing symbolic regression systems, you should integrate Description Length (DL), Fractional Bayes Factor (FBF), or BIC_SR into your multi-objective genetic programming workflows. These criteria, when used for post-selection from Pareto fronts optimizing accuracy and program length, consistently yield compact, accurate models and prevent overfitting. Crucially, avoid using DL/FBF directly as a single-objective fitness function, as this risks premature convergence to underfit expressions. BIC_SR offers a computationally efficient alternative with comparable performance.

Key insights

Description Length (DL) and Fractional Bayes Factor (FBF) effectively prevent overfitting and bloat in symbolic regression when used with multi-objective genetic programming.

Principles

• Model selection criteria incorporating function complexity outperform those based solely on parameter count in GPSR.
• Multi-objective optimization (accuracy + length) with post-selection preserves diversity and prevents premature convergence.
• Description Length adaptively adjusts model complexity to noise levels and dataset size.

Method

The DL adaptation computes parameter codelength using SVD on the Fisher Information Matrix to handle correlated parameters, improving stability in Genetic Programming Symbolic Regression.

In practice

• Combine multi-objective GPSR (NLL + program length) with DL, FBF, or BIC_SR for robust model selection.
• Avoid using DL/FBF directly as single-objective fitness in GPSR to prevent premature convergence and underfitting.
• Consider BIC_SR as a computationally cheaper alternative to DL/FBF for similar model selection performance.

Topics

• Symbolic Regression
• Genetic Programming
• Minimum Description Length
• Fractional Bayes Factors
• Model Selection
• Multi-objective Optimization

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• FunctionEvolve: Structure-Guided Symbolic Regression with LLMs — Structure-guided symbolic regression with LLMs significantly improves exact form recovery by using expression trees.
• GeNeX: Genetic Network eXperts framework for addressing Validation Overfitting — GeNeX mitigates validation overfitting by decoupling model generation and ensemble construction from validation dependence.
• Bidirectional Search for Longest Paths: Case for Front-to-Front Heuristics — BiXDFBnB uses paired-state bidirectional search and front-to-front heuristics to efficiently solve complex longest-path problems.
• A Human-in-the-Loop Bayesian Optimization Framework for Constraint-Aware Bioprocess Development — This framework extends Human-in-the-Loop Bayesian Optimization for robust, constraint-aware bioprocess development.
• Runtime Analysis of Cartesian Genetic Programming in Evolving Boolean Functions — Non-strict selection and incomplete training sets can significantly optimize Cartesian Genetic Programming's efficiency for Boolean function evolution.
• Data Presentation Over Architecture: Resampling Strategies for Credit Risk Prediction with Tabular Foundation Models — Context construction significantly impacts Tabular Foundation Model performance in imbalanced credit risk prediction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.NE updates on arXiv.org.