GeNeX: Genetic Network eXperts framework for addressing Validation Overfitting

2026-03-13 · Source: cs.NE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

The GeNeX (Genetic Network eXperts) framework addresses validation overfitting (VO) in deep learning, a phenomenon where models perform well on validation data but fail on unseen test data, especially under low-data conditions or distribution shifts. GeNeX mitigates VO at both model generation and ensemble construction stages. In the generation phase, it uses a dual-path strategy combining gradient-based training with genetic model evolution, creating diverse offspring networks via crossover and mutation without relying on validation feedback. For ensemble construction, GeNeX clusters candidate networks by prediction behavior, selects multiple diverse experts within each cluster, and fuses them at the weight level to form compact prototype networks. The final ensemble aggregates these prototypes, with predictions optimized via Sequential Quadratic Programming. Evaluated on four real-world image classification tasks (Skin Cancer, Deepfake Detection, Plant Disease, Pneumonia) using a VO-aware evaluation protocol that simulates distributional divergence, GeNeX consistently outperforms state-of-the-art ensembles, achieving strong generalization with minimal VO gaps.

Key takeaway

For AI Scientists developing robust ensemble models in low-data or distributionally shifted environments, GeNeX offers a critical solution. Its validation-independent model generation and prototype-based ensemble construction significantly reduce validation overfitting, ensuring reliable performance in real-world deployments where validation signals are often noisy or unrepresentative. Consider integrating genetic algorithms and behavioral clustering into your ensemble pipelines to enhance generalization and mitigate VO risk, especially in high-stakes applications like medical imaging or deepfake detection.

Key insights

GeNeX mitigates validation overfitting by decoupling model generation and ensemble construction from validation dependence.

Principles

• Minimize validation queries during model generation.
• Promote intrinsic diversity without validation reliance.
• Reduce ensemble size while maximizing representational diversity.

Method

GeNeX employs a two-module approach: GenE generates diverse models via supervised training and genetic weight fusion, while ProtoNeX clusters models by behavior, elects multiple experts per cluster, and fuses them into prototypes for a final SQP-optimized ensemble.

In practice

• Use JSD-guided partitioning to create VO-prone datasets.
• Employ genetic crossover for weight-level regularization.
• Cluster models by prediction behavior for complementary expert selection.

Topics

• Validation Overfitting
• Ensemble Learning
• Genetic Algorithms
• Distribution Shift
• Image Classification

Code references

• EmmanuelPintelas/Genetic-Network-eXperts-GeNeX-
• EmmanuelPintelas/-Shifted-Dataset-Creator

Best for: Computer Vision Engineer, AI Scientist, AI Researcher, Machine Learning Engineer, Research Scientist

Related on AIssential

• The first thing you need to understand when you start with AI — Effective machine learning models balance bias (underfitting) and variance (overfitting) to generalize well.
• Industrializing Prediction-Powered Inference: The GLIDE Library for Reliable GenAI and Agentic Systems Evaluation — Prediction-powered inference (PPI) combines human labels with proxy predictions for unbiased, cost-effective evaluation with valid confidence.
• ARC Prize 2025 Top Score 1st Place NVARC — Offline synthetic data generation and test-time adaptation are crucial for competitive machine learning success under compute constraints.
• Guiding Multi-Objective Genetic Programming with Description Length Improves Symbolic Regression Solutions — Description Length (DL) and Fractional Bayes Factor (FBF) effectively prevent overfitting and bloat in symbolic regression when used with multi-objective genetic programming.
• What is Random Forest? — Random Forest combines multiple decision trees via ensemble learning to achieve more stable and accurate predictions by reducing overfitting.
• Multi-Agent Conformal Prediction with Personalized Statistical Validity — PFWCP offers personalized, statistically valid uncertainty quantification in multi-agent settings, overcoming data heterogeneity and privacy limits.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.NE updates on arXiv.org.