Pre-training isn’t bitter enough

2026-07-10 · Source: ΑΙhub · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

Value-based pre-training with downstream feedback (V-pretraining) introduces a novel approach to foundation model training by making the self-supervised task objective learnable. Unlike standard pre-training, which fixes the task construction rule, V-pretraining employs a lightweight task designer that uses a small set of verifiable downstream examples to dynamically construct self-supervised tasks. The core foundation model, or learner, continues to update solely through self-supervised loss on unlabeled data, while the designer learns to create tasks whose gradients align with downstream performance. This indirect feedback mechanism was demonstrated in language models, where V-pretraining improved GSM8K Pass@1 for Qwen2.5-0.5B from 22.20 to 29.60 and Qwen1.5-4B from 56.48±1.56 to 58.98±1.03. In vision, DINOv3-ViT-L saw ADE20K mIoU increase from 51.33 to 52.47 and NYUv2 RMSE improve from 0.5752 to 0.5522.

Key takeaway

For Machine Learning Engineers optimizing foundation model pre-training, V-pretraining offers a path to more effectively align self-supervised objectives with downstream goals. You should consider implementing a learnable task designer to dynamically construct pre-training tasks, using a small set of downstream examples to guide this process indirectly. This approach can yield significant performance gains, as demonstrated by improved GSM8K Pass@1 and vision metrics, without directly supervising the learner.

Key insights

V-pretraining makes self-supervised task construction learnable via indirect downstream feedback, improving foundation model performance.

Principles

• The "Bitter Lesson" extends to learning what to predict, not just how to learn.
• Separating learner and task designer enables indirect downstream feedback.
• Aligning self-supervised gradients with downstream gradients improves utility.

Method

V-pretraining trains a task designer to construct self-supervised tasks (e.g., soft target distributions, view augmentations) whose resulting learner gradients align with gradients from a small downstream feedback batch. The learner then updates self-supervisely.

In practice

• Apply adaptive top-K soft target construction for language models.
• Modify DINO-style view construction for vision models.
• Use small downstream feedback batches to guide pre-training.

Topics

• V-pretraining
• Foundation Models
• Self-supervised Learning
• Task Design
• Downstream Alignment
• Gradient Alignment

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Pre-Training Isn’t Bitter Enough — V-pretraining learns optimal self-supervised tasks using downstream feedback, making pre-training more adaptive and effective.
• Pretraining vs. Post-Training: How a Text Predictor Becomes an Assistant — Post-training repurposes a pretrained model's latent internal structures, like the J-space, to install an assistant persona and values.
• The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models — Multi-teacher prompt distillation significantly improves VLM student performance, particularly under domain shift, through complementary teacher supervision.
• Reachability-Aware Pretraining for Efficient Target-Oriented Path Exploration in Temporal Knowledge Graph Reasoning — RAPTOR improves Temporal Knowledge Graph reasoning by pretraining agents with reachability awareness to guide reinforcement learning exploration.
• In-Context Multiple Instance Learning — Pretraining a Perceiver-style in-context learner on synthetic MIL data enables few-shot classification without gradient updates.
• Constraint-based Pre-training: From Structured Constraints to Scalable Model Initialization — Constraint-based pre-training disentangles size-agnostic knowledge from size-specific adaptation for scalable model initialization.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ΑΙhub.