Progress Reward Modeling for Robotic Learning: A Comprehensive Survey

2026-07-27 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

This survey unifies the fragmented field of progress reward modeling for robotic learning, addressing the challenge of sparse terminal success signals in complex, dynamic environments. Traditional robotic learning struggles to provide intermediate feedback, hindering credit assignment and failure detection. Progress rewards offer dense, continuous feedback during task execution. The survey organizes existing literature into three interconnected perspectives: first, the model's interface, defining its inputs (task state representation, goal specification) and outputs (scalar scores, progress deltas, rankings, programmatic rewards); second, the construction methods, including frozen foundation model scoring, learning from temporal/relative supervision, instruction-tuned prediction, and programmatic reward generation; and third, the data and benchmarks, detailing supervision sources (human-driven, human-in-the-loop, automated) and evaluation protocols (fidelity, robustness, utility). It also outlines current limitations and future research.

Key takeaway

For Robotic Learning Engineers designing reward functions for complex, long-horizon tasks, relying solely on terminal success signals is insufficient. You should integrate progress reward modeling to provide dense, intermediate feedback, improving credit assignment and enabling robust policy learning. Carefully evaluate progress model interfaces, construction methods, and data sources, considering trade-offs like temporal context versus online usability. Be mindful of current limitations such as coarse-grained sensitivity and inference latency for real-time deployment.

Key insights

Progress rewards are crucial for robotic learning, offering dense, intermediate feedback to overcome limitations of terminal success signals in complex tasks.

Principles

• Progress estimation is latent and history-dependent, harder than binary success detection.
• Longer temporal context improves progress inference but reduces online usability.
• Human supervision provides semantic grounding; automation offers scalability.

Method

Analyze progress reward models by their interface (input/output), construction methods (signal source and conversion), and data/benchmarking (supervision and evaluation protocols).

In practice

• Specify goals with language for simple tasks, or vision for physical details.
• Employ programmatic rewards for tasks with explicit subgoals or API access.
• Augment successful trajectories with failures to reduce success-only bias.

Topics

• Robotic Learning
• Progress Reward Modeling
• Reinforcement Learning
• Vision-Language Models
• Task Planning
• Reward Function Design
• Benchmarking

Code references

• sterzhang/Awesome-Progress-Models

Best for: Research Scientist, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• Stage-Transition Dense Reward Modeling for Reinforcement Learning — STDR converts expert videos into dense, logically grounded rewards for RL, combining stage-transition and within-stage feedback.
• Better Reward Models For Robots — Augmenting robot action datasets with negative examples significantly improves vision-language reward models' effectiveness.
• Temporal Self-Imitation Learning — Temporal efficiency provides a powerful, underutilized self-supervision source for reinforcement learning.
• Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning — A diverse data recipe alone significantly improves long-context reasoning in reinforcement learning for LLMs.
• Real-Time Execution with Autoregressive Policies — Autoregressive policies can achieve real-time VLA execution through tokenization horizon adjustment and constrained multi-trajectory decoding.
• Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards — Progress-SQL enhances Text-to-SQL reinforcement learning through progressive, multi-turn rewards and an Oracle-guided Diagnostic Tree for iterative SQL refinement.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.