CreFlow: Corrective Reflow for Sparse-Reward Embodied Video Diffusion RL

2026-05-15 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

CreFlow is a novel online Reinforcement Learning (RL) framework designed to improve embodied video generation models (VGMs) by addressing physical implausibility in manipulation tasks. Traditional VGMs, trained on heterogeneous data, often produce visually plausible but physically unrealistic rollouts. Existing video-RL rewards typically rely on low-level visual metrics, failing to capture the compositional task requirements of manipulation. CreFlow introduces a compositional constraint-based reward model that automatically formulates task requirements using Linear Temporal Logic (LTL) constraints, providing accurate rewards and localized error information. The framework incorporates two key designs: a credit-aware NFT loss that confines RL updates to reward-relevant regions, preventing perturbations to unrelated areas, and a corrective reflow loss that leverages within-group positive samples to estimate correction directions, stabilizing and accelerating training. Experiments on eight bimanual manipulation tasks demonstrate that CreFlow's reward judgments align better with human and simulator success labels and improve downstream execution success by 23.8 percentage points.

Key takeaway

For research scientists developing robotic manipulation systems with video generation models, CreFlow offers a robust approach to overcome physical implausibility. You should consider integrating compositional LTL constraints for more faithful reward signals and implement localized, credit-aware RL updates. This strategy will lead to more stable training and significantly higher downstream task success rates, particularly for long-horizon, multi-stage manipulation tasks.

Key insights

CreFlow enhances embodied video generation by localizing RL updates with LTL-based compositional rewards and corrective reflow.

Principles

• Manipulation success is compositional, requiring logic-based verification.
• Localized updates prevent off-manifold drift in high-dimensional video RL.
• Within-group positive samples provide effective corrective targets.

Method

CreFlow uses a compositional LTL monitor for reward and violation traces. It applies a credit-aware NFT loss to focus updates on relevant regions and a corrective reflow loss using the empirical mean of successful rollouts as a target.

In practice

• Use LTL constraints for complex manipulation task evaluation.
• Apply spatio-temporal masks to focus RL gradients.
• Aggregate successful rollouts for explicit corrective supervision.

Topics

• Embodied Video Diffusion RL
• Compositional LTL Constraints
• Credit-Aware NFT Loss
• Corrective Reflow Loss
• Robotic Manipulation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation — Distilling 4D geometry foundation model features into video world models ensures correspondence-consistent generation for robot manipulation.
• Factored Diffusion Policies:Compositionally Generalized Robot Control with a Single Score Network — Factored Diffusion Policies use a single diffusion network with additive score decomposition for compositionally generalized robot control.
• E-TTS: A New Embodied Test-Time Scaling Framework for Robotic Manipulation — E-TTS unifies reasoning and action scaling for robotic manipulation via history-aware iterative refinement.
• The Lie We Tell: Correcting the Euclidean Fallacy in Vision Language Action Policies via Score Matching on Tangent Space — Lie Diffuser Actor corrects geometric errors in VLA policies by operating intrinsically on the SE(3) manifold, ensuring geodesic optimality.
• ROVE: Unlocking Human Interventions for Humanoid Manipulation via Reinforcement Learning — ROVE improves humanoid VLA manipulation by using Optimistic Value Estimation and cross-embodiment videos to learn from imperfect human interventions.
• DeVI: Physics-based Dexterous Human-Object Interaction via Synthetic Video Imitation — DeVI enables physics-based dexterous robot control using synthetic 2D videos and a hybrid 2D/3D tracking reward.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.