Training and Evaluating Diffusion Policies with Long Context Lengths

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

A study investigates the impact of scaling context length for Diffusion Policies in robotic imitation learning, challenging prior claims that naive scaling is brittle. Researchers benchmarked policy performance by incrementally increasing context length from short to long across five tasks with varying memory requirements and local stability, and in multiple data regimes. They found that naive scaling is not catastrophically brittle, especially with an appropriate conditioning method like UNet+Cross-Attention. The work introduces a new training algorithm, "Variable History Training," which jointly trains policies at multiple context lengths, reducing sample complexity and achieving 1.25x-2x performance improvement in low-data regimes. The study involved training and evaluating nearly 200 policies, with context lengths scaled up to 80 in simulation and 92 on hardware, providing comprehensive insights into design choices for long-context imitation learning.

Key takeaway

For Robotics Engineers developing imitation learning policies, if you are addressing tasks requiring long memory or operating in data-limited scenarios, consider implementing the proposed variable history training algorithm. This approach, especially when combined with a UNet+Cross-Attention architecture, can significantly improve policy performance and robustness, challenging prior assumptions about the inherent brittleness of naive context scaling. You should also scale data where possible, as it substantially mitigates long-context learning problems.

Key insights

Naive context scaling for Diffusion Policies is viable with proper architecture and data, improved by variable history training.

Principles

• Naive context length scaling is not inherently brittle for Diffusion Policies.
• Long-context policy performance is highly task and data dependent.
• UNet+Cross-Attention provides a favorable inductive bias for long-context learning.

Method

The Variable History Training algorithm jointly trains policies over a curriculum of context lengths, using short-context datagrams to mitigate overfitting and benefit from memory.

In practice

• Prioritize UNet+Cross-Attention for long-context Diffusion Policies.
• Scale data to mitigate long-context learning challenges.
• Employ variable history training in limited-data regimes for performance gains.

Topics

• Diffusion Policies
• Imitation Learning
• Robotic Manipulation
• Context Length
• UNet+Cross-Attention
• Variable History Training

Code references

• huggingface/lerobot

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• RoboTTT: Context Scaling for Robot Policies — RoboTTT scales robot policy visuomotor context to 8K timesteps using Test-Time Training, enabling advanced capabilities.
• Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics — Ambient Diffusion Policy leverages noise-dependent data usage to extract useful features from suboptimal robotic demonstration data.
• Expanding Spatial and Temporal Context for Robotic Imitation Learning With Scene Graphs — Scene graphs provide explicit, compact memory for robots to reason over long-horizon tasks under partial observability.
• Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation — Integrating historical experience directly into a VLA model's native latent embedding space enhances long-horizon task performance.
• X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations — X-Diffusion selectively integrates noisy human demonstrations into robot policy training to overcome embodiment mismatches.
• AutoAnchor: Stable Diffusion Unlearning Using Cross-Attention as a Manifold Surrogate — AutoAnchor stabilizes diffusion unlearning by using cross-attention as an efficient surrogate for manifold proximity.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.