EA-WM: Event-Aware World Models with Task-Specification Grounding for Long-Horizon Manipulation

2026-06-12 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

EA-WM is an event-aware world-model framework designed for long-horizon robot manipulation, augmenting pretrained visual-feature dynamics with task-specification-grounded event prediction and verification. This framework projects candidate futures into visual-feature space, decodes them into structured event states, and evaluates them using task-progress, semantic-consistency, physical-feasibility, and uncertainty metrics. The integrated verifier guides sampling-based planning, filters actions, and selects proposals, as demonstrated in the contact-sensitive LIBERO wine-rack setting. Across diverse manipulation studies, including navigation, deformable-object, and wall-constrained tasks, EA-WM significantly improved performance. PointMaze random-target success increased from 0.90 to 0.94, Deformable e10 blocks achieved 94% success, and Wall-Single reached 95% success. LIBERO-goal verification achieved an AUC of 0.993947, with the wine-rack task showing 97/100 online hybrid success.

Key takeaway

For Machine Learning Engineers developing long-horizon robot manipulation policies, consider integrating event-aware verification into your world models. This approach, by explicitly predicting and verifying task-relevant events, can significantly improve task success rates and action selection reliability. You should calibrate the event verifier's influence against visual-feature costs to optimize planning performance, especially for contact-sensitive or complex tasks. Implement conservative gating to ensure imagined improvements transfer robustly to real-world execution.

Key insights

Planning with verified task-relevant events improves robot manipulation beyond visual-feature prediction alone.

Principles

• Robot planning needs predicate-level progress signals.
• Event verification improves task alignment and interpretability.
• Calibrating verifier and feature costs is crucial for planning.

Method

EA-WM uses a pretrained visual-feature world model, an automatic simulator-based event labeler, a task-grounded event predictor/verifier, and a verifier-guided CEM planner for action selection.

In practice

• Augment visual world models with task-grounded event prediction.
• Use simulator state for automatic event label generation.

Topics

• World Models
• Robot Manipulation
• Event-Aware Planning
• Task Specification
• Pretrained Visual Features
• Model-Based Planning

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• EA-WM: Event-Aware World Models with Task-Specification Grounding for Long-Horizon Manipulation — EA-WM enhances robot imagination by verifying task-relevant events in visual-feature world models for long-horizon manipulation.
• Gradient-based planning for world models at longer horizons — GRASP enables robust long-horizon planning for world models by parallelizing optimization and improving gradient signals.
• World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems — The WAV model uses latent space inference for implicit planning in VLA systems, improving long-horizon decision-making.
• A Close Look At World Model Recovery In Supervised Fine-Tuned LLM Planners — SFT enables LLMs to internally represent planning world models, with broader training data improving recovery.
• MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models — MaskWAM unifies mask prompting and prediction via a Mixture of Transformers to enhance robotic control by resolving spatial ambiguity and visual noise.
• Gradient-based Planning for World Models at Longer Horizons — GRASP enables robust, long-horizon planning with learned world models by re-engineering gradient flow and introducing state-level stochasticity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.