ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation

2026-07-09 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

ARDY is a novel streaming generation framework designed for real-time 3D human motion synthesis in interactive applications, addressing limitations of existing offline and online methods. Published on 2026-07-09, ARDY enables high-fidelity motion generation controllable via online text prompts and flexible kinematic constraints. It employs a hybrid representation, combining explicit root features with a latent body embedding, and features a two-stage autoregressive transformer denoiser that supports variable history context and long-horizon kinematic constraints. Trained on a large-scale motion capture dataset using ground truth poses and text labels, ARDY natively learns controllable generation. Evaluations on the HumanML3D benchmark and the Bones Rigplay dataset confirm its high motion quality and constraint adherence. An interactive demo further showcases its versatility for dynamic text control, keyframe pose constraints, path following, and interactive locomotion.

Key takeaway

For Robotics Engineers developing interactive humanoid applications, ARDY offers a robust solution for real-time 3D human motion generation. If you are struggling with balancing precise control and inference speed, consider integrating ARDY's framework. Its hybrid representation and autoregressive denoiser enable high-fidelity motion with dynamic text and kinematic constraint control, significantly enhancing interactive experiences. Explore the provided code and models to accelerate your development of responsive and controllable motion systems.

Key insights

ARDY enables real-time, high-fidelity 3D human motion generation with online text and kinematic control.

Principles

• Hybrid representation balances trajectory control and generative learning.
• Autoregressive denoisers support variable history and long-horizon constraints.
• Direct conditioning on ground truth data improves controllable generation.

Method

ARDY uses a hybrid representation and a two-stage autoregressive transformer denoiser, trained on large-scale motion capture data with direct text and kinematic constraint conditioning.

In practice

• Implement dynamic text control for motion.
• Apply keyframe pose constraints for specific poses.
• Integrate interactive locomotion via mouse/keyboard.

Topics

• ARDY
• Human Motion Generation
• Real-time Animation
• Kinematic Constraints
• Text-to-Motion
• Diffusion Models
• Robotics

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• T2LM: Long-Term 3D Human Motion Generation from Multiple Sentences — T2LM generates smooth, long-term 3D human motion from raw text without needing long-term training data or post-processing.
• ARAPDiffusion: ARAP Regularization for Diffusion-Based Deformable Shape Space Learning — ARAPDiffusion integrates geometric regularization into latent diffusion to learn deformable shape spaces with less data.
• OPSD-V: On-Policy Self-Distillation for Post-Training Few-Step Autoregressive Video Generators — OPSD-V uses on-policy self-distillation with real long-video context to reduce error accumulation in few-step autoregressive video generators.
• DanceDuo: Bridging Human Movement and AI Choreography — DanceDuo uses diffusion models and pose estimation to generate and compare AI-choreographed dance with human performance.
• ARDepth: Auto-regressive Monocular Depth Estimation with Progressive Visual Conditioning — Auto-regressive generation offers a promising paradigm for monocular depth estimation, progressively building depth representations for structural consistency.
• PhyGenHOI: Physically-Aware 4D Generation of Dynamic Human-Object Interactions — PhyGenHOI couples generative human motion with explicit physical object simulation for realistic 4D human-object interactions.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.