Wan-Dancer: A Hierarchical Framework for Minute-scale Coherent Music-to-Dance Generation

2026-07-10 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

Wan-Dancer is a new hierarchical framework designed for minute-scale coherent music-to-dance video generation, addressing the limitations of current diffusion models that typically fail beyond 20 seconds. This method decouples the process into global keyframe planning and local temporal refinement, utilizing full-track musical context to maintain long-range coherence. Key innovations include dynamic frame rate adaptation via time-mapped RoPE embeddings for precise alignment, an optical-flow-based loss function to improve motion continuity, and motion-speed control for high-fidelity detail preservation during rapid movements. Experiments show Wan-Dancer generates stable 720p/30fps videos exceeding one minute, demonstrating superior temporal stability. The model also exhibits robust versatility across five distinct dance genres, conditioned by both audio and textual prompts, establishing a new benchmark in long-form dance video synthesis.

Key takeaway

For Computer Vision Engineers developing long-form video generation, Wan-Dancer offers a solution to temporal coherence and duration limits. You can now generate stable, minute-scale 720p/30fps dance videos across multiple genres, overcoming the typical 20-second barrier of diffusion models. Consider integrating hierarchical planning and dynamic frame rate techniques to improve your long-duration generative models.

Key insights

Wan-Dancer uses a hierarchical approach to generate minute-scale, coherent music-to-dance videos, overcoming temporal limitations of diffusion models.

Principles

• Decouple global planning from local refinement.
• Utilize full-track musical context for coherence.
• Adapt frame rates dynamically for alignment.

Method

The method decouples generation into global keyframe planning and local temporal refinement. It uses time-mapped RoPE embeddings for dynamic frame rate adaptation, an optical-flow-based loss for motion continuity, and motion-speed control.

In practice

• Generate 720p/30fps dance videos over one minute.
• Support five distinct dance genres.
• Condition generation with audio and text prompts.

Topics

• Music-to-Dance Generation
• Hierarchical AI
• Diffusion Models
• Long-form Video Synthesis
• RoPE Embeddings
• Motion Control

Best for: Research Scientist, AI Scientist, Computer Vision Engineer, Machine Learning Engineer

Related on AIssential

• 🏯SOTA Music-to-Dance Gen🏯 👉The Tongyi Lab unveils Wan-Dancer, a novel stable... — Wan-Dancer from Tongyi Lab achieves new SOTA in minute-scale, high-resolution music-to-dance synthesis across five genres.
• Mode Seeking meets Mean Seeking for Fast Long Video Generation — Decoupling local fidelity and long-term coherence enables minute-scale video generation from limited long-form data.
• DanceCrafter: Fine-Grained Text-Driven Controllable Dance Generation via Choreographic Syntax — Fine-grained text-driven dance generation requires specialized syntax, high-quality data, and tailored model architecture.
• Learning Long-Term Motion Embeddings for Efficient Kinematics Generation — Efficient long-term motion generation is achieved by learning highly compressed motion embeddings and using conditional flow-matching.
• Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering — A unified framework combines hierarchical autoregressive planning and flow-matching for high-quality, full-song generation across diverse tasks.
• MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generation — Hierarchical latent token compression and coarse-to-fine rollout enable long-range consistency in video generation by managing detail levels efficiently.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.