Latent Dynamics for Full Body Avatar Animation

2026-05-20 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, 3D Avatar Animation · Depth: Expert, medium

Summary

Latent Dynamics for Full Body Avatar Animation introduces a novel approach to animate full-body avatars, particularly addressing the complex dynamics of loose clothing that pose-driven neural rendering struggles with. Traditional methods either rely on costly explicit simulations requiring garment templates or data-driven avatars that use auxiliary latents without modeling their temporal evolution. This new model augments a pose-conditioned 3D Gaussian avatar with a transformer-based decoder and a "dynamics residual latent." This latent captures temporal appearance and geometry variations beyond simple pose signals. At inference, a learned latent dynamics model evolves this residual latent using a short pose history and the previous latent state. The model decomposes updates into driving, restoring, and dissipative forces, producing temporally coherent, history-dependent rollouts with negligible added cost. This allows for diverse yet plausible motion trajectories from different initial conditions and exposes controls like stiffness. Quantitative metrics and a perceptual user study across nine captured sequences demonstrate improved animation quality over recent data-driven baselines.

Key takeaway

For Computer Vision Engineers developing full-body avatar animation, this research offers a compelling alternative to explicit garment simulations. You can achieve highly realistic, temporally coherent clothing dynamics by integrating a latent dynamics model that evolves a residual latent based on pose history. This approach provides fine-grained control over dynamic elements like stiffness through force decomposition, significantly improving animation quality with negligible added computational cost compared to prior data-driven baselines. Consider adopting this method to enhance avatar realism.

Key insights

A learned latent dynamics model evolves a residual latent for history-dependent clothing dynamics in 3D Gaussian avatars.

Principles

• Pose alone cannot explain dynamic clothing deformations.
• Latent dynamics models can capture history-dependent motion.
• Force decomposition allows control over dynamic elements.

Method

Augment a pose-conditioned 3D Gaussian avatar with a transformer-based decoder and a dynamics residual latent. Evolve this latent using a learned dynamics model from short pose history and previous latent state, decomposing updates into driving, restoring, and dissipative forces.

In practice

• Generate diverse motion trajectories from varied initial conditions.
• Control dynamic elements like stiffness via force decomposition.

Topics

• Full-Body Avatar Animation
• Neural Rendering
• Clothing Dynamics
• 3D Gaussian Splatting
• Latent Dynamics Models
• Transformer Decoders

Code references

• llm-lab-org/Generative-AI-for-Character-Animation-Survey

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• One-shot Compositional 3D Head Avatars with Deformable Hair — Decoupling hair from the face in 3D avatar generation significantly improves realism and dynamic behavior.
• Learning Long-Term Motion Embeddings for Efficient Kinematics Generation — Efficient long-term motion generation is achieved by learning highly compressed motion embeddings and using conditional flow-matching.
• The Latent Color Subspace: Emergent Order in High-Dimensional Chaos — FLUX.1's VAE latent space contains an emergent, interpretable structure for color control.
• EverAnimate: Minute-Scale Human Animation via Latent Flow Restoration — EverAnimate uses latent context memory and flow restoration to generate consistent, long-duration human animations.
• T2LM: Long-Term 3D Human Motion Generation from Multiple Sentences — T2LM generates smooth, long-term 3D human motion from raw text without needing long-term training data or post-processing.
• Edit3DGS: Unified Framework for Dynamic Head Editing via 2D Instruction-Guided Diffusion and 3D Gaussian Splatting — Edit3DGS unifies 2D diffusion and 3D Gaussian splatting for dynamic, temporally consistent 3D head editing from video.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.