AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild

2026-05-21 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

AnyMo is a geometry-aware framework designed for setup-agnostic human motion modeling using wearable and mobile inertial measurement units (IMUs). It addresses the challenge of IMU signal dependence on sensing setup by generating diverse synthetic signals through physics-grounded IMU simulation over dense body-surface placements. The framework pre-trains a graph encoder using paired synthetic placement views and masked partial observations, tokenizes multi-position IMU data into full-body motion tokens, and aligns these tokens with a Large Language Model (LLM) for motion-language understanding. AnyMo was evaluated across three tasks: zero-shot activity recognition on 14 unseen datasets, cross-modal retrieval, and wearable IMU motion captioning. It achieved significant improvements, including an average Accuracy/F1/R@2 increase of 11.7%/11.6%/22.6% on HAR, a 15.9% and 28.6% increase in zero-shot IMU-to-text and text-to-IMU retrieval MRR respectively, and an 18.8% improvement in zero-shot captioning BERT-F1. These results position AnyMo as a generalist model for understanding wearable motion in diverse, real-world scenarios.

Key takeaway

For Machine Learning Engineers developing wearable motion applications, AnyMo presents a significant advancement in handling IMU setup variability. Its geometry-aware framework, which uses physics-grounded simulation and LLM alignment, enables robust zero-shot activity recognition, cross-modal retrieval, and motion captioning. You should investigate AnyMo's methodology to build more generalizable models, especially when working with diverse wearable devices and "in the wild" data, to improve performance across unseen datasets.

Key insights

AnyMo enables setup-agnostic human motion understanding from wearable IMUs by combining physics-grounded simulation, graph encoders, and LLM alignment.

Principles

• IMU signal dependence on setup hinders transferability.
• Physics-grounded simulation generates diverse synthetic data.
• Aligning motion tokens with LLMs enhances understanding.

Method

AnyMo simulates IMU signals from dense body placements, pre-trains a graph encoder with synthetic data, tokenizes multi-position IMU into motion tokens, and aligns them with an LLM.

In practice

• Zero-shot activity recognition across datasets.
• Cross-modal IMU-to-text retrieval.
• Wearable IMU motion captioning.

Topics

• Human Motion Modeling
• Wearable Sensors
• Inertial Measurement Units
• Graph Neural Networks
• Large Language Models
• Zero-shot Learning
• Activity Recognition

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Latent Dynamics for Full Body Avatar Animation — A learned latent dynamics model evolves a residual latent for history-dependent clothing dynamics in 3D Gaussian avatars.
• In-Context Model Predictive Generation: Open-Vocabulary Motion Synthesis from Language Models to Physics — ICMPG integrates LLM planning with physical simulation for realistic, semantically faithful open-vocabulary motion synthesis.
• WHAR Arena: Benchmarking the State of the Art in Efficient Wearable Human Activity Recognition — Benchmarking reveals WHAR predictive performance has plateaued, shifting focus to deployment efficiency and domain adaptation.
• A machine learning study highlighting the challenges of fidgety movement recognition using vision and inertial sensors — Automated Fidgety Movement recognition in infants faces significant generalization challenges across unseen subjects.
• EWAM: An Enhanced World Action Model for Closed-Loop Online Adaptation in Embodied Intelligence — EWAM enables zero-shot online adaptation for embodied AI by integrating lightweight neural layers into a frozen backbone for inference-time co-reasoning.
• Selective Correlation Based Knowledge Distillation for Ground Reaction Force Estimation — SCKD distills knowledge from large models to create efficient, accurate GRF estimation models for wearable sensors.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.