MoAKE: Toward Unified All-in-One Action Quality Assessment via Mixture of Action Knowledge Experts

2026-07-22 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

The MoAKE (Mixture of Action Knowledge Experts) framework is proposed to address the limitations of current "one-by-one" Action Quality Assessment (AQA) methods, which require separate models for each action type. This new approach aims for "all-in-one" AQA, enabling a single unified model to assess heterogeneous actions without prior action-type knowledge. MoAKE mitigates negative knowledge transfer by learning complementary experts that capture diverse action patterns within a shared semantic space, dynamically aggregating their knowledge based on the input action. Each expert incorporates segment-aware prototypes to handle varying temporal lengths and an Adaptive Intra- and Inter-Segment Relationship Modeling (AIISRM) module for multi-granularity temporal dynamics. Extensive experiments on three long-term datasets demonstrate MoAKE's significant outperformance in all-in-one settings and consistent generalization on three short-term datasets under zero/few-shot evaluation.

Key takeaway

For Machine Learning Engineers developing video analysis systems, MoAKE offers a unified approach to Action Quality Assessment. If your current systems rely on separate models for each action type, consider adopting MoAKE. This simplifies deployment and improves generalization across diverse actions. The framework allows you to assess heterogeneous actions with a single model, significantly reducing complexity in zero/few-shot scenarios.

Key insights

MoAKE unifies Action Quality Assessment for diverse actions into a single model, overcoming "one-by-one" limitations.

Principles

• Mitigate negative knowledge transfer.
• Experts capture diverse action patterns.
• Adapt assessment to input action.

Method

MoAKE employs a Mixture of Action Knowledge Experts, each with segment-aware prototypes and an Adaptive Intra- and Inter-Segment Relationship Modeling (AIISRM) module, to dynamically aggregate knowledge for unified AQA.

In practice

• Assess heterogeneous actions with one model.
• Apply to zero/few-shot AQA scenarios.
• Utilize segment-aware prototypes for temporal variations.

Topics

• Action Quality Assessment
• Mixture-of-Experts
• Video Analysis
• Zero-shot Learning
• Few-shot Learning
• Temporal Dynamics
• Deep Learning Models

Code references

• XuHuangbiao/MoAKE

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• MoAKE: Toward Unified All-in-One Action Quality Assessment via Mixture of Action Knowledge Experts — MoAKE unifies Action Quality Assessment for diverse actions using a mixture of specialized, dynamically aggregated experts.
• MoAKE: Toward Unified All-in-One Action Quality Assessment via Mixture of Action Knowledge Experts — Unified AQA for diverse actions is achieved by adaptively mixing specialized expert knowledge to overcome negative transfer.
• WorldMark: A Unified Benchmark Suite for Interactive Video World Models — Standardized benchmarks are crucial for fair comparison and progress in interactive video world models.
• Boundary-Centric Active Learning for Temporal Action Segmentation — Focusing active learning on action boundaries significantly improves temporal action segmentation efficiency and accuracy.
• Divide, Deliberate, Decide: A Multi-Agent Framework for Fine-Grained Egocentric Action Recognition — Multi-agent VLM deliberation with heterogeneous specialists improves fine-grained egocentric action recognition by decorrelating model priors.
• CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering — CRAFT uses a critic-refined pipeline for accurate, attributable multi-video Q&A by verifying claims across sources.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.