Zero-Shot Mission-Level Evaluation for Aerial MLLM Agents

2026-07-27 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Data Science & Analytics · Depth: Expert, quick

Summary

MissionBench is a new benchmark introduced for zero-shot mission-level evaluation of Multimodal Large Language Models (MLLMs) acting as reasoning modules for aerial embodied agents. Released on July 24, 2026, this benchmark features 120 missions across five distinct simulated 3D environments and four task families. Agents must autonomously plan, navigate, and report outcomes using only egocentric observations and their action history, without requiring aerial-specific fine-tuning. Testing 22 open- and closed-source MLLMs revealed that the top-performing model achieved success on fewer than 35% of missions, significantly below the 84.4% human performance. This disparity underscores the complexity of multi-step embodied tasks, emphasizing the need for capabilities like multi-step planning and adaptive reasoning beyond mere spatial perception. The analysis also indicates that larger general-purpose models exhibit stronger zero-shot embodied capabilities, with performance gains observed from scaling.

Key takeaway

For AI Scientists developing embodied agents, this research highlights that current MLLMs fall short on complex aerial missions, achieving less than 35% success compared to human performance. You should prioritize developing MLLMs with enhanced multi-step planning and adaptive reasoning capabilities, rather than solely focusing on spatial perception. Consider using benchmarks like MissionBench for closed-loop evaluation to accurately assess and improve your agent's mission-level competence.

Key insights

MLLMs struggle with complex aerial embodied tasks, requiring advanced planning and adaptive reasoning beyond perception.

Principles

• Mission-level competence needs multi-step planning.
• Adaptive reasoning is crucial for embodied agents.
• Scaling MLLMs improves zero-shot embodied capabilities.

Method

MissionBench evaluates MLLMs in aerial 3D environments using 120 missions across five environments and four task families, requiring autonomous planning and reporting from egocentric observations.

In practice

• Evaluate MLLMs on long-horizon embodied tasks.
• Focus MLLM development on multi-step planning.
• Design benchmarks for closed-loop agent evaluation.

Topics

• Multimodal Large Language Models
• Embodied AI
• Aerial Robotics
• Zero-Shot Learning
• Mission-Level Evaluation
• Benchmarking

Best for: Research Scientist, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• Zero-Shot Mission-Level Evaluation for Aerial MLLM Agents — MissionBench reveals current MLLMs struggle with complex aerial embodied tasks, despite scaling showing promise for zero-shot capabilities.
• ActiveFly-Bench: Aligning Embodied Question Answering with Vision-Language-Action for Aerial Embodied Perception — ActiveFly-Bench is a new benchmark and agent for UAV embodied perception, highlighting current VLM/VLA limitations in aerial tasks.
• Knowing the Self, Understanding the World: A Dual-Cognition Benchmark for UAV Spatio-temporal Reasoning with MLLMs — MLLMs lack dual-cognition for UAVs, prompting UAV-DualCog to benchmark self-state and environment-state reasoning.
• Hy-Embodied-VLM-1.0: Efficient Physical-World Agents — Hy-Embodied-VLM-1.0 offers an efficient, powerful foundation for physical-world embodied agents.
• Multi-Agent Robotic Control with Onboard Vision-Language Models — Onboard multi-agent systems with compact, fine-tuned VLMs offer a cost-efficient, explainable, and generalizable solution for robotic control.
• What matters in building vision–language–action models for generalist robots — RoboVLMs provide a flexible, high-performing framework for generalist robot control by optimizing VLM integration.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.