Mosaic: Runtime-Efficient Multi-Agent Embodied Planning

2026-07-13 · Source: cs.MA updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

Mosaic is a novel multi-agent planning framework designed to overcome the prohibitively high execution latency in LLM-based multi-agent embodied planning. It specifically addresses two core challenges: inaccurate state tracking under partial observability and inefficient coordination leading to redundant or conflicting actions. Mosaic achieves accurate, lightweight state tracking through agent-centric semantic memory, which stores objects in relative coordinates and enables geometric transformations. For efficient coordination, it employs Integer Linear Programming (ILP) at every planning step to allocate actions, enforcing physical feasibility and inter-agent coordination constraints. Benchmarking on AI2-THOR and search-and-rescue tasks, Mosaic demonstrated 27-32% faster execution, 30-33% fewer LLM calls, 25-31% fewer steps, and 4-10% points higher success rates.

Key takeaway

For Machine Learning Engineers and Robotics Engineers developing multi-agent embodied systems, Mosaic offers a clear path to significantly reduce execution latency. You should consider integrating agent-centric semantic memory for state tracking and Integer Linear Programming for action coordination into your planning frameworks. This approach can yield substantial improvements in runtime efficiency, reduce LLM call overhead, and boost overall task success rates, making your multi-agent systems more practical and scalable.

Key insights

Efficient memory and constraint-guided coordination are critical for scalable, low-latency multi-agent planning.

Principles

• Failed actions are the dominant bottleneck in multi-agent embodied planning.
• Accurate state tracking requires agent-centric semantic memory with relative coordinates.
• Efficient coordination demands constraint-guided action allocation via Integer Linear Programming.

Method

Mosaic maintains agent-centric semantic memory for state tracking and uses Integer Linear Programming to allocate actions, ensuring physical feasibility and inter-agent coordination at each planning step.

In practice

• Implement agent-centric semantic memory for robust state tracking.
• Utilize ILP for optimal multi-agent action allocation.
• Reduce LLM calls by improving planning efficiency.

Topics

• Multi-Agent Systems
• Embodied AI
• LLM-based Planning
• Runtime Efficiency
• Integer Linear Programming
• Semantic Memory

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• WorldLines: Benchmarking and Modeling Long-Horizon Stateful Embodied Agents — Long-horizon embodied agents require benchmarks and memory frameworks that handle dynamic, stateful, and partially observable environments.
• TypeGo: An OS Runtime for Embodied Agents — TypeGo's OS-style runtime enables real-time, multi-tasking embodied agent control by decoupling LLM planning from execution.
• Multi-Agent Transactive Memory — MATM allows LLM agents to share procedural trajectories, improving task performance and reducing steps without coordination or joint training.
• Embodied Multi-Agent Coordination by Aligning World Models Through Dialogue — Dialogue in embodied LLM agents reduces conflicts but can degrade task success without genuine world-model alignment.
• Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots — Embodied.cpp unifies embodied AI model deployment across diverse hardware, addressing multi-rate, latency-first inference needs.
• WISE: A Long-Horizon Agent in Minecraft with Why-Which Reasoning — WISE unifies exploration, memory, and planning in Minecraft agents via causal reasoning for adaptive, efficient long-horizon task execution.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.MA updates on arXiv.org.