Dreamer-CPC: Message Learning with World Models for Decentralized Multi-agent Reinforcement Learning

2026-07-22 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

Dreamer-CPC is a new decentralized model-based multi-agent reinforcement learning (MARL) method that enhances inter-agent communication by integrating Collective Predictive Coding (CPC) message learning into the DreamerV3 world model. Unlike existing representation learning approaches that rely solely on current observations, Dreamer-CPC allows each agent to infer and exchange messages from the latent states of its independently maintained world model, which captures historical observations and actions. Evaluated in the Observer and a new CatchApple task, Dreamer-CPC significantly outperformed IPPO-CPC, an existing CPC-based method, and no-communication baselines. Notably, in CatchApple, Dreamer-CPC achieved 4 to 5 times the episode return of IPPO-CPC, demonstrating superior coordination in scenarios with temporarily missing observations. This indicates that communication grounded in world model latent dynamics is crucial for decentralized decision-making when current observations are insufficient.

Key takeaway

For Machine Learning Engineers developing decentralized multi-agent systems with partial observability, you should consider integrating world models to ground inter-agent communication. Leveraging latent states from these models, as demonstrated by Dreamer-CPC, significantly improves coordination and performance, particularly when current observations are unreliable or temporarily missing. This approach offers a robust solution for enhancing decision-making in complex, dynamic environments where traditional communication methods fall short.

Key insights

World model latent states enable robust multi-agent communication, especially with partial observability.

Principles

• Latent dynamics capture historical context for communication.
• Decentralized world models improve coordination.
• Communication grounded in history outperforms current observation-based methods.

Method

Dreamer-CPC integrates CPC message learning into DreamerV3's world model. Agents infer and exchange messages from latent states reflecting observation/action history.

In practice

• Apply world models to ground MARL communication.
• Use latent states for historical context in message passing.
• Consider Dreamer-CPC for tasks with missing observations.

Topics

• Multi-agent Reinforcement Learning
• World Models
• Collective Predictive Coding
• Decentralized Systems
• Inter-agent Communication
• Partial Observability

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers — Explicit world-state modeling via learnable registers improves multi-agent video consistency and generation quality.
• Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imagination — Koopman Dreamer uses spectrally constrained latent dynamics for stable, long-horizon world-model imagination in continuous control.
• A Survey of Multi-Agent Deep Reinforcement Learning with Graph Neural Network-Based Communication — GNNs facilitate multi-agent coordination by enabling learned communication and information sharing.
• Multi-agent Framework for Time-Sensitive Complementary Collaboration in Minecraft — Large language models struggle with real-time, dynamic multi-agent collaboration due to latency and coordination complexity.
• Learning to Communicate Locally for Large-Scale Multi-Agent Pathfinding — LC-MAPF uses learnable local communication to enhance multi-agent pathfinding coordination and scalability.
• Learning to Hand Off: Provably Convergent Workflow Learning under Interface Constraints — The IC-SMDP framework and IC-Q algorithm enable provably convergent decentralized workflow learning for LLM agents with limited, scalar-based coordination.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.