LLM-Guided Communication for Cooperative Multi-Agent Reinforcement Learning

2025-04-14 · Source: cs.MA updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

LLM-driven Multi-Agent Communication (LMAC) is a novel framework designed to enhance cooperative multi-agent reinforcement learning (MARL) by improving state awareness under partial observability. LMAC leverages large language models (LLMs) to iteratively refine an agent-wise communication protocol. This refinement process, guided by an explicit state-awareness criterion, aims to enable all agents to reconstruct the underlying global state as accurately and uniformly as possible. Experiments on diverse MARL benchmarks, including StarCraft Multi-Agent Challenge with Communication (SMAC-Comm), Level-Based Foraging (LBF), Google Research Football (GRF), and SMACv2, demonstrate that LMAC significantly improves state reconstruction across agents and yields substantial performance gains over existing communication baselines. The framework also shows strong scalability, adaptability to stochastic environments, and robustness to initial data quality and moderate environment changes, while incurring modest computational overhead.

Key takeaway

For research scientists developing cooperative MARL systems, LMAC offers a robust approach to mitigate partial observability. By leveraging LLMs for offline, iterative communication protocol refinement, you can achieve more accurate and uniform state reconstruction across agents, leading to faster convergence and higher performance. Consider integrating meta-cognitive representation learning with cycle-consistency to ensure compact, task-relevant message encoding, especially in complex or stochastic environments like SMACv2 where LMAC even surpassed QMIX+State.

Key insights

LLMs can iteratively refine communication protocols in MARL to improve agents' shared state awareness.

Principles

• Explicit state-awareness criteria guide protocol refinement.
• Iterative feedback improves reconstruction accuracy and uniformity.
• Meta-cognitive representations distinguish reliable knowledge.

Method

LMAC uses an LLM to design an initial communication protocol, then iteratively refines it via two-step feedback (recovery enhancement, imbalance mitigation) based on quantitative state-awareness indicators derived from offline RL transition data.

In practice

• Use LLMs for offline protocol design, not online interaction.
• Integrate cycle-consistency loss to suppress redundant information.
• Calibrate reconstruction threshold α based on environment semantics.

Topics

• Cooperative MARL
• LLM-driven Communication
• Communication Protocols
• State Reconstruction
• Iterative Protocol Refinement

Code references

• uoe-agents/epymarl
• hijkzzz/pymarl2
• TonghanWang/NDQ
• chenf-ai/Multi-Agent-Communication-Considering-Representation-Learning
• mansicer/MAIC

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs — LLMs can achieve faithful uncertainty expression and improved metacognition through Reinforcement Learning with Metacognitive Feedback.
• Cooperation Breakdown in LLM Agents Under Communication Delays — Communication delays in LLM-MAS non-monotonically impact cooperation, with moderate delays increasing exploitation and excessive delays reducing it.
• Training-Free Agentic AI: Probabilistic Control and Coordination in Multi-Agent LLM Systems — Probabilistic control with belief-guided delegation significantly boosts multi-agent LLM system efficiency and robustness without training.
• Meta-Offline and Distributional Multi-Agent RL for Risk-Aware Decision-Making — Combining offline MARL with meta-learning enables UAVs to adapt quickly to dynamic wireless network conditions.
• Dynamic Trust-Aware Sparse Communication Topology for LLM-Based Multi-Agent Consensus — DySCo dynamically optimizes multi-agent LLM communication by selecting high-value interactions based on trust and relevance, reducing overhead.
• Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution — LLMs can effectively orchestrate RL skills for complex multi-agent coordination, achieving human-like adaptability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.MA updates on arXiv.org.