Agentic MPC for Semantic Control System Resynthesis

2026-06-01 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

The Agentic MPC framework integrates Model Predictive Control (MPC) with large language model (LLM)-based agents to enable context-aware, semantically adaptive control synthesis. While MPC excels at structured, low-level specifications, it traditionally lacks the ability to dynamically incorporate high-level contextual information like social norms or natural language instructions. This new framework allows an LLM-driven agent to interpret diverse inputs, including natural language messages, environmental observations, and external knowledge, to dynamically resynthesize MPC control specifications. A key aspect is the modularization of control specifications into reusable "primitives" that the agent can attach or detach using predefined tools. The system employs Fourier MPPI for efficient optimization, addressing computational complexity. Demonstrated in CARLA autonomous driving scenarios using the IBM Granite-4.1 8B LLM, the framework successfully adapts vehicle behavior based on user preferences, such as preferred steering direction around an obstacle, and responds to social situations like yielding to an emergency vehicle by temporarily adjusting path tracking and speed objectives.

Key takeaway

For autonomous driving engineers designing adaptive systems, you should consider integrating LLM-based agents with Model Predictive Control. This approach allows your system to dynamically adjust to high-level contextual information, such as user preferences or social norms, by resynthesizing control specifications. Implement modular control primitives and tool-based interfaces to enable flexible, interpretable adaptation, enhancing real-world performance and safety in complex, ambiguous situations.

Key insights

Agentic MPC combines LLM-based agents with modular MPC to enable context-aware, semantically adaptive control.

Principles

• LLM agents can interpret diverse inputs for control resynthesis.
• Control specifications can be modularized into reusable primitives.
• Invokable tools enable structured agent-controller interaction.

Method

An LLM agent interprets heterogeneous inputs (NL, observations, knowledge) to determine control specification modifications. It uses predefined tools to attach/detach modular primitive objective functions within an MPC controller, which then solves the optimization problem.

In practice

• Use LLM agents for high-level contextual control adaptation.
• Decompose MPC objectives into attachable/detachable primitives.
• Implement tool-based interfaces for agent-controller communication.

Topics

• Agentic AI
• Model Predictive Control
• Large Language Models
• Autonomous Vehicles
• Semantic Control Systems
• CARLA Simulation

Best for: Research Scientist, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• Agentic MPC for Semantic Control System Resynthesis — Agentic MPC integrates LLM agents for context-aware, semantically adaptive control synthesis.
• Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning — Coupling RL for speed guidance with MPC for trajectory optimization improves autonomous driving safety and efficiency.
• From Digital to Physical: Digital Agents as Autonomous Coaches for Physical Intelligence — LLM agents can autonomously engineer embodied policies, outperforming human baselines and self-correcting through iterative environment feedback.
• Agentic AI for Robot Teams — Applying LLM-based AI agents enables autonomy, coordination, and adaptability in heterogeneous robot teams.
• Why AI Agents Need to Check Their Own Work — Semantic control loops enable AI agents to self-correct and adapt by understanding the meaning of goals, not just numerical targets.
• Learning to Choose: An Empowerment-Guided Multi-Agent System with semantic communication for Adaptive Method Selection — Reliable autonomous learning in multi-agent systems requires preserving action-outcome fidelity through semantic checkpoints to prevent drift.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.