Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents

2026-07-17 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

Lucid, a black-box adversarial framework, exposes a critical vulnerability in multimodal AI agents that rely on persistent long-term memory grounded in visual and textual episodes. Operating under a strictly image-bounded threat model, Lucid crafts imperceptible perturbations to visual data without requiring access to the target MLLM, retrieval encoder, or text channel. It enables two distinct failure modes: Memory poisoning, an in-context attack where an adversarial image replaces a benign one reinforced by prior text, corrupting visual recall and steering the agent toward attacker-chosen narratives. Memory injection, an out-of-context attack, replaces a benign image in a conversation turn lacking prior textual grounding, causing the agent to generate attacker-influenced responses without corrective memory signals. Evaluated across various conversation domains and five black-box memory architectures, including graph-structured, LLM-summarized, and commercially deployed systems, Lucid achieved a 61.6% Attack Success Rate (ASR) on poisoning and 58.4% ASR on injection.

Key takeaway

For AI Security Engineers and Machine Learning Engineers designing or deploying multimodal AI agents with long-term memory, you must recognize the structural vulnerability to black-box visual attacks like Lucid. Your systems are susceptible to memory poisoning and injection, which can corrupt visual recall or inject false narratives. Implement robust visual input validation and memory integrity checks to mitigate these critical security risks and ensure reliable agent behavior.

Key insights

Unconditional trust in visual data creates a critical vulnerability in multimodal AI agents' long-term memory.

Principles

• Black-box adversarial attacks can compromise multimodal memory pipelines.
• Visual data perturbations can corrupt memory recall and steer agent narratives.

Method

Lucid crafts imperceptible perturbations to visual data, enabling memory poisoning (in-context) or memory injection (out-of-context) to compromise multimodal AI agent memory.

In practice

• Evaluate multimodal memory architectures against visual poisoning attacks.
• Implement robust visual input validation for AI agents.

Topics

• Multimodal AI Agents
• Long-term Memory
• Black-box Adversarial Attacks
• Memory Poisoning
• Memory Injection
• Machine Learning Security

Best for: AI Architect, Research Scientist, CTO, AI Scientist, AI Security Engineer, Machine Learning Engineer

Related on AIssential

• Memory Is a Feature. It Is Also an Attack Surface — Agentic AI memory, context, and local state are critical attack surfaces, enabling persistent compromise beyond single interactions.
• MemPoison: Uncovering Persistent Memory Threats and Structural Blind Spots in LLM Agents — LLM agent external memory introduces persistent, complex vulnerabilities that static write-time defenses fail to address.
• How Adversarial Environments Mislead Agentic AI? — Agentic AI is vulnerable to environmental deception through compromised tool outputs, exhibiting distinct epistemic and navigational weaknesses.
• Defending your Memory in Microsoft Foundry Agent Service against memory poisoning — Agent memory, while enabling continuity, creates a security boundary vulnerable to persistent poisoning, requiring write-path validation.
• AI Hallucinations Are Not the Biggest Problem. False Memories Might Be. — AI's confident misinformation can create false memories, influencing human judgment and collective understanding.
• Hidden prompts can plant false memories in AI agents, researchers warn — Hidden prompts can induce false memories in LLM-powered AI agents, posing a risk to their reliability.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.