MRMMIA: Membership Inference Attacks on Memory in Chat Agents

2026-05-27 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, medium

Summary

Multi-Recall Memory MIA (MRMMIA) is a novel, unified attack framework designed to assess privacy leakage in chat agent memory systems. Developed by Kai Chen, Yan Pang, and Tianhao Wang, MRMMIA addresses the overlooked vulnerability of sensitive user-agent interactions, retrieved facts, and user preferences stored in agent memory. Unlike prior membership inference attacks (MIAs) that primarily targeted training corpora or retrieval databases, MRMMIA specifically infers whether a candidate memory unit belongs to a chat agent's memory store. The attack utilizes multiple recall probes to extract membership signals, operating effectively across black-box, gray-box, and white-box settings. Experimental results demonstrate MRMMIA's consistent superior performance compared to existing baselines, highlighting significant privacy risks in current chat agents and establishing an initial evaluation framework for memory-based membership leakage.

Key takeaway

For AI Security Engineers or developers deploying chat agents, you must recognize the significant privacy risks associated with sensitive data stored in agent memory. Your current privacy evaluations, often focused on training data, should extend to include memory-based membership inference attacks. Implement frameworks like MRMMIA to proactively test your agents' memory systems across black-box, gray-box, and white-box scenarios, ensuring robust protection against potential data leakage from user interactions and preferences.

Key insights

Chat agent memory is vulnerable to membership inference attacks, which MRMMIA effectively demonstrates across various access levels.

Principles

• MIAs measure privacy leakage in ML systems.
• Agent memory holds sensitive user interactions.
• Multi-recall probes enhance MIA effectiveness.

Method

MRMMIA employs multiple recall probes to a chat agent, extracting membership signals across black-box, gray-box, and white-box access settings to infer memory unit inclusion.

In practice

• Assess chat agent memory for privacy risks.
• Implement multi-recall probing for MIAs.
• Design agents with memory privacy in mind.

Topics

• Membership Inference Attacks
• Chat Agent Memory
• Privacy Leakage
• Black-box Attacks
• AI Security
• Data Privacy

Code references

• NExT-ChatV/NExT-Chat
• milad1378yz/MASPRM

Best for: CTO, Research Scientist, VP of Engineering/Data, AI Scientist, AI Security Engineer

Related on AIssential

• Detecting Data Contamination in Large Language Models — Black-box MIAs are currently ineffective at reliably detecting data contamination in LLM training corpora.
• The Decision to Verify: How Warmth and User Characteristics Shape Reliance on Conversational Agents for Information Search — User reliance on conversational AI persists despite fact-checking access, driven by prior trust and indirectly influenced by chatbot warmth.
• POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents — POLAR-Bench diagnoses LLM agent privacy-utility trade-offs, showing frontier models significantly outperform smaller, on-device counterparts.
• When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems — Embedding-based MAS defenses fail when attackers craft near-benign messages, necessitating token-level confidence signals for robustness.
• SuperLocalMemory: Privacy-Preserving Multi-Agent Memory with Bayesian Trust Defense Against Memory Poisoning — Local-first AI memory with Bayesian trust and adaptive re-ranking defends against poisoning and personalizes retrieval without cloud or LLMs.
• Agent Memory Patterns in Cognitive Science and AI Systems — Effective AI agents require structured memory systems mirroring human cognition to maintain context and learn.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.