Adversarial Agents: Black-Box Evasion Attacks with Reinforcement Learning

2026-06-06 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, extended

Summary

A novel reinforcement learning (RL) approach, "Adversarial Agents," is introduced for black-box evasion attacks against machine learning models. This method formulates adversarial example generation as a Markov Decision Process, allowing agents to learn and exploit past attack experiences to improve future attacks, unlike traditional optimization-based methods. Evaluated on the CIFAR-10 dataset against a ResNet50 victim model using the PPO algorithm, the RL-based agent significantly improves attack effectiveness and efficiency. Specifically, it increased the success rate of adversarial examples by 19.4% and reduced the median number of victim model queries by 53.2% during training. In a head-to-head comparison with a leading image attack, SquareAttack, the RL approach generated 13.1% more successful adversarial examples after 5000 training episodes, demonstrating a powerful new attack vector for efficiently attacking ML models at scale.

Key takeaway

For AI Security Engineers evaluating model robustness, this research highlights a new, efficient black-box attack vector. You should consider that reinforcement learning agents can learn to generate adversarial examples with 13.1% higher success than traditional methods after 5000 attacks, requiring 53.2% fewer queries. This implies existing defenses against static attacks may be insufficient. Prioritize developing dynamic defenses that adapt to learning adversaries, and regularly test models against RL-driven evasion techniques.

Key insights

RL agents can learn to generate black-box adversarial examples more efficiently and effectively by utilizing past attack experience.

Principles

• Adversarial example generation can be modeled as an MDP.
• Learning from attack experience improves future attack efficacy.
• Hyperparameters like ε and c balance attack objectives.

Method

Formulate adversarial example generation as a Markov Decision Process. Train an RL agent (e.g., using PPO) to learn perturbation policies, optimizing for either maximum loss or minimum distortion, then use the learned policy to craft adversarial examples.

In practice

• Model adversarial example generation as an MDP for learning attacks.
• Use RL to reduce victim model queries in black-box attacks.
• Tune ε or c to balance distortion and misclassification goals.

Topics

• Reinforcement Learning
• Adversarial Machine Learning
• Black-Box Attacks
• Evasion Attacks
• Markov Decision Process
• CIFAR-10

Best for: Research Scientist, Computer Vision Engineer, AI Scientist, AI Security Engineer, Machine Learning Engineer

Related on AIssential

• Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimization — Adversarial suffix optimization can mislead black-box LLM routers into selecting expensive, high-capability models.
• Beyond Rewards in Reinforcement Learning for Cyber Defence — Sparse, goal-aligned reward functions enhance DRL cyber defense agent effectiveness, reliability, and risk profiles.
• When Actions Disappear: Adversarial Action Removal in Self-Play Reinforcement Learning — Adversarial action removal severely degrades self-play RL agents by targeting high-value decision points, with no recovery.
• Pushing the Frontier of Black-Box LVLM Attacks via Fine-Grained Detail Targeting — M-Attack-V2 improves black-box LVLM attacks by denoising gradients and refining local alignment for better transferability.
• This new AI-powered computer worm can learn to attack any device - ThePrint — AI-powered worms can adaptively exploit diverse vulnerabilities, posing an unprecedented cybersecurity threat.
• A No-Defense Defense Against Gradient-Based Adversarial Attacks on ML-NIDS: Is Less More? — Careful DNN architectural choices can inherently reduce adversarial vulnerability in NIDS.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.