Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Models

2026-06-25 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

A new survey, "Adversarial Diffusion Across Modalities," integrates four previously disconnected research tracks concerning adversarial evaluation of AI systems: diffusion-based attacks on text/LLMs, image classifiers, vision-language model jailbreaks, and diffusion-based input purification defenses. Published on 2026-06-25, this meta-research effort unifies these areas into a single conceptual framework with a shared taxonomy, evaluation criteria, and research agenda, specifically focusing on the LLM domain. The survey catalogs 50 published papers across four scope areas, including four diffusion-LLM-as-victim entries and ten non-diffusion baselines. It proposes a six-class taxonomy for diffusion roles in adversarial pipelines, complemented by a threat-model axis, and applies a five-dimension evaluation framework uniformly across modalities. The analysis also covers four diffusion-based defenses and identifies five recurring weaknesses in the current LLM-side literature, concluding with open questions and experimental designs.

Key takeaway

For AI Security Engineers or ML Scientists developing adversarial defenses or attacks, this survey provides a critical unified framework. You should adopt its proposed six-class diffusion taxonomy and five-dimension evaluation framework to standardize your research and compare results across modalities effectively. This approach helps identify current LLM-side literature weaknesses and guides future experimental designs, ensuring your work contributes to a more robust and comparable adversarial evaluation landscape.

Key insights

The survey unifies disparate adversarial diffusion research across modalities, providing a common framework for attacks, defenses, and evaluation.

Principles

• Adversarial evaluation benefits from unified taxonomies.
• Diffusion models are a shared generative mechanism.
• A dual attacker-defender perspective is crucial.

Method

The survey integrates four adversarial evaluation tracks into a single framework, proposing a six-class diffusion role taxonomy, a threat-model axis, and a five-dimension evaluation framework applied uniformly across modalities.

In practice

• Compare new attacks against ten non-diffusion baselines.
• Use the six-class diffusion taxonomy for pipeline analysis.
• Apply the five-dimension evaluation framework.

Topics

• Adversarial Diffusion
• Large Language Models
• Vision-Language Models
• Adversarial Attacks
• Adversarial Defenses
• Evaluation Frameworks

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, AI Security Engineer, Machine Learning Engineer

Related on AIssential

• Disentangling Adversarial Prompts: A Semantic-Graph Defense for Robust LLM Security — A semantic-graph defense framework proactively disentangles adversarial prompt components to secure LLMs against jailbreaking and injection attacks.
• Understanding Cross-Modal Contributions in Continual Vision-Language Models: A Theoretical Perspective — A new theoretical perspective clarifies cross-modal contributions in continual vision-language models, addressing catastrophic forgetting.
• PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models — The article introduces a large-scale, pre-generated dataset, PHANTOM, to democratize VLM adversarial attack research and improve safety evaluations.
• Diffusion Language Models: An Experimental Analysis — Diffusion Language Models' performance and efficiency are critically shaped by generation-time design choices.
• Systematic Discovery of Semantic Attacks in Online Map Construction through Conditional Diffusion — Semantic-level adversarial attacks exploiting diffusion model latent spaces bypass standard defenses and appear realistic.
• The Little Book of Generative AI Foundations: An Intuitive Mathematical Primer — This book offers a derivation-oriented mathematical primer to unify understanding across major generative AI model families.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.