Systematic Discovery of Semantic Attacks in Online Map Construction through Conditional Diffusion

2026-05-15 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

Mirage is a novel framework introduced in 2026 for systematically discovering semantic attacks against camera-based online HD map construction in autonomous vehicles. Unlike traditional pixel-level perturbations, Mirage exploits the latent manifold of real-world driving data learned by diffusion models to find plausible environmental variations, such as shadows or wet roads, that degrade mapping predictions. Evaluated on the nuScenes dataset, Mirage demonstrates two key attacks: "boundary removal," which suppresses 57.7% of detections and corrupts 96% of planned trajectories, and "boundary injection," which is the only method tested that successfully injects fictitious boundaries, increasing detections by 1.88 per scene. These semantic attacks remain potent against standard adversarial defenses like JPEG compression, median filtering, and DiffPure, which largely neutralize pixel-level attacks. Mirage-generated adversarial samples are rated as realistic by two independent VLM judges 80-84% of the time, significantly higher than pixel PGD (28-52%) and AdvPatch (0-9%).

Key takeaway

For Computer Vision Engineers and Research Scientists developing autonomous driving systems, this work highlights a critical gap in current adversarial defenses. You should prioritize developing and deploying semantic anomaly detection, temporal consistency enforcement, and multi-sensor fusion techniques, as pixel-level defenses are insufficient against plausible, distribution-consistent environmental manipulations discovered by tools like Mirage. Incorporate semantic adversarial examples into your training and testing pipelines to harden perception models against these stealthier threats.

Key insights

Semantic-level adversarial attacks exploiting diffusion model latent spaces bypass standard defenses and appear realistic.

Principles

• Semantic perturbations are harder to mitigate than pixel-level noise.
• Diffusion models can synthesize coherent, boundary-like visual features.
• CLIP guidance can steer latent perturbations towards plausible environmental changes.

Method

Mirage inverts ground-truth images into a diffusion model's latent space, then searches for nearby latents using ControlNet and CLIP direction loss to generate semantically mutated scenes that mislead mapping predictions while preserving road topology.

In practice

• Use Mirage to red-team AV perception stacks for semantic vulnerabilities.
• Implement multi-sensor cross-validation to counter visual semantic attacks.
• Integrate adversarial environmental augmentation into AV training data.

Topics

• Semantic Attacks
• Online HD Map Construction
• Conditional Diffusion Models
• Autonomous Driving Security
• Latent Space Perturbation

Code references

• WiSeR-Lab/MIRAGE

Best for: Computer Vision Engineer, Research Scientist, AI Security Engineer, AI Scientist, Robotics Engineer

Related on AIssential

• Adversarial Flow Matching for Imperceptible Attacks on End-to-End Autonomous Driving — AFM leverages Flow Matching to create imperceptible, transferable gray-box attacks against Transformer-based autonomous driving systems.
• Watch Your Step: Information Injection in Diffusion Models via Shadow Timestep Embedding — Diffusion model timestep embeddings offer a covert channel for injecting information without altering visible model architecture or training objectives.
• Generalised Eigenvalue Geometry of Semantic Adversarial Attacks — Semantic adversarial vulnerability is governed by the relative local geometry of proxy and target embedding models.
• Roots Beneath the Cut: Uncovering the Risk of Concept Revival in Pruning-Based Unlearning for Diffusion Models — Pruning-based unlearning in diffusion models is vulnerable to concept revival via side-channel information from pruned weight locations.
• Proto-LeakNet: Towards Signal-Leak Aware Attribution in Synthetic Human Face Imagery — Proto-LeakNet exploits diffusion model "signal leaks" in latent space for robust, interpretable synthetic image source attribution.
• Physically-Induced Atmospheric Adversarial Perturbations: Enhancing Transferability and Robustness in Remote Sensing Image Classification — FogFool generates physically plausible fog-based adversarial perturbations for robust remote sensing image classification attacks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.