Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models

2026-06-14 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

Deep Visual Residual MLLM (Deep-VRM) addresses the challenge of detecting increasingly realistic AI-generated images using multimodal large language models (MLLMs). While MLLMs offer robust semantic understanding for forensics, relying solely on semantic inconsistencies is insufficient for advanced fakes. The research investigates whether MLLMs can achieve full-spectrum forensic signal perception, capturing low-level generator artifacts without compromising pre-trained semantic knowledge. A layer-wise analysis revealed that semantic information is primarily formed in early-to-middle MLLM layers, and direct fine-tuning for artifact learning disrupts these crucial semantic representations. Deep-VRM overcomes this by preserving early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer. These signals are fused with semantic token representations and propagated, allowing later layers to jointly model semantic reasoning and signal-level forensic cues. This approach enables adaptive use of forensic signals, achieving leading detection performance across most benchmarks.

Key takeaway

For Computer Vision Engineers developing forensic tools against AI-generated content, Deep-VRM offers a critical advancement. You should consider integrating residual injection techniques to preserve semantic understanding while simultaneously capturing low-level generator artifacts. This method allows your models to adaptively utilize diverse forensic signals, significantly improving detection robustness and generalizability against increasingly sophisticated synthetic media.

Key insights

MLLMs can achieve full-spectrum forensic signal perception by integrating low-level artifact signals via residual injection without losing semantic understanding.

Principles

• Semantic information forms in early-to-middle MLLM layers.
• Direct artifact fine-tuning disrupts MLLM semantic representations.
• Adaptive signal use enhances forensic detection robustness.

Method

Deep-VRM preserves early semantic processing, injecting artifact-specific visual signals as a residual path into an intermediate layer for fusion with semantic tokens, enabling joint semantic and signal-level modeling.

In practice

• Apply Deep-VRM for robust AI-generated image detection.
• Integrate residual paths for low-level signal fusion.
• Employ adaptive signal modeling for generalizable forensics.

Topics

• Multimodal LLMs
• AI-Generated Image Detection
• Digital Forensics
• Residual Networks
• Computer Vision
• Semantic Representation

Code references

• KQL11/Deep-VRM

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• ForensicConcept: Transferable Forensic Concepts for AIGI Detection — ForensicConcept enables transferable forensic concept extraction for robust AI-generated image detection across unseen generators.
• Detecting Deepfakes with Multivariate Soft Blending and CLIP-based Image-Text Alignment — Multimodal image-text alignment and blended data augmentation significantly improve deepfake detection generalization and robustness.
• AI ASMR videos that fool humans AND VLMs? How close are we to peak fakery? — Synchronized AI-generated video and audio can fool both humans and advanced AI detection systems.
• Multi-Frequency Fusion for Robust Video Face Forgery Detection — Lightweight fusion of handcrafted cues can significantly improve face forgery detection accuracy with minimal model overhead.
• TAP into the Patch Tokens: Leveraging Vision Foundation Model Features for AI-Generated Image Detection — Modern vision foundation models significantly outperform CLIP for AI-generated image detection.
• Revealing Artifacts via Noise Amplification: A Novel Perspective for AI-Generated Video Detection — AI-generated video detection can be significantly improved by amplifying subtle noise signals extracted via bit-planes.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.