Amount of AI-generated child sexual abuse material found online surged in 2025

2026-03-24 · Source: AI (artificial intelligence) | The Guardian · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Fundamental Awareness, quick

Summary

The Internet Watch Foundation (IWF) reported a significant surge in AI-generated child sexual abuse material (CSAM) identified online in 2025, with a 14% overall increase. The watchdog verified 8,029 pieces of realistic AI-made CSAM images and videos, noting a more than 260-fold increase specifically in videos. Of the 3,443 AI-generated videos, 65% were classified as category A, the most severe under UK law, compared to 43% for non-AI videos, indicating AI's use in creating more violent content. The IWF highlighted discussions among offenders on the dark web regarding AI's improving realism, ability to add audio, and manipulate real child imagery. The UK government is enabling tech companies and child protection agencies to test AI tools for CSAM generation and has banned possessing, creating, or distributing AI models designed for such material.

Key takeaway

For CTOs and VPs of Engineering developing generative AI, your teams must prioritize safety-by-design, especially concerning content generation. The dramatic increase in AI-generated CSAM, particularly severe content, underscores the urgent need for robust safeguards and proactive testing of models to prevent misuse. Ensure your AI systems are developed with "future-proofed" harm prevention and comply with evolving legislation, as lives are directly impacted by these technological advancements.

Key insights

AI advancements are significantly increasing the volume and severity of online child sexual abuse material.

Principles

• Technology advances must not compromise child safety.
• AI systems require robust safety safeguards.
• Legislation is crucial for AI safety development.

Method

The IWF monitors online content globally, including dark web discussions, to identify and classify AI-generated CSAM, using UK legal classifications like Category A for severity.

In practice

• Test AI tools for CSAM generation capabilities.
• Implement safeguards in generative AI models.
• Monitor dark web for AI misuse trends.

Topics

• AI-generated CSAM
• Generative AI misuse
• AI safety regulation
• Content moderation
• Deepfake technology

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Ethicist, Policy Maker, AI Security Engineer

Related on AIssential

• Introducing the Child Safety Blueprint — A new blueprint outlines a multi-faceted approach to combat AI-enabled child sexual exploitation through legal, operational, and technical safeguards.
• X blames users for Grok-generated CSAM; no fixes announced — X's existing CSAM moderation, reliant on hash technology, struggles with novel AI-generated content from Grok.
• AI Toys Reach Children Before Privacy and Safety Rules Catch Up — AI toys are deployed faster than privacy, safety, and child development research can establish adequate safeguards.
• GenAI's 'Double-Edged Sword': Safer Internet Day 2026 — AI safety, deepfakes, and governance are central to Safer Internet Day 2026, highlighting global regulatory divergence and rising corporate risk.
• Wave of Grok AI fake images of women and girls appalling, says UK minister — Grok AI's generation of non-consensual intimate deepfakes highlights urgent regulatory and platform responsibility gaps.
• Elon Musk’s Grok AI generates images of ‘minors in minimal clothing’ — AI models require robust, continuously updated safeguards to prevent the generation of harmful and illegal content.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI (artificial intelligence) | The Guardian.