Hugging Face Has a Deepfake Nudes Problem

2026-07-28 · Source: WIRED - Ai · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Fundamental Awareness, medium

Summary

A new report by the European nonprofit AI Forensics reveals that the open-source AI platform Hugging Face has a widespread problem with nonconsensual deepfake nudes. Researchers tested nine top image editing models on Hugging Face, finding seven could easily transform a clothed image of a woman into a topless one using a simple six-word prompt. A "honey-pot" experiment further tracked over 1,000 user prompts, with 73 percent being sexual in nature, and 83 percent of those seeking to undress or sexualize the submitted photo's subject, 95 percent of whom were women. Alarmingly, 6.7 percent of sexual requests targeted apparent children. Unlike mainstream generative AI models, the open-source models inspected by AI Forensics appeared to lack safety mechanisms, allowing such harmful content generation. This highlights a significant gap in platform-level content moderation and safeguards.

Key takeaway

For AI Ethicists and Policy Makers evaluating platform responsibility, this report underscores the urgent need for robust content moderation on open-source AI platforms. You must advocate for mandatory safety mechanisms and proactive filtering at the platform level, not solely relying on individual model developers. Your policies should address the documented high user intent for creating nonconsensual intimate images, including those targeting children, to mitigate widespread harm.

Key insights

Open-source AI platforms like Hugging Face host models easily exploited for nonconsensual deepfake nudes due to absent safeguards.

Principles

• Open-source AI models often lack inherent safety mechanisms.
• User intent for harmful content is demonstrably high.
• Platform-level moderation is crucial for preventing abuse.

Method

AI Forensics tested top image editing models and deployed a "honey-pot" Space to track over 1,000 user prompts, analyzing their sexual and undressing intent.

In practice

• Implement platform-level content filtering for AI models.
• Audit open-source models for deepfake generation capabilities.
• Monitor user prompt data for abuse patterns.

Topics

• Deepfake Nudes
• Nonconsensual Intimate Images
• Hugging Face
• AI Content Moderation
• Open-Source AI Safety
• Generative AI Harms

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Ethicist, Policy Maker, Tech Journalist

Related on AIssential

• Hugging Face hosts nudification tools targeting a former Trump cabinet official and other senior US political figures — Hugging Face hosts deepfake nudification tools targeting political figures, violating its own content policies despite prior investigations and public statements.
• The problem AI content moderation cannot solve — AI content moderation fails to address image-based abuse because it cannot account for consent and context, focusing only on explicit content.
• Wave of Grok AI fake images of women and girls appalling, says UK minister — Grok AI's generation of non-consensual intimate deepfakes highlights urgent regulatory and platform responsibility gaps.
• How the AI Deepfake Boom is Outpacing Europe's Safeguards — AI-powered "nudification" tools enable widespread, realistic sexualized deepfakes, exacerbating digital violence and outpacing regulatory and institutional responses.
• OpenAI’s own mental health experts unanimously opposed “naughty” ChatGPT launch — OpenAI's "adult mode" launch faces internal expert opposition and significant safety concerns regarding user well-being and minor access.
• YouTube and X Have Become ‘Gateways’ to Nudify Apps — Mainstream social media platforms inadvertently enable the proliferation of nonconsensual deepfake tools despite explicit policies.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by WIRED - Ai.