How to actually protect against digital sexualized violence

2026-04-30 · Source: AlgorithmWatch · Field: Legal & Regulatory — Compliance & Risk Management, Regulatory Affairs & Government Relations, Criminal Law & Public Safety · Depth: Intermediate, medium

Summary

AlgorithmWatch has issued recommendations on April 30, 2026, for implementing a ban on non-consensual sexualized deepfakes within the European AI Act's AI Omnibus procedure. Generative AI tools are increasingly exploited for sexualized violence against women and girls, creating deepfakes and undressed bodies shared on platforms like X, causing severe psychological distress and intimidation. The current regulatory landscape is fragmented, with gaps in protection and enforcement, exemplified by Germany's lack of a comprehensive legal framework. AlgorithmWatch advocates for a ban that precisely defines consent, clearly assigns liability to AI companies, strengthens security measures, and mandates consent requests for real people, while avoiding overregulation of open-source AI. The organization also stresses the need for platforms to be held accountable for content dissemination and enforcement, citing X's refusal to provide data for DSA-mandated risk investigations. Effective protection requires consistent accountability for AI companies, platforms, and perpetrators, alongside societal changes to support victims.

Key takeaway

For legal professionals drafting or enforcing digital violence legislation, you must ensure clear, enforceable obligations for AI companies and platforms. Your frameworks should precisely define consent, assign unambiguous liability for non-consensual deepfakes, and mandate robust security measures. Additionally, you should push for consistent punishment and substantial fines for platforms failing to mitigate systemic risks, like X's refusal to provide data, to truly protect victims and shift the burden from affected individuals to perpetrators and enablers.

Key insights

Effective protection against digital sexualized violence demands accountability from AI companies, platforms, and perpetrators via robust legislation.

Principles

• Define consent as free, informed, and explicit.
• Assign clear liability for AI system misuse.
• Platforms must mitigate systemic risks.

Method

AlgorithmWatch collects and reports non-consensual sexualization tools (NSTs) to platform providers for removal, while advocating for stricter legislation and enforcement against digital sexualized violence.

In practice

• Report non-consensual sexualization tools (NSTs).
• Advocate for stronger legislative enforcement.
• Implement continuous AI safeguard monitoring.

Topics

• Digital Sexualized Violence
• AI Deepfakes
• EU AI Act
• Digital Services Act
• Platform Accountability
• Generative AI Ethics
• Consent Definition

Best for: CTO, VP of Engineering/Data, Director of AI/ML, Policy Maker, Legal Professional, AI Ethicist

Related on AIssential

• New Study Examines Features and Policies for 29 AI ‘Undressing’ Apps — AI "undressing" apps are vehicles for systemic gender-based violence, objectifying women and commodifying their exploitation.
• Chinese activist in UK told by X that abusive deepfakes do not breach rules — X's content moderation failed to protect a Chinese activist from deepfake abuse, raising concerns about platform accountability and protection for dissidents.
• New claimants seek to sue Elon Musk’s xAI after Labour MP’s test case — AI developers face increasing legal liability for harmful content generated by their models, particularly regarding design choices.
• YouTube and X Have Become ‘Gateways’ to Nudify Apps — Mainstream social media platforms inadvertently enable the proliferation of nonconsensual deepfake tools despite explicit policies.
• FBI agent explains how easy it is to ID people posting AI porn without consent — Law enforcement can readily identify individuals posting nonconsensual AI deepfakes using digital footprints and public records.
• Non-consensual AI porn doesn’t violate privacy – but it’s still wrong — Generative AI fuels deepfake pornography, exposing legal gaps and raising complex ethical questions beyond privacy violations.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AlgorithmWatch.