AI is already reshaping the information environment in ways that benefit violent extremists, yet the counter-extremism sector remains under-prepared to deploy AI responsibly...

2025-11-28 · Source: Pascal’s Substack · Field: Government & Public Sector — Public Safety & Security, Public Policy & Governance, AI for Public Safety · Depth: Intermediate, medium

Summary

The UN Office of Counter-Terrorism's Practice Guide on AI and Preventing and Countering Violent Extremism (PCVE) argues for disciplined restraint in AI deployment within the PCVE sector. The guide highlights that AI amplifies existing dynamics like discrimination and surveillance, benefiting violent extremists through new propaganda production and targeted recruitment, while also weakening trusted information institutions. A survey of 120 respondents across 45 countries reveals that fewer than 25% use AI in PCVE, with only 10% among government respondents. Capacity is low, with 73% reporting no AI-related training. The guide identifies organizational unreadiness, policy vacuums, and fear of reputational blowback as key blockers, rather than a lack of AI capability. It also addresses risks like surveillance drift, weaponized predictive analytics, intellectual property theft, and the ineffectiveness of deepfake detection.

Key takeaway

For CTOs and Directors of AI/ML evaluating AI solutions for counter-extremism or similar high-stakes domains, recognize that organizational readiness and robust governance are more critical than raw AI capability. Your teams should prioritize human-in-the-loop systems, comprehensive risk assessments, and foundational training over "AI solutionism" to avoid unintended harm and maintain legitimacy. Focus on building ethical frameworks and ensuring transparency from the outset.

Key insights

AI amplifies both opportunities and risks in counter-extremism, demanding disciplined restraint and robust governance.

Principles

• AI amplifies existing dynamics, both positive and negative.
• Human-in-the-loop governance is crucial for high-risk AI decisions.
• Organizational readiness is a primary barrier to AI adoption.

Method

The guide proposes a pragmatic risk assessment method, scoring likelihood and impact to map risks to levels like 12–36 for low risk and 37–60 for medium risk, requiring mitigation for scores 6+.

In practice

• Prioritize capacity-building over unproven tech initiatives.
• Implement human review for high-risk AI decisions.
• Fund training and rights-based practices for AI in PCVE.

Topics

• AI in Counter-Extremism
• AI Ethics & Governance
• Predictive Analytics
• Deepfakes
• Human-in-the-Loop AI

Best for: CTO, VP of Engineering/Data, Director of AI/ML, Policy Maker, AI Ethicist, AI Operations Specialist

Related on AIssential

• AI as a Double-Edged Sword: Opportunities and Ethical Challenges — AI's dual nature demands responsible development, balancing innovation with ethical safeguards like transparency and human oversight.
• Today, with the unprecedented power of algorithmic radicalization and decentralized digital networks, the far right poses a more sophisticated threat... — Far-right ideology persisted post-WWII through adaptation, digital tools, and institutional inertia, posing a modern threat.
• Cyber Lack of Security and AI Governance — Advanced AI models like Mythos are rapidly enhancing cyber capabilities, forcing governments to confront regulatory challenges and inter-agency power struggles.
• Militaries mull the use of AI in war — The rapid, unconstrained advancement of AI is creating urgent military, safety, and societal governance challenges.
• Guiding a Safe Future for AI – Part 2 — AI's rapid advancement necessitates a focus on safety, ethical deployment, and robust infrastructure to serve humanity.
• The AI Ethics Brief #182: When Guardrails Fracture — Eroding international norms and operationalized surveillance tools are reshaping AI ethics and governance priorities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Pascal’s Substack.