Ai Safety Is Becoming Ai Security

2025-02-14 · Source: Our Work | Center for AI Policy (CAIP) · Field: Government & Public Sector — Public Policy & Governance, Regulatory & Compliance, Artificial Intelligence & Machine Learning · Depth: Fundamental Awareness, quick

Summary

The UK's AI Safety Institute (AISI) rebranded to the AI Security Institute on February 14, 2025, reflecting an evolving perspective on AI risks and a potentially dangerous narrowing of focus. This change highlights how English differentiates "safety" (inherent faults) from "security" (adversarial elements), unlike Chinese. The rebranding aligns with two broader AI policy trends: viewing technology as national power, particularly concerning US-China capabilities, and the politicization of "AI safety" to include emotional harm. The new institute will focus exclusively on harms with clear perpetrators, such as biological weapons, cyberattacks, fraud, and child sexual abuse, explicitly excluding bias and freedom of speech. This narrow scope, however, overlooks objective harms from AI malfunction, including strategic deception or resistance to shutdown, which could have devastating consequences in critical infrastructure or financial markets.

Key takeaway

For policymakers shaping national AI strategy, focusing solely on AI security risks, such as misuse by adversaries, overlooks critical inherent technical failures. You must ensure your institute's mandate covers both adversarial threats and the potential for AI systems to malfunction or act unpredictably. Prioritize a comprehensive approach to AI risk management to prevent devastating consequences in critical sectors.

Key insights

The UK's AI Safety Institute rebranding to AI Security Institute signals a policy shift towards adversarial risks, potentially neglecting inherent system failures.

Principles

• AI policy is increasingly framed as national power.
• Differentiating "safety" from "security" impacts risk focus.
• Over-narrowing AI risk scope can overlook critical failures.

In practice

• Consider both AI misuse and inherent technical risks.
• Evaluate policy scope for comprehensive AI risk management.

Topics

• AI Safety
• AI Security
• AI Policy
• Risk Management
• National Security
• Technical Risks

Best for: CTO, VP of Engineering/Data, Director of AI/ML, Policy Maker, AI Ethicist, Tech Journalist

Related on AIssential

• A Recommendation For The First Meeting Of Ai Safety Institutes — The International Network of AI Safety Institutes must re-center global discussions on AI safety to address escalating risks effectively.
• Ai Safety And The Us China Arms Race — AI safety measures are compatible with US national competitiveness and do not hinder its lead in innovation.
• Germany's National Security Council greenights an AI Safety Institute modeled after the UK's AISI — The establishment of DE-AISI reflects Germany's strategic move to independently assess frontier AI risks, mirroring the UK's successful model.
• We're frozen out (for good?) — Government actions are bifurcating AI access, creating restricted frontier models for strategic uses and general-purpose versions for broader civilian use.
• The one question everyone should be asking after OpenAI’s deal with the Pentagon — Existing generative AI safety guardrails are insufficient for high-stakes applications like surveillance and military operations.
• AI #176 Part 2: Plan B — The rapid, accelerating advancement of AI capabilities is outpacing policy, safety, and ethical frameworks, creating urgent, complex challenges.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Our Work | Center for AI Policy (CAIP).