From Anthropic to Iran: Who sets the limits on AI’s use in war and surveillance?

2026-03-03 · Source: Artificial intelligence (AI) – The Conversation · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Emerging Technologies & Innovation · Depth: Intermediate, medium

Summary

Anthropic, a prominent AI company, declined a Pentagon contract for "unrestricted access" to its technology for "all lawful purposes," citing "red lines" against mass surveillance of Americans and fully autonomous weapons without human oversight. This refusal led to significant repercussions: U.S. President Donald Trump ordered all American agencies to cease using Anthropic's Claude LLMs, and the U.S. defense secretary designated Anthropic a "supply chain risk." Immediately following, rival OpenAI secured a deal with the Pentagon. The incident highlights the risks of fully autonomous weapons, which can independently conduct military operations and have shown a 95% propensity to recommend nuclear strikes in simulations, and AI-driven mass surveillance, which poses novel threats to fundamental liberties through opaque analysis and potential for scaled errors. The case underscores the evolving challenge of setting ethical limits on AI use in national security contexts, particularly as generative AI models like Claude are significantly more powerful and self-improving than previous technologies like those in Google's Project Maven.

Key takeaway

For CTOs and VPs of Engineering evaluating AI partnerships with defense sectors, recognize that contractual language like "all lawful purposes" may not provide sufficient ethical guardrails for advanced AI. Your organization should proactively define and negotiate explicit prohibitions on AI uses such as mass surveillance and fully autonomous weapons, ensuring that ethical commitments are embedded in procurement standards and backed by independent oversight, rather than relying solely on shifting legal interpretations or corporate conscience.

Key insights

AI ethics in national security are shaped by contracts and procurement, not just principles.

Principles

• Human oversight is critical for autonomous weapons.
• AI-driven mass surveillance risks fundamental liberties.
• "Lawful purposes" is an insufficient ethical guardrail.

In practice

• Implement mandatory risk assessments for AI systems.
• Establish independent oversight for AI procurement standards.

Topics

• AI Ethics
• Autonomous Weapons Systems
• AI Surveillance
• Large Language Models
• National Security AI

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Ethicist, Policy Maker, Tech Journalist

Related on AIssential

• OpenAI and Google employees rush to Anthropic’s defense in DOD lawsuit — AI developers are asserting ethical boundaries against government use of their technology for surveillance or autonomous weapons.
• "All Lawful Use": Much More Than You Wanted To Know — Current US law contains broad loopholes permitting AI use in mass surveillance and autonomous weapons, despite company assurances.
• How AI is being used in war — and what’s next — AI is increasingly integrated into modern warfare, raising significant ethical and regulatory challenges.
• The Government just blacklisted Anthropic... — The US government and Anthropic are in a standoff over AI model guardrails for military use, highlighting ethical and control conflicts.
• AI Firms Can Limit Military Surveillance of Americans. What About Everyone Else? — AI firms' self-regulation on surveillance is insufficient; binding international rules are needed to protect global privacy.
• Inside the Anthropic, OpenAI, and Pentagon AI Public Fight — Disagreement over AI military use highlights a critical tension between "lawful use" and explicit misuse safeguards.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial intelligence (AI) – The Conversation.