Summary: An International Agreement to Prevent the Premature Creation of Artificial Superintelligence

2026-05-12 · Source: Machine Intelligence Research Institute · Field: Legal & Regulatory — Regulatory Affairs & Government Relations, Compliance & Risk Management, Artificial Intelligence & Machine Learning · Depth: Intermediate, long

Summary

MIRI's Technical Governance Team has published a draft international agreement titled "An International Agreement to Prevent the Premature Creation of Artificial Superintelligence," aiming to prevent catastrophic outcomes from superhuman AI development. Modeled on nuclear nonproliferation treaties, the agreement proposes a bilateral framework initially between the U.S. and China, with provisions for other nations to join. It outlines fifteen articles, focusing on restricting large-scale AI training runs by setting FLOP thresholds (e.g., 10^24 FLOP prohibited) and controlling the hardware supply chain, particularly NVIDIA H100-equivalent chips. The agreement also proposes restricting research in sensitive AI areas and establishing an Executive Council and a Coalition Technical Body (CTB) for governance, verification, and dispute resolution, including the right to targeted military action in extreme cases.

Key takeaway

For CTOs and VPs of Engineering evaluating long-term AI strategy, this draft agreement signals a potential future regulatory landscape. You should begin assessing your organization's compute infrastructure and research areas against proposed thresholds like 10^22 FLOP for monitoring or 16 H100-equivalents for facility oversight. Proactive engagement with emerging AI governance frameworks, even in draft form, will be critical for future compliance and strategic planning, especially concerning large-scale training runs and hardware procurement.

Key insights

International cooperation is essential to prevent catastrophic risks from artificial superintelligence through verifiable controls.

Principles

• Preventing proliferation of dangerous AI.
• Controlling hardware bottlenecks is key.
• Verification is crucial in low-trust environments.

Method

The proposed method involves a bilateral agreement (U.S.-China) to restrict AI training via FLOP thresholds and hardware monitoring, establish a Coalition Technical Body for verification, and control sensitive AI research.

In practice

• Monitor AI chip supply chains.
• Consolidate large AI compute clusters.
• Develop secure AI model testing protocols.

Topics

• Artificial Superintelligence
• International Agreement
• AI Governance
• Compute Thresholds
• AI Chip Supply Chain

Best for: Investor, CTO, VP of Engineering/Data, Policy Maker, Legal Professional, AI Ethicist

Related on AIssential

• My AI Opinions — The future of AI involves complex, probabilistic timelines for AGI, superintelligence, and critical safety and geopolitical challenges.
• The US Is Fighting for Control of AI. It Would Be Better Off Building Standards. — US leadership in AI requires establishing global standards for interoperability, not strong-arming private firms.
• Zero knowledge verification for frontier AI training is possible — Zero-knowledge proofs can technically verify frontier AI training, enabling enforceable governance beyond self-reporting.
• Solipsistic Superintelligence is Unlikely to be Cooperative — Superintelligence designed without considering interdependence will likely be uncooperative, necessitating a non-solipsistic research paradigm.
• The AI Doc: Your Questions Answered — AI's rapid, opaque development necessitates an international ban to prevent unsteerable superintelligence.
• Promising Signals on AI Governance from China — China advocates for international cooperation and robust governance frameworks to mitigate AI's existential risks and ensure human control.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Intelligence Research Institute.