Preventing extinction from ASI on a $50M yearly budget

2026-04-21 · Source: AI Alignment Forum · Field: Government & Public Sector — Public Policy & Governance, Regulatory & Compliance, International Relations & Diplomacy · Depth: Intermediate, extended

Summary

ControlAI aims to prevent human extinction from superintelligent AI (ASI) by securing an international prohibition on its development, estimating a $50 million yearly budget is needed for a "concrete chance" within a few years. The organization's strategy centers on pervasive public and governmental awareness of ASI extinction risk, emphasizing that this specific risk, not general AI dislike, is the critical bottleneck. ControlAI employs a scalable, "industrial" approach to advocacy, breaking down goals into measurable intermediate steps, such as securing public support from lawmakers and tracking public declarations. With a $50 million budget, ControlAI plans to expand US and G7 policy advocacy, achieve billions of ad impressions in the US to reach millions of citizens, mobilize a grassroots movement, develop concrete policy proposals, and conduct thought-leader outreach, while also improving talent attraction and retention.

Key takeaway

For philanthropic institutions and major donors considering high-impact investments, ControlAI presents a structured, scalable approach to mitigating existential risk from superintelligent AI. Your funding of approximately $50 million annually could significantly advance efforts to establish an international prohibition on ASI development by fueling targeted policy advocacy, widespread public awareness campaigns, and grassroots mobilization, offering a concrete chance to address this global security threat.

Key insights

Pervasive, specific awareness of superintelligent AI extinction risk is the bottleneck to achieving an international development prohibition.

Principles

• Focus exclusively on ASI extinction risk.
• Directly communicate risks in plain terms.
• Scalable processes yield measurable progress.

Method

ControlAI uses a scalable, "industrial" approach by breaking down ambitious goals into measurable intermediate steps, akin to sales funnels, to track progress in policy advocacy and public awareness campaigns.

In practice

• Target G7 countries for policy advocacy.
• Utilize digital ads for broad public awareness.
• Build grassroots support via direct calls to action.

Topics

• Superintelligent AI
• Extinction Risk
• International Prohibition
• Policy Advocacy
• Public Awareness

Best for: Executive, Entrepreneur, Investor, Policy Maker, Consultant

Related on AIssential

• ControlAI 2025 Impact Report — Direct engagement with democratic institutions effectively raises awareness and drives action on superintelligence extinction risks.
• You can only build safe ASI if ASI is globally banned — Building controllable ASI requires knowledge that also enables easier, unsafe ASI, necessitating a global ban.
• Import AI 459: AI oversight is difficult; scaling laws for protein folding models; and pricing the extinction risk of AI systems — AI's rapid, hidden economic growth and complex safety challenges demand new measurement and governance frameworks.
• AI doom warnings are getting louder. Are they realistic? — AI existential risk concerns center on superintelligent systems with misaligned goals and superior capabilities.
• xAI says it raised $20B in Series E funding — xAI secured $20 billion in Series E funding amid international investigations into Grok's generation of illicit content.
• Summary: An International Agreement to Prevent the Premature Creation of Artificial Superintelligence — International cooperation is essential to prevent catastrophic risks from artificial superintelligence through verifiable controls.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.