Import AI 459: AI oversight is difficult; scaling laws for protein folding models; and pricing the extinction risk of AI systems
Summary
New analyses reveal the US "AI economy" is growing at an unprecedented 2,000% annually, projected to reach \$250 billion in 2025 with 2,600% quality-adjusted real growth, yet this expansion remains largely invisible in conventional GDP statistics due to falling per-unit prices and AI's potential as a labor substitute. Concurrently, researchers from the UK AI Security Institute detail why automated AI alignment is challenging, citing issues like optimization pressure and non-human-evaluable arguments. In other developments, Stanford University and collaborators released GPIC, a 100 million-image dataset permissively licensed for research and commercial use, while Biohub introduced ESMFold2, a protein prediction model that outperforms AlphaFold 3 in some benchmarks and aids cancer research. An Australian Assistant Minister also urged economists to better price AI extinction risks, emphasizing that current economic models fail to account for irreversible, civilization-scale harms.
Key takeaway
For policymakers and economists evaluating AI's societal implications, it is crucial to recognize that current GDP metrics significantly understate AI's rapid economic expansion and its potential to disrupt labor markets. You must adapt economic measurement frameworks, such as implementing "AI satellite accounts," and integrate AI productive-capacity into projections to accurately assess future tax bases and prepare for potential labor-tax-base shocks. Simultaneously, prioritize funding and developing robust AI safety and oversight mechanisms, including red teaming automated alignment programs, to mitigate existential risks that conventional economic models currently fail to price.
Key insights
AI's rapid, unmeasured economic growth and complex safety challenges demand new frameworks for assessment and governance.
Principles
- AI's economic impact is largely invisible in conventional GDP statistics.
- Automated AI alignment faces unique, hard-to-identify error modes.
- Protein prediction models exhibit scaling laws linked to compute and parameters.
Method
To measure AI economy: develop "AI satellite accounts," generate better data, and incorporate AI capacity into projections. For AI safety: recreate research, test agent prediction, use mechanistic interpretability, develop scalable oversight, and red team alignment programs.
In practice
- Utilize the Giant Permissive Image Corpus (GPIC) for large-scale, permissibly licensed visual generation research.
- Apply Biohub's ESMFold2 for accelerated, computation-guided therapeutic binder discovery in cancer research.
- Implement "AI satellite accounts" within statistical agencies to capture AI's true economic growth.
Topics
- AI Economy
- Economic Measurement
- AI Safety
- Protein Folding
- Large Datasets
- Existential Risk
- AI Governance
Best for: Research Scientist, Policy Maker, AI Scientist
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Import AI.