Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026)

2026-05-08 · Source: METR · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, quick

Summary

METR conducted an external review of the "Risks from automated R&D" section within Anthropic's February 2026 Risk Report, which asserts that catastrophic risk from Claude Opus 4.6 or less capable Anthropic models automating R&D is very low. METR identified significant issues with Anthropic's analytical rigor, citing problems with the model use survey's sample size, question granularity, and framing, concluding the survey provided little evidence for the risk level. Concerns also included the report's failure to consider AI R&D acceleration before full automation and a misrepresentation of survey results by miscounting a missing response. Despite these criticisms of Anthropic's methodology and evidence presentation, METR ultimately concurs with the report's bottom-line conclusion that catastrophic risk from Opus 4.6 automating R&D is very low, supported by METR's own evaluations and the absence of public reports of the model automating key domains.

Key takeaway

For AI Ethicists or Policy Makers evaluating AI risk reports, recognize that a report's conclusion can be valid even if its supporting evidence is flawed. You should critically assess the analytical rigor, survey methodology, and evidence presentation within any risk assessment. Supplement internal report findings with external evaluations and real-world model performance data. This approach ensures a robust understanding of AI risks, moving beyond potentially inadequate internal justifications.

Key insights

METR agrees with Anthropic's low-risk conclusion for Opus 4.6 R&D automation, despite finding the report's supporting evidence inadequate.

Principles

• Survey design impacts evidence validity.
• Consider AI acceleration beyond full automation.
• External evaluations provide crucial evidence.

Method

The article describes METR's review process, which involved assessing Anthropic's case, evaluating analytical rigor, information adequacy, and risk reduction recommendations, supplemented by non-public materials and METR's own evaluations.

In practice

• Improve survey sample size and granularity.
• Frame survey questions carefully.
• Report diverse leading indicators of AI progress.

Topics

• AI Risk Assessment
• Automated R&D
• Claude Opus 4.6
• Model Evaluation
• Survey Methodology
• AI Safety

Best for: AI Ethicist, Policy Maker, Consultant

Related on AIssential

• Review of the Anthropic Sabotage Risk Report: Claude Opus 4.6 — Catastrophic outcomes from Claude Opus 4.6's misaligned actions are deemed very low but not negligible.
• ‘Safety first’ puts Anthropic ahead in game of AI spin — Independent validation of AI safety claims is crucial, especially when public releases are restricted.
• Weekly Review 26 December 2025 — AI's rapid expansion creates complex challenges across data, ethics, economics, and resource consumption.
• 🗞️ Anthropic published a 53 page Sabotage risk report for Opus 4.6 — AI development is accelerating, bringing both advanced capabilities and complex safety, ethical, and infrastructure challenges.
• Fraud and the false optimism of AI for science — The ethical status of AI-generated scientific papers hinges on human oversight, intent, and the preservation of scientific integrity.
• AI Policy Lab Day 2025: Highlights and Reflections (Recording Available) — AI policy requires continuous reflection on risks and opportunities, integrating social realities and collective well-being.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by METR.