New RFP on Interpretability from Schmidt Sciences

2026-03-17 · Source: AI Alignment Forum · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, medium

Summary

Schmidt Sciences has launched a "Request for Proposals" (RFP) for a pilot program in AI interpretability, seeking methods to detect and mitigate deceptive behaviors in "Large Language Models" (LLMs) by May 26, 2026. The program focuses on three key directions: detecting contradictions between an LLM's output and its internal representation, steering models to enhance truthfulness through mechanistic understanding, and applying these techniques to improve "human-AI collaboration" and "multi-agent systems". Successful proposals, eligible for \$300k-\$1M over 1-3 years, should leverage interpretability to outperform baselines not relying on weight access and address concrete risks beyond academic benchmarks. Schmidt Sciences also offers significant "compute resources", "software engineering support", and "API credits" to grantees, emphasizing ambitious, field-shaping contributions to "AI safety".

Key takeaway

Schmidt Sciences invites proposals for AI interpretability research focused on detecting and mitigating deceptive LLM behaviors like misleading advice or false claims. Projects, funded \$300k-\$1M over 1-3 years, must develop methods leveraging model internals to outperform black-box baselines in identifying and steering model reasoning. This initiative aims to advance universal deception detection and reliable truthfulness steering, crucial for building safer, more trustworthy AI systems and human-AI collaboration.

Topics

• AI Interpretability
• Deceptive AI Behaviors
• LLM Steering
• Mechanistic Interpretability
• AI Safety

Best for: Research Scientist, AI Researcher, AI Scientist, AI Ethicist

Related on AIssential

• Making AI systems more transparent and trustworthy: an interview with Ximing Wen — Interpretable AI models can achieve high accuracy by showing reasoning and evidence, crucial for trust.
• The Download: a new Christian phone network, and debugging LLMs — Technological and policy shifts are impacting content access, AI development, scientific funding, and global AI strategy.
• Project Ariadne: A Structural Causal Framework for Auditing Faithfulness in LLM Agents — LLM agent reasoning traces often act as "Reasoning Theater," not faithful generative drivers, due to Causal Decoupling.
• Interpretability — Understanding LLM internal mechanisms is crucial for AI safety and mitigating undesirable behaviors.
• Test your best methods on our hard CoT interp tasks — New objective tasks and a testbed are released to advance CoT interpretability beyond simple reasoning analysis.
• LLMs Can’t Provide Faithful Explanations Needed for AI Accountability — LLMs struggle to provide faithful explanations, hindering accountability in high-stakes AI applications.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.