The Great Security Update: AI ∧ Formal Methods with Kathleen Fisher of RAND & Byron Cook of AWS

2025-12-24 · Source: The Cognitive Revolution · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Software Development & Engineering · Depth: Advanced, extended

Summary

Kathleen Fisher of RAND and Byron Cook of AWS discuss the critical role of automated reasoning and formal verification in securing software and AI systems. They highlight how these methods can harden critical infrastructure against AI-enabled cyberattacks and improve the security of coding models. The conversation covers the application of formal methods in AWS's automated reasoning checks for AI agents and policy compliance, emphasizing the combination of assumptions, specifications, and proofs to deliver security guarantees. They note that while AI enhances cyberattack capabilities across all expertise levels, it also offers a path to significantly reduce software vulnerabilities through the generation of provably secure code. The discussion also touches on the challenges of translating natural language policies into formal rules and the iterative process of refining these specifications.

Key takeaway

For CTOs and VPs of Engineering grappling with escalating AI-driven cyber threats, integrating formal methods with generative AI is no longer optional. Your teams should prioritize adopting tools like AWS's automated reasoning checks to formalize policies and verify AI agent outputs, ensuring a higher standard of code security. This approach can significantly reduce vulnerabilities, moving towards a future where software is inherently more robust against sophisticated attacks, thereby mitigating operational risks and compliance challenges.

Key insights

Formal methods, augmented by AI, offer a path to provably secure software and AI systems, mitigating escalating cyber threats.

Principles

• 100% security is not a realistic goal; aim for appropriate assurance.
• Assumptions are inherent in all proofs; focus on raising assurance.
• Formal methods can provide system-wide guarantees.

Method

Translate natural language policies into formal logic, iteratively refine specifications with domain experts, and use automated tools to prove or disprove properties, leveraging AI for proof discovery and code generation.

In practice

• Use formal methods for critical infrastructure and sensitive data.
• Employ AI to assist in formalizing natural language policies.
• Prioritize memory-safe and input-validated code generation.

Topics

• Formal Methods
• AI Cybersecurity
• Automated Reasoning
• AI Agent Safety
• Software Security

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Engineer, AI Security Engineer, Research Scientist

Related on AIssential

• How Automated Reasoning checks in Amazon Bedrock transform generative AI compliance — Automated Reasoning checks provide mathematically provable AI output verification, moving beyond probabilistic validation for regulated industries.
• Autonomous AI needs safeguards beyond model-level guardrails, study finds — Model-level safeguards are insufficient for autonomous AI agents, necessitating formal verification for real-world deployment.
• How academic collaboration delivers real-world security to Amazon customers — Academic-industrial collaboration can yield powerful open-source tools that scale to critical real-world security challenges.
• Learning-Infused Formal Reasoning: From Contract Synthesis to Artifact Reuse and Formal Semantics — Learning-Infused Formal Reasoning (LIFR) integrates AI with formal methods for scalable, trustworthy verification through synthesis and artifact reuse.
• A Subjective Logic-based method for runtime confidence updates in safety arguments — A Subjective Logic method dynamically updates safety case confidence using runtime Safety Performance Indicators.
• Cryptographic certificates of validity for trustworthy AI — Cryptographic certificates enable verifiable AI agent compliance with formal policies via succinct proofs, without trust or re-execution.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Cognitive Revolution.