How Claude Mythos found a 15-year-old bug in Mozilla Firefox | Brian Grinstead

2026-06-17 · Source: Lenny's Newsletter · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Intermediate, extended

Summary

Mozilla Firefox recently leveraged AI agents, including Anthropic's unreleased Mythos and Claude Code, to identify and resolve almost 500 security bugs within a single month. This significant achievement, which involved 100 engineers, was driven by a custom-built "harness" that orchestrates LLM interactions with Firefox's vast codebase, comprising tens of millions of lines of code. The system employs an LLM judge for prioritizing files, an analyzer agent to hypothesize vulnerabilities and generate HTML test cases, and a verifier sub-agent to eliminate false positives. A patching agent then proposes fixes. This approach successfully uncovered long-standing issues, such as a 15-year-old XSLT bug, by automating tedious "archaeology" tasks that human engineers find cognitively exhausting.

Key takeaway

For MLOps Engineers or Security Engineers tasked with scaling vulnerability detection in large codebases, consider developing custom AI agent harnesses. Your team can significantly accelerate bug fixes by integrating LLM-powered prioritization, analyzer agents for test case generation, and verifier sub-agents to ensure high-quality, actionable reports. This approach, proven to find hundreds of bugs including a 15-year-old one, reduces the cognitive load on human engineers and moves closer to a "zero bugs" goal.

Key insights

AI agents, when integrated with custom harnesses and verification loops, can relentlessly find and fix deep-seated software vulnerabilities.

Principles

• Constrain agent problems for exhaustive attempts.
• Guardrails prevent agents from "cheating" or misinterpreting goals.
• Existing DevEx tools accelerate agent integration.

Method

A custom harness workflow involves LLM-based file prioritization, an analyzer agent generating HTML test cases, fuzzing for crash detection, a verifier sub-agent for false positive reduction, and a patching agent for fix generation.

In practice

• Build custom harnesses for large codebases.
• Use LLM judges to prioritize code for agent analysis.
• Implement verifier agents to reduce false positives.

Topics

• AI Agents
• Security Vulnerabilities
• Custom Harnesses
• Firefox Development
• LLM Prioritization
• Software Archaeology

Best for: AI Architect, CTO, VP of Engineering/Data, AI Engineer, MLOps Engineer, Software Engineer

Related on AIssential

• How Mozilla Uses Claude Mythos to find Firefox bugs before hackers do — Custom agentic harnesses, integrated into existing pipelines, are key to scalable, high-quality bug discovery and remediation, surpassing raw LLM power.
• New Firefox update patches a whopping 271 bugs with help from Claude Mythos — AI models like Claude Mythos Preview can effectively identify a high volume of software vulnerabilities.
• Partnering with Mozilla to improve Firefox’s security — AI models like Claude Opus 4.6 can autonomously find and help fix high-severity software vulnerabilities at accelerated speeds.
• Mozilla says 271 vulnerabilities found by Mythos have "almost no false positives" — An AI agent harness significantly reduces false positives in vulnerability detection by integrating LLMs into existing developer toolchains.
• 🎙️ How I AI: How to write AI agent loops in Claude Code and Codex + How Claude Mythos found a 15-year-old bug in Mozilla Firefox — AI agent loops, especially goal-based and subagent architectures, automate complex tasks and enhance bug detection, requiring precise definition and validation.
• Given Enough Agents, All Bugs Become Shallow — AI agents are rapidly advancing offensive security capabilities, enabling faster vulnerability discovery and exploit generation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Lenny's Newsletter.