Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproduction

2026-07-02 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

Mastermind is a novel dual-loop framework designed to enhance repository-scale vulnerability reproduction by improving the strategic decision-making of large language model (LLM) agents. It addresses the common failure point where agents execute steps correctly but choose an ineffective overall strategy. Mastermind posits that strategy, being compact, concrete, and stable, is the optimal learning unit for such agents. The framework features a trainable planner that learns reusable vulnerability-reproduction strategies through SFT and milestone-based GRPO, alongside an experience loop for task-local strategy records. This independent planner improves multiple frozen executors without modifying their action-generation capabilities. Evaluated on CyberGym with 260 training and 200 evaluation tasks, Mastermind achieved an 84.5% pass rate using GPT-5.5, outperforming open-book PoC context (60.0%), Best-of-8 sampling (63.0%), and iterative improvement (77.0%). The planner also boosted GPT-5.4 mini from 45.0% to 60.0% and GLM~5.1 from 58.5% to 71.0%, demonstrating effective and transferable strategy learning for software engineering agents.

Key takeaway

For Machine Learning Engineers developing LLM agents for complex software engineering tasks like vulnerability reproduction, you should prioritize designing systems that learn and apply high-level strategies. Focusing on strategy optimization, rather than just refining action generation, can significantly boost agent performance and transferability across different base models. Consider implementing a separate, trainable planner using techniques like SFT and GRPO to guide your frozen executors, as demonstrated by Mastermind's 84.5% pass rate with GPT-5.5. This approach offers a robust path to more capable and adaptable agents.

Key insights

Learning high-level strategies, not just actions, significantly improves LLM agents for complex software engineering tasks.

Principles

• Strategy is the optimal learning unit for SE agents.
• Decouple strategy learning from executor capabilities.
• High-level strategies are transferable across LLM executors.

Method

Mastermind employs a dual-loop framework: a trainable planner learns strategies via SFT and milestone-based GRPO, while an experience loop maintains task-local strategy records.

In practice

• Implement a dedicated planner for LLM agent guidance.
• Optimize high-level strategies for complex task performance.
• Apply SFT and GRPO for agent strategy acquisition.

Topics

• LLM Agents
• Vulnerability Reproduction
• Strategy Learning
• Software Engineering Agents
• Reinforcement Learning
• CyberGym Benchmark

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• Beyond Trajectory Imitation: Strategy-Guided Policy Optimization for LLM Reasoning — Strategy-Guided Policy Optimization distills reasoning by transferring reusable problem-solving strategies, not just specific solution steps.
• ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End? — Current LLM agents struggle significantly with end-to-end operations research tasks, failing to achieve reliable, high-quality solutions.
• Self Evolving AI Skills w/ GPT-5.5 (SkillOpt) — SkillOpt enables LLMs to self-evolve skills by optimizing natural language policies through a frozen target model and an external optimizer.
• Bilevel Optimization of Agent Skills via Monte Carlo Tree Search — Bilevel optimization with MCTS and LLMs effectively refines LLM agent skills by separating structure search from content refinement.
• AIエージェントが最適化プログラミングコンテストで初優勝 — AI agents can now win complex, real-time optimization contests by scaling LLM inference and self-learning.
• SkillMAS: Skill Co-Evolution with LLM-based Multi-Agent System — SkillMAS couples skill evolution and MAS restructuring using verified execution traces for adaptive LLM agent specialization.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.