PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization

2026-07-22 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, quick

Summary

PerfAgent is a novel profiler-guided, verifier-in-the-loop workflow. It enhances large language model (LLM) agents' capability for repository-level code optimization. While LLM agents perform well on correctness tasks like SWE-Bench, they struggle with improving runtime performance while preserving code behavior. PerfAgent provides an off-the-shelf coding agent with feedback. This feedback helps identify real hotspots, move beyond initial shallow speedups, and use profiler evidence for optimization decisions. This prevents agents from missing bottlenecks or silently breaking edge cases. On the GSO benchmark, PerfAgent more than doubles expert-matching patches, improving from 19.6% to 39.2% with OpenHands and GPT-5.1. On SWE-fficiency-Lite, it boosts performance from 26% to 74%, surpassing an oracle best-of-five baseline at lower cost.

Key takeaway

For AI Engineers developing LLM agents for code optimization, you should integrate profiler-guided, verifier-in-the-loop workflows. This approach significantly improves optimization success rates and patch quality. It helps your agents move beyond shallow speedups and avoid silently breaking code. Implement iterative refinement steps, using profiler data to guide subsequent optimization efforts. This strategy can more than double expert-matching patches, as demonstrated by PerfAgent's results.

Key insights

PerfAgent uses profiler-guided, verifier-in-the-loop feedback to significantly improve LLM agent code optimization performance.

Principles

• Profiler feedback identifies true bottlenecks.
• Iterative refinement surpasses initial patches.
• Verifier-in-the-loop ensures behavior preservation.

Method

PerfAgent employs a profiler-guided, verifier-in-the-loop workflow. It provides an off-the-shelf coding agent with feedback to find hotspots, improve beyond initial patches, and use profiler evidence for optimization.

In practice

• Integrate profilers into LLM agent workflows.
• Implement iterative patch refinement loops.
• Use verifiers to prevent silent code breaks.

Topics

• LLM Agents
• Code Optimization
• Profiler-Guided Optimization
• Code Verification
• Software Engineering
• Performance Benchmarking

Best for: Machine Learning Engineer, Research Scientist, AI Scientist, AI Engineer

Related on AIssential

• PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization — Profiler-guided, iterative feedback enables LLM agents to achieve expert-level repository-wide code optimization.
• Probe-and-Refine Tuning of Repository Guidance for Coding Agents — Iteratively refined, failure-informed guidance significantly improves LLM coding agent performance by boosting patch coverage.
• LLM-as-a-Verifier: A General-Purpose Verification Framework — LLM-as-a-Verifier uses scoring token logits to provide fine-grained, scalable verification for agentic tasks without retraining.
• PyAgent: A Design Pattern Orchestrator for Multi-Agent LLM Systems — PyAgent standardizes multi-agent LLM system development through a comprehensive, pattern-driven, and observable infrastructure-as-code stack.
• DIY #19 - Evaluator-Optimiser LLM Agent with LangChain — The Evaluator-Optimiser pattern uses iterative feedback between two LLMs to achieve higher quality outputs for complex tasks.
• LLM Consortium for Software Design Refinement: A Controlled Experiment on Multi-Agent Collaboration Topologies — Structural adversarial and cross-model review topologies significantly enhance multi-agent LLM software design.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.