I built an open-source multi-agent SDLC harness that beats a cold Claude Code run on large repos, by learning the repo once. Real benchmarks (incl. where it loses) inside. [P]

· Source: Machine Learning · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning · Depth: Intermediate, quick

Summary

AutoDev Studio is an open-source multi-agent AI coding harness designed to improve efficiency and cost-effectiveness in software development. It significantly outperforms cold "claude -p" runs, demonstrating 7%–75% cost savings on 6 out of 6 well-localized tasks across repositories up to approximately 82,000 lines of code. For instance, a specific bug fix cost ~\$1.70 with AutoDev Studio compared to \$6.83 for a cold agent run. This efficiency stems from building a persistent knowledge base via static analysis and local embedding indexing, allowing localization to become a lookup rather than repeated exploration. The system features a PM agent for clarification, a Dev agent for code, QA for testing, a separate model for diff review, and a bounded revise loop, culminating in a real GitHub PR. It also tracks token usage and cost, supports various providers, and can run free/offline.

Key takeaway

For AI Engineers developing or deploying AI coding agents, consider integrating persistent knowledge bases to significantly cut operational costs and improve performance on large repositories. Your current single-shot agents might be inefficiently re-exploring codebases. By adopting an approach like AutoDev Studio's, which localizes changes via a learned index, you can achieve substantial cost reductions (7%-75%) and streamline development workflows, especially for complex tasks. Evaluate its open-source architecture for potential integration into your SDLC.

Key insights

Persistent knowledge bases for AI coding agents drastically reduce costs and improve efficiency on large codebases.

Principles

Method

AutoDev Studio ingests a repository to build a persistent knowledge base using static analysis and a local embedding index. It then orchestrates PM, Dev, and QA agents, followed by diff review and a revise loop, opening a GitHub PR.

In practice

Topics

Code references

Best for: AI Architect, AI Product Manager, Entrepreneur, AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.