How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests

2026-07-23 · Source: Machine Learning · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Advanced, quick

Summary

An empirical study published on 2026-07-23 investigates the contributions of AI coding agents to software development by analyzing agentic pull requests (PRs) against human-generated PRs. Utilizing the AIDev dataset, the research characterizes how agentic contributions evolve across the software development lifecycle. Specifically, it examines differences in merge rates between agentic and human PRs over time, identifies the predominant development tasks where AI coding agents are applied, and tracks how these task distributions change across development quarters. Furthermore, the study compares key characteristics of agentic and human PRs, focusing on their implications for software quality and their temporal dynamics. The findings offer an empirical and longitudinal perspective, providing a nuanced understanding of AI coding agents' real-world benefits and limitations.

Key takeaway

For engineering managers evaluating AI coding agent adoption, this study highlights the need to understand their specific impact on pull request merge rates and task distribution. You should analyze your own agentic PR data to identify where AI agents genuinely enhance software quality versus where human oversight remains critical. This empirical perspective helps you make informed decisions about agent integration and resource allocation.

Key insights

The study empirically characterizes AI coding agent contributions to software development via pull request analysis.

Method

The study analyzed agentic vs. human PRs using the AIDev dataset, comparing merge rates, task distributions, and key characteristics over time to assess AI agent impact.

Topics

• AI Coding Agents
• Pull Requests
• Software Quality
• Empirical Research
• AIDev Dataset

Best for: AI Scientist, Research Scientist, Software Engineer

Related on AIssential

• Coding Agents Meet Data Science — AI coding agents boost developer productivity but introduce new bottlenecks and require specialized adaptation for data science workflows.
• Prompt Quality and Pull Request Outcomes: A Stage-Based Empirical Study of LLM-Assisted Development — Prompt quality dimensions (Context, Specificity, Verification) exert distinct, stage-dependent effects on LLM-assisted pull request outcomes.
• Early Adoption of Agentic Coding Tools by GitHub Projects — Agentic coding tools see uneven adoption, concentrated in small projects with single-human oversight, and varied productivity.
• Mining Architectural Quality Under Agentic AI Adoption: A Causal Study of Java Repositories — AI coding tool adoption does not inherently improve software architecture quality; density metrics can be misleading.
• Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests — Instruction files for AI agents do not guarantee improved pull request merge rates or reduced effort.
• My 24/7 Claude Code AI Agent’s Biggest Win...that won’t happen again — AI agents can autonomously identify and contribute code changes to open-source projects via browser navigation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.