From Human-Centric to Agentic Code Review: The Impact of Different Generations of Generative AI Technology on Review Quality

2026-04-09 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

This study analyzed 1.02 million reviewed pull requests from 207 GitHub projects between May 2022 and February 2026, examining the evolution of code review across human-centric, LLM-assisted, and agentic eras. Researchers identified three AI reviewer adoption practices: Gradual AI Adoption (46% of projects), Rapid LLM Adoption (22%), and Rapid AI Agent Adoption (32%). Findings indicate that Gradual AI Adoption and Rapid AI Agent Adoption significantly improved review efficiency in the agent era, reducing review time by 2.5 days/KLOC and 4.5 days/KLOC respectively, but without improving review quality. Conversely, Rapid LLM Adoption led to a significant increase in code review smells, such as "Review Buddies," by 8.0% points in the LLM era and 4.4% points in the agent era. While agent-initiated and multi-agent reviews were faster under Gradual AI and Rapid AI Agent Adoption, they also showed higher review smell prevalence than human-only reviews. Human-AI collaboration patterns emerged as a dominant factor explaining review efficiency once LLM and AI agents were involved.

Key takeaway

For software engineering managers designing AI-supported code review processes, you should prioritize gradual AI adoption or rapid AI agent integration to achieve efficiency gains. Avoid rapid, heavy LLM-only adoption, as it significantly increases review quality risks, such as "Review Buddies." Integrate AI agents for initial inspection, but ensure human oversight and diverse reviewer participation to maintain quality standards. Balance AI collaboration with established review practices.

Key insights

AI-supported code review boosts efficiency but often increases quality risks like "Review Buddies."

Principles

• Gradual AI adoption improves review efficiency.
• Rapid LLM adoption increases review quality risks.
• Agent-initiated reviews accelerate decisions.

Method

The study clustered AI adoption time series using soft-DTW, identified collaboration patterns via Markov chains with expectation-maximization, and modeled review quality using logistic regression with traditional and human-AI factors.

In practice

• Adopt AI reviewers selectively by pull request context.
• Balance AI collaboration with established review practices.
• Design benchmarks reflecting pull request context.

Topics

• Code Review
• Generative AI
• AI Agents
• LLM Adoption
• Software Quality
• GitHub Data

Code references

• Azure/azure-cli
• flutter/packages
• marketplace/actions
• search/advanced
• marketplace/llamapreview

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Scientist, Research Scientist, Software Engineer

Related on AIssential

• 3100 Opinions on Code Review in an AI World: Building Causal Theory from Practitioner Discourse — Code review is the control point for AI's impact on software quality, shaped by human expertise and process structure.
• Slow down to speed up: so much has changed in 6 months’ time — Rapid AI agent adoption boosts individual dev output but risks software quality and organizational integrity without human oversight.
• Stop Asking One LLM to Review Your Code. Here’s the 7-Agent Pipeline That Actually Works — Multi-agent orchestration with aggressive filtering significantly reduces false positives in AI code review.
• Benchmarking Agentic Review Systems — Agentic AI review systems can track human quality judgments and detect errors, showing promise despite improvement needs.
• The End of Code Review: Coding Agents Supersede Human Inspection — Coding agents now supersede human code review by fulfilling its goals more efficiently and scalably.
• Same Scrutiny, More Time: Eye Tracking Insights into Reviewing LLM-Labelled Code — Explicitly labeling code as LLM-generated increases review time without improving thoroughness, driven by perceived origin.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.