Detecting Vulnerability-Inducing Commits via Multi-Stage Reasoning with LLM-Based Agents

2026-07-08 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Software Development & Engineering · Depth: Expert, extended

Summary

VIC-RAGENT is an LLM-based multi-agent framework designed to detect vulnerability-inducing commits (VICs) at submission time, enhancing software security. It employs specialized agents for structural analysis, intent understanding, and multi-stage vulnerability inspection, including preliminary analysis, reanalysis, and a final decision. The framework integrates code diffs, commit messages, and contextual code, supported by a vulnerability knowledge base for retrieval-augmented reasoning. Evaluated on the V-SZZ dataset with 241 commits (106 VICs, 135 VFCs), VIC-RAGENT consistently outperformed baselines like Direct, CoT, and CodeAgent across DeepSeek-V3.2, Qwen-Plus, and GPT-4o-mini. It achieved 1.2–1.7× higher F1-scores and up to 2× recall improvement over the strongest baseline, demonstrating robust and explainable detection.

Key takeaway

For AI Security Engineers integrating LLMs into CI/CD pipelines, consider adopting a multi-agent, multi-stage framework like VIC-RAGENT. Its structured approach significantly boosts recall and F1-scores in detecting vulnerability-inducing commits, crucial for preventing supply chain attacks. You should prioritize frameworks that offer explainability and leverage knowledge bases for robust, continuously improving security analysis, even if it incurs a moderate increase in computational cost.

Key insights

A multi-agent, multi-stage LLM framework significantly improves vulnerability-inducing commit detection and explainability.

Principles

• Decompose complex tasks into specialized agent roles.
• Employ coarse-to-fine reasoning for reliability.
• Augment LLM analysis with a knowledge base.

Method

VIC-RAGENT uses Code Analyst and Target Analyst for preparation, then a Vulnerability Inspector performs preliminary inspection, reanalysis (with knowledge base retrieval), and a final decision stage. A Document Specialist generates reports.

In practice

• Implement multi-agent systems for code review.
• Use retrieval-augmented generation for security analysis.
• Structure LLM reasoning into sequential verification steps.

Topics

• Vulnerability Detection
• LLM Agents
• Multi-Stage Reasoning
• Software Supply Chain Security
• Commit Analysis
• Retrieval-Augmented Generation

Code references

• KeLeXueBi/VIC-RAGRENT

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• CyberChainBench: Can AI Agents Secure Smart Contracts Against Real-World On-Chain Vulnerabilities? — LLM agents show promise but significant limitations in smart contract security tasks, especially patching.
• VulnAgent-R2: Evidence-Calibrated Multi-Agent Auditing for Repository-Level Vulnerability Detection — Vulnerability detection is improved by a staged, evidence-driven auditing process using multi-agent collaboration and selective verification.
• Code-Augur: Agentic Vulnerability Detection via Specification Inference — Agentic vulnerability detection improves by making LLM assumptions explicit and continuously validating them with fuzzing.
• Building an Agentic Security Pipeline That Finds, Proves, and Patches Vulnerabilities — LLM-powered agentic pipelines can automate finding, proving, and patching vulnerabilities in complex codebases.
• I Read Cursor's Security Agent Prompts, So You Don't Have To — Simple LLM prompts, when supported by robust orchestration, can effectively automate security reviews at scale.
• Using LLMs to Secure Source Code — LLMs streamline vulnerability discovery, shifting the security bottleneck to verification, triage, and patching.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.