AI Finds Vulnerabilities. Security Experts Find Impact.

2026-06-24 · Source: Bishop Fox - Bishopfox.com · Field: Technology & Digital — Cybersecurity & Data Privacy, Artificial Intelligence & Machine Learning · Depth: Intermediate, medium

Summary

A security assessment for a web application, primarily using Claude AI for source-code review, demonstrates AI's significant role in accelerating initial security tasks. AI effectively processed information, traced requests, identified entry points, and translated code, reducing tasks that typically take hours or days to minutes. This accounts for roughly 80% of the work. However, the assessment by Bishop Fox revealed that human expertise remains critical for the final 20%, involving validating findings, determining actual impact, chaining vulnerabilities, and discerning real issues from false positives. Specific examples include an AI-identified phone verification bypass that a human expert escalated to an unlimited account creation vulnerability, and an AI-flagged SSRF candidate that required human analysis to differentiate between a Google Cloud Vision-handled request and a true internal server-side request with data extraction capabilities. The analysis also highlighted AI's tendency to provide confident but incorrect explanations, underscoring the need for human verification.

Key takeaway

For security consultants integrating AI into web application assessments, recognize that while AI significantly speeds up initial vulnerability pattern identification, your judgment is paramount for determining true impact. You must rigorously validate AI's findings, especially when chaining issues or extracting data, as AI can confidently present incorrect explanations. Prioritize human-led impact assessment and verification to deliver accurate risk reports, preserving customer trust and avoiding unproductive rabbit holes.

Key insights

AI accelerates initial vulnerability pattern identification but human expertise is indispensable for impact assessment and validation.

Principles

• AI excels at pattern recognition, not impact assessment.
• Verify all AI-generated system and code behavior claims.
• Customer trust hinges on accurate risk assessment, not theoretical possibilities.

Method

Integrate AI for initial code review, tracing requests, and identifying data flows. Follow with human validation, impact assessment, and chaining of findings.

In practice

• Use AI for initial source code review.
• Manually validate AI findings with live application testing.
• Fuzz request fields to uncover hidden data extraction paths.

Topics

• AI in Security Testing
• Vulnerability Assessment
• Web Application Security
• Source Code Review
• Server-Side Request Forgery
• AI Limitations

Best for: AI Security Engineer, Security Engineer, Consultant

Related on AIssential

• The Trust Gap Behind the AI Coding Boom: What 200 Security Practitioners Just Told Us — AI-accelerated development outpaces security's manual validation, amplifying complex vulnerabilities.
• Human Verification Tools Help Make Smarter Data-Driven Decisions — Authenticating human interaction is critical for reliable data-driven decisions amidst pervasive AI and bot activity.
• How AI Supports Modern Penetration Testing — AI tools enhance penetration testing by automating tasks, improving continuous monitoring, and accelerating vulnerability detection.
• Claude Helped a Hacker Find a Way to Issue Tickets to Almost Every US Music Festival — AI models can autonomously discover and exploit complex web vulnerabilities, bypassing security controls.
• Mythos Doesn't Deploy Itself — Human expertise and robust orchestration are critical for effective AI-driven vulnerability discovery, mitigating "AI slop."
• AI companies should publish security assessments — AI companies should publicly disclose third-party security assessment findings against defined threat models to improve collective security.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Bishop Fox - Bishopfox.com.