Making secret scanning more trustworthy: Reducing false positives at scale

2026-06-11 · Source: The GitHub Blog · Field: Technology & Digital — Cybersecurity & Data Privacy, Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, short

Summary

GitHub, in collaboration with Microsoft Security & AI's Agents Offense team, has significantly enhanced its secret scanning capabilities to reduce false positives and improve developer trust. The improvement integrates an LLM-based contextual verification approach, derived from Agentic Secret Finder, into GitHub's existing detection pipeline. This system, which already combines pattern-based and AI-powered detection, now uses "better context" by extracting high-signal usage information—like a value being passed into an API request or authentication header—rather than analyzing entire files. This focused contextual reasoning helps distinguish real secrets from benign strings. The approach achieved a 75.76% reduction in false positives, surpassing its 65% target, based on hundreds of customer-confirmed alerts, leading to clearer signals and faster action on real risks.

Key takeaway

For DevSecOps Leads evaluating secret scanning solutions, GitHub's 75.76% false positive reduction highlights the power of LLM-based contextual verification. You should prioritize systems that extract focused usage signals rather than relying on broad code analysis. This approach significantly improves alert precision, reduces developer fatigue, and accelerates the remediation of real security risks. Consider integrating similar context-aware reasoning into your security pipelines.

Key insights

LLM-based contextual verification significantly reduces false positives in secret scanning by focusing on usage signals.

Principles

• Alert noise erodes trust and slows remediation.
• Focused usage context beats broad data for verification.
• High precision is achievable with file-level context.

Method

An LLM-based verification step analyzes extracted high-signal usage context, such as a value's assignment to an API call, to confirm if a detected pattern is an actual secret.

In practice

• Implement contextual reasoning in security alert verification.
• Design context extraction for usage signals, not raw data.
• Evaluate false positives using file-level usage context.

Topics

• Secret Scanning
• False Positive Reduction
• LLM Verification
• Contextual Reasoning
• Code Security
• Developer Trust

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Security Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• The Logic Auditor: Why Your LLM Needs a “Constructive Lie” to Achieve 99% Accuracy — Adversarial reasoning, using "Constructive Lies," forces LLMs into verification mode, significantly improving accuracy and grounding.
• Analyzing Chain of Thought (CoT) Approaches in Control Flow Code Deobfuscation Tasks — CoT prompting significantly enhances LLM performance in code deobfuscation by guiding step-by-step reasoning.
• LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs — LLMs can be forced to leak training data, but rarely do so under ordinary, non-adversarial use.
• Beyond Problem Solving: UOJ-Bench for Evaluating Code Generation, Hacking, and Repair in Competitive Programming — LLMs can complement traditional online judges by identifying subtle code errors, but effective error detection remains computationally intensive.
• Why Adding More Context Makes LLMs Less Reliable — Excessive or unstructured context degrades LLM reliability by creating competing signals that models cannot effectively prioritize.
• Flow-Based Token Credit for Reasoning RL — Methods to enhance LLM reasoning and efficiency involve targeted attention, dynamic memory, and knowledge distillation trade-offs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The GitHub Blog.