GenTI: Benchmarking LLMs for Autonomous IDPS Rule Generation for Unseen Attacks

2026-01-13 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Cybersecurity & Data Privacy, Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

GenTI is an LLM-driven benchmark for autonomously generating Intrusion Detection and Prevention System (IDPS) rules, specifically targeting unseen attacks. The Generative Threat Intelligence (GTI) dataset comprises over 150,000 IDPS rules from Snort, Suricata, and Emerging Threats. It also includes 50,000 YARA rules, all enriched with Cyber Threat Intelligence (CTI) mappings and actionable response types. The GenTI framework employs structured prompt engineering, Chain-of-Thought (CoT) reasoning, and a Chain-of-Verification (CoVe) loop. This transforms analyst prompts and payloads into deployable rules. This system achieved an 89.4% composite rule-quality score and 94.8% CTI coverage. It improved unseen attack detection from 45% to 87.4% and reduced false-positive rates from 8.5% to 2.3%.

Key takeaway

For AI Security Engineers developing adaptive threat detection systems, GenTI demonstrates a robust methodology for automating IDPS rule generation. You should consider integrating LLM-driven pipelines with CTI-enrichment and verification loops to enhance your system's ability to counter unseen attacks. Leverage techniques like CoT and CoVe to improve rule quality and reduce false positives, boosting detection rates and operational efficiency.

Key insights

LLMs can autonomously generate CTI-aware IDPS rules for unseen attacks, significantly improving detection and reducing false positives.

Principles

• CTI-enrichment is crucial for adaptive rule generation.
• Iterative refinement and verification enhance rule quality.
• Curriculum learning stabilizes LLM training for complex security tasks.

Method

The GenTI pipeline uses structured prompts, Chain-of-Thought (CoT) for reasoning, and Chain-of-Verification (CoVe) for syntactic, semantic, and security validation of generated rules.

In practice

• Generate Snort/Suricata/YARA rules for zero-day threats.
• Improve detection rates for novel attack patterns.
• Reduce false positives in IDPS.

Topics

• LLM-driven Security
• IDPS Rule Generation
• Cyber Threat Intelligence
• Zero-Day Detection
• Snort/Suricata
• YARA Rules
• Chain-of-Verification

Best for: Research Scientist, CTO, AI Engineer, AI Scientist, AI Security Engineer, Machine Learning Engineer

Related on AIssential

• GenTI: Benchmarking LLMs for Autonomous IDPS Rule Generation for Unseen Attacks — GenTI leverages LLMs and a comprehensive dataset to autonomously generate adaptive IDPS rules for unseen threats.
• CTIConnect: A Benchmark for Retrieval-Augmented LLMs over Heterogeneous Cyber Threat Intelligence — LLMs require domain-specific knowledge augmentation and tailored retrieval strategies for effective cyber threat intelligence analysis.
• Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations — CTF challenge families and Evolve-CTF enable robust evaluation of agentic LLMs against code transformations.
• Spoon-feeding a Monster — Autonomous security testing requires separating LLM reasoning from deterministic execution to ensure ground truth and prevent hallucinations.
• Decoupled Smart Contract Audits: Lightweight LLM Framework via Distillation and Aggregation — Lightweight LLMs can achieve high accuracy in smart contract auditing by decoupling tasks and employing distillation and aggregation strategies.
• Continuous Offensive Security: The Line We've Been Walking — AI-driven offensive security is crucial for continuous defense against autonomous attackers and new AI-specific attack surfaces.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.