Evaluating Open-Weight LLMs for Generating Structured Threat Information for Autonomous Vehicle Vulnerabilities

2026-07-17 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

An evaluation of open-weight Large Language Models (LLMs) demonstrates their capability in generating Structured Threat Information Expression (STIX) for Connected and Autonomous Vehicle (CAV) vulnerabilities. Researchers constructed the CAV-STIXGen dataset, mapping CAV vulnerability descriptions to STIX domain objects (SDO), STIX relationship objects (SRO), Common Weakness Enumeration (CWE), and MITRE ATT&CK techniques. Eleven LLMs, ranging from 4B to 120B parameters, were tested using various prompting strategies. Single-model configurations achieved F1 scores of 0.94 for SDO, 0.63 for SRO, and 0.99 for CWE mapping, though complete MITRE ATT&CK mapping proved challenging. In a multi-agent setup, Gemma-4-31B and Codestral-22B achieved F1 scores of 0.91 for SDOs and 0.43 for SROs, respectively, highlighting potential for automating threat intelligence and prioritizing transportation security defenses.

Key takeaway

For AI Security Engineers focused on Connected and Autonomous Vehicle (CAV) security, you should consider integrating open-weight LLMs to automate the generation of Structured Threat Information Expression (STIX) from CVEs. While single models achieve high F1 scores for SDO (0.94) and CWE (0.99) mapping, be aware that complete MITRE ATT&CK mapping remains challenging. Explore multi-agent LLM setups like Gemma-4-31B and Codestral-22B for improved SDO and SRO generation, streamlining threat intelligence and defense prioritization.

Key insights

Open-weight LLMs can automate structured threat intelligence generation for CAV vulnerabilities, though MITRE ATT&CK mapping remains difficult.

Principles

• Structured threat information is crucial for CAV risk mitigation.
• Open-weight LLMs can translate plain text CVEs to STIX.
• Multi-agent LLM setups can enhance STIX generation accuracy.

Method

Construct CAV-STIXGen dataset mapping CVEs to STIX SDO, SRO, CWE, and MITRE ATT&CK. Evaluate 11 LLMs with various prompts and temperatures. Analyze CWE/ATT&CK co-occurrences.

In practice

• Utilize LLMs for SDO and CWE mapping from CVEs.
• Explore multi-agent LLM configurations for improved SROs.
• Analyze CWE/ATT&CK co-occurrence patterns for defense.

Topics

• Connected and Autonomous Vehicles
• Large Language Models
• STIX
• CVE Database
• MITRE ATT&CK
• Threat Intelligence Automation
• Cybersecurity

Best for: AI Scientist, AI Security Engineer, Research Scientist

Related on AIssential

• Week Ending 7.19.2026 — LLMs can structure CAV vulnerability data, but struggle with full attack chains.
• Data-Centric Benchmarking of Exploit Generation in LLMs: Understanding the Impact of Fine-Tuning — Data quality and structured evaluation are paramount for effective LLM-based exploit generation.
• Application of LLMs to Threat Assessment of Foreign Peacekeeping Missions — LLMs can effectively automate and enhance threat assessment for peacekeeping missions by processing OSINT.
• Building an Agentic Security Pipeline That Finds, Proves, and Patches Vulnerabilities — LLM-powered agentic pipelines can automate finding, proving, and patching vulnerabilities in complex codebases.
• Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering — LLM agents significantly reduce SAST false positives, but effectiveness is highly dependent on the backbone model and vulnerability category.
• Widening the Gap: Exploiting LLM Quantization via Outlier Injection — Outlier injection during quantization can predictably induce malicious behavior in LLMs, even with advanced schemes.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.