Multi-Class vs. Multi-Label BERT for CVE-to-CWE Mapping: How Taxonomy Structure Shapes the Errors

2026-07-08 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

A study investigates the automated assignment of Common Weakness Enumeration (CWE) categories to Common Vulnerabilities and Exposures (CVE) records, comparing multi-class and multi-label text classification approaches. Researchers evaluated BERT Base, SecureBERT, and CySecBERT transformer encoders across three nested CWE label spaces (83, 47, and 25 classes). Multi-class training consistently achieved higher macro-F1 scores, though the performance gap with multi-label models decreased from 21 to 2 percentage points as the label space was reduced. Post-hoc threshold optimization further closed this gap for multi-label models in the 25-class setting. Analysis revealed that misclassification patterns strongly align with the CWE hierarchy, suggesting taxonomy design is a primary driver of error structure, rather than the specific encoder used (Pearson r > 0.92). A hierarchy-relaxed evaluation, which tolerates within-family confusions, boosted macro-F1 from approximately 81% to 90%, indicating better branch-level quality than strict metrics suggest. CySecBERT demonstrated the strongest overall performance, particularly in the multi-label configuration.

Key takeaway

For vulnerability analysts or ML engineers developing automated CVE-to-CWE mapping systems, you should prioritize understanding the CWE taxonomy's impact on model errors. While multi-class BERT models show initial macro-F1 advantages, consider multi-label approaches with post-hoc threshold optimization, especially for smaller label spaces, as CySecBERT demonstrated strong gains here. Evaluate your models using hierarchy-relaxed metrics to accurately assess branch-level classification quality, preventing strict metrics from understating performance.

Key insights

CWE taxonomy structure significantly influences CVE-to-CWE mapping errors more than the choice of BERT encoder.

Principles

• Multi-class models generally outperform multi-label for CVE-to-CWE.
• Taxonomy hierarchy dictates misclassification patterns.
• Relaxed evaluation reveals higher branch-level classifier quality.

Method

Compares multi-class vs. multi-label BERT (Base, SecureBERT, CySecBERT) on CVE-to-CWE mapping across 3 nested CWE label spaces, using macro-F1 and confusion analysis.

In practice

• Consider multi-class for initial CVE-to-CWE mapping.
• Apply post-hoc thresholding for multi-label models.
• Use hierarchy-aware metrics for vulnerability classification.

Topics

• CVE-to-CWE Mapping
• Text Classification
• BERT Models
• Vulnerability Analysis
• Taxonomy Structure
• Machine Learning Security

Best for: NLP Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• Indigenous Letters to Brazil: Multi-Label Classification — Multi-label classification of indigenous letters presents challenges due to corpus imbalance and semantic overlap.
• ZYC at SemEval-2026 Task 5: Application of BERT-based Contextual Embeddings Similarity for WSD — Finetuning and training curriculum are more impactful for WSD than the choice of BERT-like pretrained model.
• Measuring the sensitivity of LLM-based structured extraction to prompt, model, and schema choices in clinical discharge summaries — LLM-based clinical data extraction sensitivity varies significantly with model and schema, especially regarding "absence-versus-silence" distinctions.
• The Hardest Part of Fine-Tuning Isn’t the Training — The most challenging aspect of NLP fine-tuning is effective label design and meticulous data annotation, not the model training itself.
• Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection — A mechanism-oriented taxonomy of indirect linguistic encoding significantly enhances LLM-based coded language detection.
• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.