Bitzkrieg at SemEval-2026 Task 13: Calibration-Aware Dual CodeBERT for Multilingual Machine-Generated Code Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, medium

Summary

The "Bitzkrieg" system, submitted to SemEval-2026 Task 13, addresses multilingual machine-generated code (MGC) detection across three subtasks. For binary detection (Subtask A), the system fine-tuned two CodeBERT models with complementary sampling and applied percentile-based post-hoc calibration, boosting Macro-F1 from 0.47 to 0.56 without extra training. For generator attribution (Subtask B), it combined TF-IDF n-grams, frozen CodeBERT embeddings, and language features with XGBoost, employing synthetic augmentation and class weighting to manage an 11-class dataset skewed 88% towards human code, achieving a Macro-F1 of 0.289. Subtask C, focusing on four-way authorship classification, utilized a fine-tuned CodeBERT classifier, reaching a Macro-F1 of 0.49. The results emphasize the critical role of probability calibration for binary detection and class balancing for multi-class attribution.

Key takeaway

For Machine Learning Engineers developing code authorship or detection systems, you should prioritize probability calibration for binary classification tasks to improve model reliability. When tackling multi-class attribution with imbalanced datasets, ensure you implement robust class balancing techniques like synthetic augmentation and class weighting. Your approach should also consider combining diverse feature sets, such as CodeBERT embeddings with TF-IDF n-grams, to enhance attribution accuracy, especially for skewed distributions.

Key insights

Probability calibration and class balancing are crucial for robust machine-generated code detection.

Principles

• Calibrate probabilities for binary classification.
• Balance classes in multi-class attribution.
• Combine diverse features for attribution.

Method

Fine-tune CodeBERT models, apply percentile-based post-hoc calibration for binary tasks, and use XGBoost with TF-IDF, CodeBERT embeddings, and language features for multi-class attribution, incorporating synthetic augmentation and class weighting.

In practice

• Implement percentile-based post-hoc calibration.
• Use synthetic augmentation for skewed datasets.
• Combine CodeBERT embeddings with TF-IDF.

Topics

• Machine-Generated Code Detection
• CodeBERT
• Probability Calibration
• Class Imbalance
• Generator Attribution
• SemEval-2026

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Divide et Calibra: Multiclass Local Calibration via Vector Quantization — Vector Quantization enables stable, compositional multiclass local calibration by sharing parameters across sparse latent space regions.
• Code Is More Than Text: Uncertainty Estimation for Code Generation — Code generation uncertainty estimation requires specialized methods accounting for code's unique properties, outperforming natural language-derived baselines.
• High-Precision APT Malware Attribution with Out-of-Scope Resilience — High-precision APT malware attribution is achievable by explicitly abstaining on out-of-scope samples.
• Practical estimation of the optimal classification error with soft labels and calibration — Isotonic calibration enables consistent Bayes error estimation even with corrupted soft labels, provided their order is preserved.
• Mondrian conformal for imbalanced classes — Mondrian Conformal Prediction provides class-specific uncertainty quantification for imbalanced classification.
• Calibrating Generative Models to Feature Distributions with MMD Finetuning — kCGM corrects generative model miscalibration by matching feature distributions with MMD and KL regularization, improving validity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.