Archaeology at SemEval-2026 Task 13: Fine-Tuning Pre-Trained Code Models for AI-Generated Code Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, quick

Summary

Team Archaeology's system for SemEval-2026 Task 13 addresses AI-generated code detection, participating in two subtasks. Subtask-A involves binary classification of human-written versus AI-generated code, while Subtask-B focuses on 11-class attribution of the generating model. Moving beyond a TF-IDF and Logistic Regression baseline, the team fine-tuned four pre-trained code models: CodeBERT, GraphCodeBERT, UniXcoder, and CodeT5+. For Subtask-A, strategies included leave-one-language-out cross-validation, code augmentation, chunked inference with trimmed-mean aggregation, and threshold calibration. Subtask-B employed sandwich token packing, class-balanced loss, and multi-seed ensembling with test-time augmentation. The system achieved macro-F1 scores of 0.737 on Subtask-A and 0.422 on Subtask-B.

Key takeaway

For Machine Learning Engineers developing AI-generated code detection systems, you should consider fine-tuning established pre-trained code models like CodeBERT or CodeT5+. Your approach should involve task-specific strategies, such as leave-one-language-out cross-validation and chunked inference for binary detection, and class-balanced loss with multi-seed ensembling for multi-class attribution. This tailored methodology can significantly enhance your system's macro-F1 scores, as demonstrated by achieving 0.737 and 0.422 on distinct subtasks.

Key insights

Fine-tuning pre-trained code models with tailored strategies significantly improves AI-generated code detection and attribution performance.

Principles

• Tailor fine-tuning strategies per subtask.
• Augmentation and ensembling boost robustness.
• Cross-validation improves generalization across languages.

Method

Fine-tune CodeBERT, GraphCodeBERT, UniXcoder, CodeT5+ for binary classification (Subtask-A) and 11-class attribution (Subtask-B) using distinct augmentation, inference, and loss strategies.

In practice

• Apply leave-one-language-out CV for code tasks.
• Use chunked inference with trimmed-mean aggregation.
• Implement class-balanced loss for imbalanced attribution.

Topics

• AI-generated Code Detection
• SemEval-2026 Task 13
• Pre-trained Code Models
• CodeBERT
• Fine-tuning
• Multi-class Attribution

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• Use model-to-model comparison to improve code quality — Comparing AI-generated code with a dedicated AI reviewer improves code quality and architectural consistency.
• Training Agents: Live tutorial on how to fine-tune a coding agent for continual learning — The tutorial outlines a supervised fine-tuning (SFT) baseline for coding agents using public traces and TRL + LoRA.
• Finetuning and Bulk Labelling Images with Prodigy — Fine-tuning pre-trained models with minimal annotations significantly improves image embeddings for more effective bulk labeling.
• Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation — Reward-free alignment significantly improves code LLM non-functional properties, with pretrained models showing higher relative gains and finetuned models offering better absolute performance.
• How I Fine-Tuned DistilBERT to Classify Complaints — And What I Learned Along the Way — Deeply understanding transformer fine-tuning components and their interconnections is crucial for effective model development.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.