WWTC@UniA at SemEval-2026 Task 13: BERT-based Code Authorship Detection and Qualitative Analysis

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, short

Summary

WWTC@UniA's system for SemEval-2026 Task 13 focuses on detecting machine-generated code and identifying the large language model (LLM) family responsible for its creation. The team fine-tuned small encoder-only models, discovering that ModernBERT, a general-purpose model, surprisingly outperformed models specifically pre-trained for code. In the official evaluation, their system achieved 5th place on subtask B and 6th on subtask C. A detailed analysis showed that natural language elements within code snippets, such as comments, are crucial for identifying the generating LLM family. However, the fine-tuned ModernBERT's embeddings did not effectively differentiate between LLM families, though they successfully clustered human-written code by programming language.

Key takeaway

For AI Security Engineers assessing code provenance, you should consider general-purpose BERT models like ModernBERT for detecting machine-generated code and attributing its LLM family. Your analysis should specifically examine natural language elements, such as comments, within code snippets, as these provide strong signals for LLM identification. Do not solely rely on code embeddings for LLM family distinction, as they may primarily cluster by programming language.

Key insights

A general-purpose BERT model can effectively detect machine-generated code and identify LLM families, outperforming code-specific models.

Principles

• General-purpose models can surpass domain-specific ones in certain code tasks.
• Natural language in code (comments) aids LLM family identification.
• Code embeddings may cluster by programming language, not LLM family.

Method

Fine-tuning small encoder-only models, specifically ModernBERT, for binary classification (human vs. machine) and multi-class LLM family identification.

In practice

• Prioritize general-purpose BERT models for code authorship detection.
• Analyze code comments for LLM attribution clues.
• Evaluate embedding clustering for programming language identification.

Topics

• Code Authorship Detection
• Machine-Generated Code
• Large Language Models
• BERT Models
• SemEval-2026 Task 13
• Code Embeddings

Best for: AI Scientist, Research Scientist, AI Security Engineer

Related on AIssential

• Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection — A mechanism-oriented taxonomy of indirect linguistic encoding significantly enhances LLM-based coded language detection.
• I Built a BPE Tokenizer From Scratch in Python And Finally Understand How LLMs Work — Building a BPE tokenizer from scratch demystifies how LLMs process text, revealing its underlying simplicity.
• Use of Machine Learning techniques and Large Language Models for automatic evaluation of Celpe-Bras exam texts — BERT adaptations achieved the best automatic evaluation for Celpe-Bras texts, but at a higher computational cost.
• Assessing Capabilities of Large Language Models in Social Media Analytics: A Multi-task Quest — Modern LLMs were systematically evaluated across key social media analytics tasks using a Twitter (X) dataset.
• Sem-Detect: Semantic Level Detection of AI Generated Peer-Reviews — AI-generated peer reviews converge semantically, while human reviews, even LLM-refined, retain unique ideas.
• Authorship Attribution in Multilingual Machine-Generated Texts — Multilingual authorship attribution for LLM-generated text is complex, with current monolingual methods showing limited cross-lingual transferability across diverse language families.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.