SYSUpporter at SemEval-2026 Task 13: Leveraging Stylistic Signals and Language-Aware Truncation for Machine-Generated Code Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

The SYSUpporter system, developed for SemEval-2026 Task 13 Subtask B, addresses the challenge of attributing source code to either a human author or one of 10 specific LLM families. This system tackles practical issues like formatting fingerprints often discarded by tokenizers, heterogeneous code lengths, and extreme class imbalance within datasets. Building upon unixcoder-base, SYSUpporter integrates Explicit Stylistic Prompting, Language-Aware Truncation, and imbalance-aware training techniques such as Focal Loss, GeM pooling, multi-sample dropout, and bucket batching. The system achieved a 0.434 Macro F1 score on the official hidden test set, ranking 4th out of 34 teams with only 125M parameters. Controlled 5-fold cross-validation confirmed each component's contribution, and a formatting-normalization study quantified the model's reliance on formatting cues.

Key takeaway

For Machine Learning Engineers developing code attribution models, consider integrating the SYSUpporter system's approach. Its success in SemEval-2026 Task 13 highlights the importance of leveraging stylistic signals and language-aware truncation. You should explore applying imbalance-aware training techniques like Focal Loss to improve performance on datasets with skewed class distributions, especially when distinguishing between multiple LLM families.

Key insights

Stylistic signals and language-aware truncation are key for robust machine-generated code detection and LLM attribution.

Principles

• Formatting fingerprints are critical for code authorship.
• Heterogeneous code lengths require specific truncation strategies.
• Extreme class imbalance necessitates imbalance-aware training.

Method

The system enhances unixcoder-base with Explicit Stylistic Prompting, Language-Aware Truncation, and imbalance-aware training using Focal Loss, GeM pooling, multi-sample dropout, and bucket batching.

In practice

• Employ Explicit Stylistic Prompting for code attribution.
• Implement Language-Aware Truncation for varied code inputs.
• Apply Focal Loss to address dataset class imbalance.

Topics

• Machine-Generated Code Detection
• LLM Attribution
• Stylistic Analysis
• SemEval-2026
• Focal Loss
• Code Authorship

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• Detecting Functional Memorization in Code Language Models — Code LLMs can functionally memorize logic even when textually dissimilar, requiring advanced auditing.
• The Significance of Style Diversity in Annotation-Free Synthetic Data Generation — Annotation-free synthetic data generation for intent classification significantly improves utility by prioritizing style diversity during generation over post-hoc adaptation.
• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• CodeChat-Eval: Evaluating Large Language Models in Multi-Turn Code Refinement Dialogues — LLMs significantly degrade in functional correctness during multi-turn code refinement, especially with semantic or additive changes.
• Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross-Script Transfer Learning — Adapting TrOCR for Tigrinya text requires tokenizer extension and Word-Aware Loss Weighting to overcome script-specific challenges.
• Authorship Attribution in Multilingual Machine-Generated Texts — Multilingual authorship attribution for LLM-generated text is complex, with current monolingual methods showing limited cross-lingual transferability across diverse language families.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.