UCSC-NLP at SemEval-2026 Task 13: Multi-View Generalization and Diagnostic Analysis of Machine-Generated Code Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, quick

Summary

UCSC-NLP's system for SemEval-2026 Task 13 addresses machine-generated code detection, covering both binary detection (Subtask A) and multi-class attribution (Subtask B). For Subtask A, the team developed a multi-view training framework utilizing UniXcoder-base, enhanced with domain-specific structural prefixes, delexicalization incorporating symmetric KL consistency loss, and token dropout. This approach achieved a macro F1 of 0.845 on an out-of-distribution test set, demonstrating strong generalization across five unseen languages and two unseen domains. For Subtask B, a diagnostic analysis revealed significant majority-class bias in transformer-based detectors. Despite an 88.4% accuracy, the system showed a near-complete failure in minority-class attribution, evidenced by a 0.086 Macro F1. This highlights that standard fine-tuning is inadequate for fine-grained generator identification and underscores the necessity for imbalance-aware, structure-focused modeling in future code detection research.

Key takeaway

For Machine Learning Engineers developing code detection systems, your focus on overall accuracy metrics like 88.4% can mask critical failures in fine-grained attribution. You should prioritize diagnostic analyses for minority classes, especially when dealing with diverse code generators, as standard fine-tuning is insufficient. Consider implementing imbalance-aware and structure-focused modeling techniques to ensure robust and equitable performance across all generator types, moving beyond simple binary detection.

Key insights

The system achieved high binary detection but exposed severe multi-class attribution bias in machine-generated code detection.

Principles

• Multi-view training enhances generalization in code detection.
• Majority-class bias severely impacts fine-grained attribution.
• Standard fine-tuning is insufficient for imbalanced code generation tasks.

Method

A multi-view training framework for binary detection uses UniXcoder-base, structural prefixes, delexicalization with symmetric KL consistency loss, and token dropout.

In practice

• Implement multi-view training with UniXcoder-base for robust code detection.
• Analyze minority-class performance beyond overall accuracy metrics.
• Explore imbalance-aware modeling for fine-grained generator attribution.

Topics

• SemEval-2026 Task 13
• Machine-Generated Code Detection
• Multi-View Training
• UniXcoder-base
• Code Attribution
• Imbalance-Aware Modeling

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• CodeChat-Eval: Evaluating Large Language Models in Multi-Turn Code Refinement Dialogues — LLMs significantly degrade in functional correctness during multi-turn code refinement, especially with semantic or additive changes.
• FLARE: Fine-Grained Diagnostic Feedback for LLM Code Refinement — FLARE improves LLM code refinement by using fine-grained, line-level bug diagnostics and a candidate search mechanism.
• Detecting Functional Memorization in Code Language Models — Code LLMs can functionally memorize logic even when textually dissimilar, requiring advanced auditing.
• From Logistic Regression to GPT-2: Building a Complete Spam Detection & Sentiment Analysis Pipeline — For imbalanced text classification, prioritize F1-score and Precision-Recall AUC over accuracy and ROC-AUC.
• Code Is More Than Text: Uncertainty Estimation for Code Generation — Code generation uncertainty estimation requires specialized methods accounting for code's unique properties, outperforming natural language-derived baselines.
• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.