Osint at SemEval-2026 Task 13: A Distribution-Aware Framework for Machine-Generated Code Detection and Multi-Source Authorship Attribution

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

The paper "Osint at SemEval-2026 Task 13" presents a distribution-aware framework for detecting machine-generated code and attributing authorship. This framework addresses challenges posed by code-generating LLMs like DeepSeek, Qwen, and Meta-LLaMA, which increase risks of plagiarism and insecure code. SemEval-2026 Task 13 includes two subtasks: binary human–machine code detection (Subtask A) and multi-class authorship attribution across ten LLM families (Subtask B). For Subtask A, the authors fine-tuned models such as RoBERTa, CodeBERT, GraphCodeBERT, and StarCoderBase-1B, employing a stratified sampling strategy with class-weighted loss to handle data imbalance and out-of-distribution (OOD) shifts. For Subtask B, they used undersampling, inverse-frequency weights, syntactic noising, and curriculum-based dual-path training with TinyStarCoderPy and CodeBERT to mitigate extreme human-class imbalance. The findings emphasize the importance of long-context modeling, distribution-aware sampling, and noise-robust training for reliable real-world performance in both detection and attribution tasks.

Key takeaway

For Machine Learning Engineers developing code detection systems, you should prioritize integrating long-context modeling and distribution-aware sampling strategies. When facing imbalanced datasets or out-of-distribution code, implement techniques like stratified sampling with class-weighted loss or undersampling with inverse-frequency weights. Additionally, consider noise-robust training, potentially with syntactic noising, to enhance the reliability of your models in real-world, multi-language, and multi-LLM environments, mitigating risks of plagiarism and insecure code.

Key insights

Long-context modeling, distribution-aware sampling, and noise-robust training are crucial for reliable machine-generated code detection.

Principles

• Distribution-aware sampling is crucial for OOD shifts.
• Noise-robust training improves real-world reliability.
• Long-context modeling is key for code detection.

Method

For Subtask A, fine-tune RoBERTa, CodeBERT, GraphCodeBERT, and StarCoderBase-1B with stratified sampling and class-weighted loss. For Subtask B, use undersampling, inverse-frequency weights, syntactic noising, and curriculum-based dual-path training with TinyStarCoderPy and CodeBERT.

In practice

• Apply stratified sampling to imbalanced code datasets.
• Use class-weighted loss for out-of-distribution scenarios.
• Implement syntactic noising for robust code attribution.

Topics

• Machine-Generated Code Detection
• Authorship Attribution
• Large Language Models
• SemEval-2026 Task 13
• CodeBERT
• Data Imbalance Mitigation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Authorship Attribution in Multilingual Machine-Generated Texts — Multilingual authorship attribution for LLM-generated text is complex, with current monolingual methods showing limited cross-lingual transferability across diverse language families.
• A Zipf-preserving, long-range correlated surrogate for written language and other symbolic sequences — A new model generates symbolic sequence surrogates preserving both symbol frequencies and long-range correlations.
• Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection — A mechanism-oriented taxonomy of indirect linguistic encoding significantly enhances LLM-based coded language detection.
• Detecting Data Contamination in Large Language Models — Black-box MIAs are currently ineffective at reliably detecting data contamination in LLM training corpora.
• Code Is More Than Text: Uncertainty Estimation for Code Generation — Code generation uncertainty estimation requires specialized methods accounting for code's unique properties, outperforming natural language-derived baselines.
• The Power of Test-Time Training for Approximate Sampling — Test-Time Training (TTT) for approximate sampling is formalized, revealing that its query complexity can be circumvented by bounding the distribution class size.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.