FMISUYotkovaKastreva at SemEval-2026 Task 13: Lightweight Detection of LLM-Generated Code via Stylometric Signals

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, quick

Summary

FMISUYotkovaKastreva's system for SemEval-2026 Task 13 addresses the challenge of detecting LLM-generated code across diverse programming languages and application scenarios, specifically focusing on generalization to unseen languages and domains. Participating in Subtask A (binary classification), their approach combines pretrained code encoders with lightweight feature-based methods. The system employs ratio-based features designed to be less sensitive to code snippet length, alongside parsing engines and a programming-language classifier to extract descriptiveness-related signals. It also integrates a dedicated code-vs-text line classifier to identify natural language segments embedded within code samples. The final predictions are generated by a shallow decision tree augmented with heuristic rules derived from data analysis. This method is computationally efficient, requiring only CPU resources for training, and achieves near-instant inference, positioning it as a lightweight alternative to larger pretrained models.

Key takeaway

For Machine Learning Engineers developing code analysis tools, you should consider lightweight, feature-based approaches for detecting LLM-generated code. This method, which uses CPU-only training and near-instant inference, offers a computationally efficient alternative to large pretrained models. By focusing on stylometric signals, ratio-based features, and heuristic rules, you can build effective detection systems that generalize across languages and domains, reducing your infrastructure costs and deployment complexity.

Key insights

Lightweight detection of LLM-generated code can be achieved using stylometric signals and shallow models, offering efficient, CPU-only inference.

Principles

• Ratio-based features reduce snippet length sensitivity.
• Stylometric signals aid LLM-generated code detection.
• Heuristic rules enhance shallow model predictions.

Method

Combine ratio-based features, parsing engines, and a code-vs-text line classifier. Train a shallow decision tree with heuristic rules for binary classification of LLM-generated code.

In practice

• Implement CPU-only LLM code detection.
• Utilize stylometric features for code analysis.
• Integrate line classifiers for mixed code/text.

Topics

• LLM-Generated Code Detection
• Stylometric Analysis
• Code Classification
• SemEval-2026 Task 13
• Lightweight Machine Learning
• Feature Engineering

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Beyond Problem Solving: UOJ-Bench for Evaluating Code Generation, Hacking, and Repair in Competitive Programming — LLMs can complement traditional online judges by identifying subtle code errors, but effective error detection remains computationally intensive.
• VeriCWEty: Embedding enabled Line-Level CWE Detection in Verilog — An embedding-based framework improves RTL code vulnerability detection and localization.
• LLM-Driven Feature Discovery — LLMs can effectively discover and label qualitative behavioral features from model transcripts without internal access.
• KRONE: Scalable LLM-Augmented Log Anomaly Detection via Hierarchical Abstraction — Krone leverages hierarchical log structures and hybrid LLM-pattern matching for efficient, accurate, and interpretable anomaly detection.
• Detecting Functional Memorization in Code Language Models — Code LLMs can functionally memorize logic even when textually dissimilar, requiring advanced auditing.
• Detecting Overflow in Compressed Token Representations for Retrieval-Augmented Generation — Token overflow in compressed LLM representations can be detected by incorporating query information into lightweight classifiers.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.