wangkongqiang@EEUCA 2026: Understanding Toxic Behavioral Intent in Gaming Chat Logs

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Gaming & Interactive Media · Depth: Advanced, quick

Summary

The wangkongqiang team participated in the EEUCA 2026 shared task on Understanding Toxic Behavioral Intent in Gaming Chat Logs, aiming to classify player utterances into categories like Hate and Harassment, Threats, or Non-toxic. Utilizing a dataset of 53,000 game chat utterances from World of Tanks, the team employed a supervised learning method, fine-tuning Qwen2 LLMs. Their best result on the test set, achieved with a fine-tuned qwen2_7B LLM, yielded a Macro F1 score of 0.5776, Accuracy of 0.9075, Precision (Macro) of 0.6847, and Recall (Macro) of 0.5343. This performance secured an 8th-place ranking among all participating teams in the task.

Key takeaway

For NLP engineers developing content moderation systems for online gaming, this work demonstrates that fine-tuning large language models like Qwen2_7B can achieve competitive results in classifying toxic chat intent. While the Macro F1 score of 0.5776 indicates room for improvement, you should consider LLM fine-tuning as a viable approach for multi-class toxicity detection, especially when dealing with diverse offensive categories. Evaluate your models rigorously using macro-averaged metrics.

Key insights

Fine-tuning Qwen2 LLMs can classify toxic behavioral intent in gaming chat logs with moderate F1 scores.

Principles

• Supervised learning is effective for chat toxicity classification.
• Macro F1 score is a key metric for multi-class imbalance.

Method

Apply supervised learning by fine-tuning pre-trained models, specifically Qwen2 LLMs, on labeled gaming chat utterances to classify toxic intent.

In practice

• Use Qwen2_7B for gaming chat toxicity detection.
• Evaluate classification systems using Macro F1, Accuracy, Precision, and Recall.

Topics

• Toxicity Detection
• Gaming Chat Analysis
• LLM Fine-tuning
• Qwen2
• Content Classification
• Natural Language Processing

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• Holy $#!t: Are popular toxicity models simply profanity detectors? — Toxicity models often misclassify positive profanity due to inadequate training data and contextual understanding.
• FineREX: Fine-Tuned NER-RE for Human Smuggling Knowledge Graphs — Domain-specific fine-tuning of LLMs substantially improves knowledge graph quality and efficiency for complex legal text.
• Harder to Defend: Towards Chinese Toxicity Attacks via Implicit Enhancement and Obfuscation Rewriting — Chinese implicit toxicity, combining indirectness and obfuscation, presents a major challenge for LLM detectors, with CITA revealing high missed-detection rates.
• Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection — LLM labels can match human-level F1 scores for hostility detection but exhibit distinct error profiles.
• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.