AGAI at SemEval-2026 Task 10: Enhancing Conspiracy Detection via Instruction-tuned LLMs

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

AGAI's solution for SemEval-2026 Task 10, subtask2, addresses the complex challenge of automated conspiracy detection in text, a task more difficult than traditional toxic text detection due to the lack of explicit semantic indicators. Their approach utilizes a Large Language Model (LLM) as its core, fine-tuned with Low-Rank Adaptation (LoRA) to boost detection performance. To generate probabilistic confidence scores while retaining the LLM's generative capabilities, the team implemented a hybrid loss function, combining both generative and token classification losses. Furthermore, semi-supervised learning was integrated using unlabeled data to enhance classification accuracy. This method achieved a test accuracy of 0.87, securing 2nd place in both stages of the competition leaderboard.

Key takeaway

For NLP Engineers developing robust text classification models, especially for nuanced tasks like conspiracy detection, you should consider instruction-tuned LLMs fine-tuned with LoRA. Implementing a hybrid loss function that combines generative and token classification losses can yield probabilistic confidence scores while maintaining model flexibility. Additionally, integrating semi-supervised learning with unlabeled data can significantly refine your model's accuracy, as demonstrated by the 0.87 test accuracy achieved.

Key insights

Instruction-tuned LLMs with hybrid loss and semi-supervised learning effectively detect subtle textual conspiracies.

Principles

• Conspiracy detection needs more than explicit semantic indicators.
• Hybrid loss functions can balance generative and classification needs.
• Semi-supervised learning refines classification accuracy.

Method

Fine-tuning an LLM backbone with LoRA, using a hybrid loss (generative + token classification) to get probabilistic scores, and incorporating semi-supervised learning with unlabeled data.

In practice

• Apply LoRA for efficient LLM fine-tuning.
• Combine generative and classification losses for nuanced output.
• Utilize unlabeled data via semi-supervised learning.

Topics

• Conspiracy Detection
• Large Language Models
• Instruction Tuning
• LoRA
• Hybrid Loss
• Semi-supervised Learning
• SemEval-2026

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• Auditing Training Data in Domain-adapted LLMs: LoRA-MINT — LoRA-MINT uses perplexity and synthetic samples to audit training data membership in LoRA-fine-tuned LLMs.
• Fact4ac at the Financial Misinformation Detection Challenge Task: Reference-Free Financial Misinformation Detection via Fine-Tuning and Few-Shot Prompting of Large Language Models — Detecting financial misinformation without external references is achievable using LLMs combined with specific fine-tuning.
• Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditing — STATEWITNESS uses a separate decoder to explain LLM hidden states, improving deception detection and providing inspectable evidence.
• A Lightweight LLM Framework for Disaster Humanitarian Information Classification — Parameter-efficient fine-tuning significantly outperforms prompting and RAG for disaster tweet classification, enabling efficient, accurate crisis intelligence.
• LLMs Are Not Enough for Multimodal Fake News Detection: Why Global Label Propagation Helps — LLMs are better as weak annotators than direct classifiers for multimodal fake news detection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.