NJUSTKMG at SemEval 2026 Task 10 PsyCoMark—Subtask 2:Conspiracy Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

NJUSTKMG's system achieved first place in SemEval-2026 Task 10: PsyCoMark—Subtask 2: Conspiracy Detection, securing an F1 Score of 0.8874. Their two-stage approach effectively identifies conspiracy theories in text. The initial stage employs a large pre-trained model to filter out samples clearly unrelated to conspiracy theories. Subsequently, the remaining samples are processed using a retrieval-enhanced custom prompt strategy combined with the Roberta-Large model. This second stage involves fine-tuning the model with weighted predictions based on relevant retrieved information, significantly enhancing detection accuracy. The team also provided an analysis of their methods' effectiveness, including the advantages and limitations of both large model-based filtering and retrieval-augmented fine-tuning.

Key takeaway

For NLP Engineers developing robust text classification systems, this work demonstrates a highly effective strategy for complex tasks like conspiracy detection. You should consider implementing a two-stage architecture, leveraging a large model for initial filtering to reduce noise, followed by retrieval-augmented fine-tuning with a smaller, specialized model like Roberta-Large. This approach significantly boosts accuracy, as evidenced by the 0.8874 F1 Score, and can be adapted for other challenging content moderation or misinformation detection applications.

Key insights

A two-stage approach combining large model filtering and retrieval-augmented fine-tuning excels at conspiracy detection.

Principles

• Filter irrelevant data early
• Augment fine-tuning with retrieval
• Weighted predictions enhance accuracy

Method

A large pre-trained model first filters unrelated samples. Remaining samples use a retrieval-enhanced custom prompt with Roberta-Large, fine-tuned via weighted predictions.

In practice

• Implement two-stage text classification
• Use large models for initial filtering
• Integrate retrieval for context-aware fine-tuning

Topics

• Conspiracy Detection
• SemEval 2026
• Two-Stage Classification
• Roberta-Large
• Retrieval-Augmented Models
• Text Classification

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• Multi-Stage Training for Abusive Comment Detection in Indic Languages — A multi-stage pipeline combining language preprocessing and model ensembles effectively detects abusive Indic language comments while minimizing false positives.
• Identification of fake news in Portuguese: a look at the generalization of models — Language models for fake news detection show significant performance degradation in cross-generalization scenarios.
• LLM-based Detection of Manipulative Political Narratives — LLM-driven pipelines can effectively identify and cluster manipulative political narratives in raw social media data.
• Shapley Value-Guided Adaptive Ensemble Learning for Explainable Financial Fraud Detection with U.S. Regulatory Compliance Validation — Explainable AI models can achieve high fraud detection accuracy while meeting stringent U.S. financial regulations.
• Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking — WKGFC unifies structured KGs and web evidence via an LLM agent for adaptive, self-improving fact-checking.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.