JIA at SemEval-2026 Task 10: A Dual-Track System with BERT-based Encoders and LLMs for Conspiracy Analysis

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

JIA's dual-track system, developed for SemEval-2026 Task 10, addresses conspiracy theory detection and psycholinguistic marker extraction. The system evaluates several architectures, including DistilBERT, BERT-Base, DeBERTa-V3, RoBERTa, and instruction-tuned Qwen2.5 models. For the detection task, the Qwen2.5-14B (full-shot) model achieved the highest performance, securing a Weighted F1-score of 0.80. However, the psycholinguistic marker extraction component proved more challenging. While the fine-tuned LLM performed best on "Actors" markers, its generalization was limited in categories like "Evidence" and "Effect," indicating persistent semantic ambiguity in these areas.

Key takeaway

For NLP Engineers developing misinformation detection systems, JIA's dual-track approach suggests prioritizing instruction-tuned LLMs like Qwen2.5-14B for robust conspiracy detection. However, prepare for significant challenges in accurately extracting fine-grained psycholinguistic markers such as "Evidence" or "Effect" due to inherent semantic ambiguities. You may need to explore specialized techniques or domain-specific fine-tuning for these difficult categories.

Key insights

A dual-track system combining BERT-based encoders and LLMs excels at conspiracy detection but struggles with nuanced marker extraction.

Principles

• Instruction-tuned LLMs like Qwen2.5-14B achieve strong conspiracy detection.
• Semantic ambiguity significantly impedes fine-grained psycholinguistic marker extraction.

Method

The system evaluates multiple architectures (DistilBERT, BERT-Base, DeBERTa-V3, RoBERTa, Qwen2.5) for conspiracy detection and psycholinguistic marker extraction.

In practice

• Consider Qwen2.5-14B for robust conspiracy detection tasks.
• Anticipate challenges extracting nuanced markers like "Evidence" or "Effect."

Topics

• Conspiracy Theory Detection
• Psycholinguistic Marker Extraction
• SemEval-2026 Task 10
• BERT-based Encoders
• Large Language Models
• Qwen2.5

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• Explainable Detection of Depression Status Shifts from User Digital Traces — A BERT-LLM framework detects mental health shifts from digital traces, providing explainable, temporally coherent reports.
• The Geometry of Truth: Detecting LLM Hallucinations with Geometric Algebra — Type II hallucinations are context contradictions, dangerous due to their plausibility and standard similarity metric blindness.
• A Multitask Transformer for Offensive Language Detection and Target Identification in HateBR — Multitask learning improves offensive language detection consistency but may reduce fine-grained target attribution.
• Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection — A mechanism-oriented taxonomy of indirect linguistic encoding significantly enhances LLM-based coded language detection.
• Robust Dual-Signal Fusion: Hybrid Neuro-Symbolic Gating with Compressed Chain-of-Thought Refinement for Irony Detection in Social Media Texts — Hybrid neuro-symbolic fusion with compressed Chain-of-Thought effectively overcomes LLM literalism for zero-shot irony detection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.