AILS-NTUA at SemEval-2026 Task 10: Agentic LLMs for Psycholinguistic Marker Extraction and Conspiracy Endorsement Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

AILS-NTUA developed a novel agentic LLM pipeline for SemEval-2026 Task 10, designed to jointly extract psycholinguistic conspiracy markers and detect conspiracy endorsement. This system employs a decoupled architecture, distinguishing semantic reasoning from structural localization, unlike traditional classifiers. For marker extraction, it utilizes Dynamic Discriminative Chain-of-Thought (DD-CoT) with deterministic anchoring to address semantic ambiguity and character-level brittleness. For conspiracy detection, an "Anti-Echo Chamber" architecture, featuring an adversarial Parallel Council adjudicated by a Calibrated Judge, mitigates the "Reporter Trap" where models misclassify objective reporting. The system achieved a 0.24 Macro F1 score (+100% over baseline) on S1 and 0.79 Macro F1 (+49%) on S2, securing 3rd place on the S1 development leaderboard and 8th on the test set. This demonstrates structured agentic deliberation as an effective, interpretable alternative to fine-tuning for psycholinguistic NLP tasks.

Key takeaway

For NLP Engineers developing systems for misinformation detection, you should consider agentic LLM architectures as a robust alternative to traditional fine-tuning. This approach, which decouples semantic and structural challenges, can significantly improve performance and interpretability in psycholinguistic analysis. Implement techniques like Dynamic Discriminative Chain-of-Thought and adversarial deliberation to enhance marker extraction and overcome reporting biases, potentially achieving higher F1 scores than baseline methods.

Key insights

Agentic LLMs with decoupled reasoning and adversarial architectures effectively detect conspiracy endorsement and extract psycholinguistic markers.

Principles

• Decouple semantic reasoning from structural localization.
• Structured agentic deliberation offers an interpretable alternative to fine-tuning.

Method

The pipeline uses Dynamic Discriminative Chain-of-Thought (DD-CoT) for marker extraction and an "Anti-Echo Chamber" architecture with a Parallel Council and Calibrated Judge for conspiracy detection.

In practice

• Implement DD-CoT for precise marker extraction.
• Employ adversarial councils to counter reporting bias.

Topics

• Agentic LLMs
• Psycholinguistic Markers
• Conspiracy Detection
• SemEval-2026 Task 10
• Chain-of-Thought
• Misinformation Detection

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations — NLAs translate LLM activations into human-readable text, revealing internal thoughts and hidden motivations for auditing.
• Beating the Style Detector: Three Hours of Agentic Research on the AI-Text Arms Race — Agentic research rapidly reproduces and extends NLP studies, demonstrating LLMs' ability to evade AI-text detection.
• MADE: Beyond Scoring via a Multilingual Agentic Diagnosing Engine for Fine-Grained Evaluation Insights — MADE offers a structured, agentic approach for fine-grained, multilingual LLM evaluation diagnosis, moving beyond simple benchmark scores.
• Identifying High-Confidence Social Biases in LLMs for Trustworthy Conversational Tutoring Agents — LLMs in tutoring agents exhibit overconfident social biases, challenging detection in conversational contexts.
• Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering — LLM agents significantly reduce SAST false positives, but effectiveness is highly dependent on the backbone model and vulnerability category.
• Summarisation and Knowledge Distillation — How Agents Summarise Without Hallucinating — Robust LLM summarization requires architectural patterns to prevent hallucinations and ensure factual grounding and completeness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.