Team Macaroni at SemEval-2026 Task 10: PsyCoMark: Psycholinguistic Conspiracy Marker Extraction and Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

Team Macaroni's submission to SemEval-2026 Task 10, "PsyCoMark," focused on identifying psycholinguistic conspiracy markers at the span level and classifying documents for conspiracy content. For Subtask 1, they fine-tuned several pretrained transformer encoders, analyzing their performance across various training configurations. Subtask 2 involved a hybrid system that combined ModernBERT-large with surface-level linguistic features. Their findings indicate that straightforward fine-tuning of robust pretrained models generally outperforms more complex pipelines, and adding handcrafted features does not consistently improve results. On the official test set, Team Macaroni ranked 18th in Subtask 1 with an overlap-based macro F1 of 0.16, and 20th in Subtask 2 with a macro F1 of 0.76.

Key takeaway

For NLP Engineers developing text classification systems, particularly for nuanced tasks like conspiracy detection, you should prioritize fine-tuning strong pretrained transformer models. The results from SemEval-2026 Task 10 suggest that this approach can be more effective than building complex pipelines or extensively engineering handcrafted features, potentially saving development time while achieving competitive performance. Focus your efforts on optimizing the fine-tuning process rather than over-complicating the model architecture.

Key insights

Simple fine-tuning of strong pretrained transformers often surpasses complex pipelines for psycholinguistic conspiracy detection.

Principles

• Straightforward fine-tuning of strong pretrained models is effective.
• Additional handcrafted features do not yield consistent improvements.

Method

Fine-tune pretrained transformer encoders for span-level marker identification; combine ModernBERT-large with surface-level linguistic features for document classification.

In practice

• Fine-tune transformer encoders for text classification tasks.
• Evaluate simple fine-tuning before complex feature engineering.

Topics

• SemEval-2026
• Conspiracy Detection
• Psycholinguistics
• Transformer Encoders
• Fine-tuning
• ModernBERT

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• Muted: Multilingual Targeted Offensive Speech Identification and Visualization — Muted identifies targeted offensive speech using transformer models and spaCy, requiring no fine-tuning for toxic span detection.
• Robust Dual-Signal Fusion: Hybrid Neuro-Symbolic Gating with Compressed Chain-of-Thought Refinement for Irony Detection in Social Media Texts — Hybrid neuro-symbolic fusion with compressed Chain-of-Thought effectively overcomes LLM literalism for zero-shot irony detection.
• FineREX: Fine-Tuned NER-RE for Human Smuggling Knowledge Graphs — Domain-specific fine-tuning of LLMs significantly enhances knowledge graph construction quality and efficiency for complex, jargon-heavy legal texts.
• MentalMARBERT: Domain-Adaptive Pre-training and Two-Stage Fine-Tuning for Arabic Mental Health Disorders Detection — Domain-adaptive pre-training and hierarchical classification significantly improve Arabic mental health disorder detection.
• Detecting HIV-Related Stigma in Clinical Narratives Using Large Language Models — LLMs can effectively detect HIV-related stigma in clinical notes, with encoder models outperforming generative models in zero-shot settings.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.