UCSC NLP at SemEval-2026 Task 10: Boundary-Aware Span Extraction and RoBERTa Classification for Conspiracy Detection

2026-07-06 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

UCSC NLP developed systems for SemEval-2026 Task 10 (PsyCoMark), focusing on conspiracy marker extraction (Subtask 1) and document-level conspiracy detection (Subtask 2). For marker extraction, the team formulated the task as multi-label span classification, employing enumerated candidate spans, IoU >= 0.95 positive labeling, hard-negative sampling, and containment-based non-maximum suppression (NMS) with boundary-aware span representations. Document classification was handled independently using a sequence classifier, incorporating label smoothing and a stratified train-validation split. Analysis revealed robust detection for entity-like roles such as Actor and Victim, while abstract roles like Action, Effect, and Evidence showed sensitivity to boundary criteria. On the official test set, their systems achieved 7th place in Subtask 1 with a 0.2251 macro F1 score and 11th place in Subtask 2 with a 0.7694 weighted F1 score.

Key takeaway

For NLP Engineers developing conspiracy detection systems, this work highlights the effectiveness of boundary-aware span representations. You should prioritize robust detection strategies for entity-like roles, as abstract roles remain sensitive to boundary criteria. Consider implementing multi-label span classification with IoU >= 0.95 positive labeling and hard-negative sampling to improve marker extraction. Further research into refining boundary criteria for abstract concepts is crucial for enhancing overall system performance.

Key insights

The UCSC NLP team used boundary-aware span extraction and RoBERTa classification for conspiracy detection in SemEval-2026 Task 10.

Principles

• Multi-label span classification can identify conspiracy markers.
• Entity-like roles are more robustly detected than abstract roles.
• Boundary criteria significantly impact abstract role detection.

Method

Marker extraction uses multi-label span classification with IoU >= 0.95 positive labeling, hard-negative sampling, and containment-based NMS. Document classification employs a sequence classifier with label smoothing.

In practice

• Apply IoU >= 0.95 for positive span labeling.
• Use hard-negative sampling for span classification.
• Implement containment-based NMS for boundary refinement.

Topics

• Conspiracy Detection
• Span Extraction
• RoBERTa Classification
• SemEval-2026 Task 10
• Natural Language Processing
• Multi-label Classification

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Explicit Evidence Grounding via Structured Inline Citation Generation — LLMs excel at document retrieval but struggle with precise evidence span identification for faithful inline citations.
• Introducing Span Categorization in Prodigy and spaCy — Span Categorization (spancat) in spaCy extracts and classifies long, overlapping text spans, providing explicit control and confidence scores.
• Extending an Ensemble Baseline with Corpus-Based Graph Features for Portuguese Pun Detection — Graph-based features can augment pun detection, but their utility depends on specific lexical relation encoding.
• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• A Beginner’s Guide to NLP Token Classification: NER, POS Tagging, Chunking, and BERT — Token classification, including NER, POS tagging, and chunking, is fundamental for machines to understand human language.
• Structured Sentiment Analysis in Brazilian Portuguese: An Exploratory Study Using BERTimbau — Span-level sentiment extraction provides a viable baseline for Structured Sentiment Analysis in Brazilian Portuguese.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.