UCSC NLP at SemEval-2026 Task 10: Boundary-Aware Span Extraction and RoBERTa Classification for Conspiracy Detection

2026-07-08 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, long

Summary

UCSC NLP presented its systems for SemEval-2026 Task 10 (PsyCoMark), focusing on conspiracy marker extraction (Subtask 1) and document-level conspiracy detection (Subtask 2). For Subtask 1, a multi-label span classification approach was implemented, utilizing boundary-aware span representations, IoU≥0.95 positive labeling, hard-negative sampling, and containment-based non-maximum suppression. This system achieved a 0.2251 macro F1 score on the official test set, placing 7th. Subtask 2 employed an independent RoBERTa-large sequence classifier with label smoothing and a stratified train–validation split, which scored 0.7694 weighted F1 and ranked 12th. Both models were trained separately on a corpus of over 4,100 English Reddit submissions. Analysis revealed that entity-like roles like "Actor" and "Victim" were robustly detected, but abstract roles such as "Action," "Effect," and "Evidence" remained highly sensitive to boundary criteria.

Key takeaway

For NLP Engineers developing span extraction models, particularly for abstract semantic roles, you should prioritize precise boundary learning. Implement hard-negative sampling and strict IoU thresholds (e.g., IoU≥0.95) during training to improve span localization. Also, ensure your development metrics closely match the target evaluation criteria to avoid performance gaps. This approach can enhance the robustness of your models in complex information extraction tasks.

Key insights

Conspiracy detection benefits from explicit narrative role modeling, but abstract roles pose significant boundary extraction challenges.

Principles

• Hard-negative mining improves span boundary localization.
• Independent subtask models avoid error propagation.
• Evaluation metric alignment is crucial for development.

Method

For span extraction, multi-label classification over candidate spans uses IoU≥0.95 labeling, hard-negative sampling, and containment-based NMS. Document classification uses a RoBERTa-large sequence classifier with label smoothing.

In practice

• Use IoU≥0.95 for precise span boundary learning.
• Implement hard-negative sampling for boundary discrimination.
• Align validation metrics precisely with official evaluations.

Topics

• Conspiracy Detection
• Span Extraction
• RoBERTa
• SemEval-2026
• Multi-label Classification
• Natural Language Processing

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Explicit Evidence Grounding via Structured Inline Citation Generation — LLMs excel at document retrieval but struggle with precise evidence span identification for faithful inline citations.
• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• Introducing Span Categorization in Prodigy and spaCy — Span Categorization (spancat) in spaCy extracts and classifies long, overlapping text spans, providing explicit control and confidence scores.
• Extending an Ensemble Baseline with Corpus-Based Graph Features for Portuguese Pun Detection — Graph-based features can augment pun detection, but their utility depends on specific lexical relation encoding.
• FineREX: Fine-Tuned NER-RE for Human Smuggling Knowledge Graphs — Domain-specific fine-tuning of LLMs significantly enhances knowledge graph construction quality and efficiency for complex, jargon-heavy legal texts.
• Subject-level Inference for Realistic Text Anonymization Evaluation — Subject-level inference evaluation is crucial for realistic text anonymization, as span masking alone is insufficient.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.