FactUEP at SemEval-2026 Task 4: Structured Narrative Similarity Scoring with Aspect Decomposition and Weak-Signal Gating

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

The FactUEP system, an LLM-based approach, was specifically developed for narrative similarity prediction within the SemEval-2026 Task 4 Track A competition. This system operationalizes three core dimensions—Abstract Theme, Course of Action, and Outcomes—by employing schema-constrained prompting. This technique enforces structured outputs and ensures alignment with the competition's annotation protocol. FactUEP proceeds through three distinct stages: initial structured aspect decomposition and scoring, followed by a weak-signal gating mechanism designed for low-confidence cases, and concludes with a targeted LLM-based tiebreak. The final model achieved near-human performance, securing the second rank on the Track A leaderboard.

Key takeaway

For NLP Engineers developing narrative understanding systems, FactUEP's approach offers a robust framework. You should consider integrating structured aspect decomposition for Abstract Theme, Course of Action, and Outcomes, enforced by schema-constrained prompting. Implementing a multi-stage pipeline with weak-signal gating for low-confidence predictions and a targeted LLM tiebreak can significantly improve accuracy, potentially achieving near-human performance in complex similarity tasks.

Key insights

LLM-based narrative similarity benefits from structured decomposition, weak-signal gating, and targeted tiebreaks for near-human performance.

Principles

• Decompose narratives into core dimensions.
• Use schema-constrained prompting.
• Gate low-confidence predictions.

Method

The system performs structured aspect decomposition and scoring, applies weak-signal gating for low-confidence cases, then uses a targeted LLM-based tiebreak.

In practice

• Apply schema-constrained prompting.
• Implement multi-stage prediction pipelines.
• Utilize weak signals for uncertainty handling.

Topics

• Narrative Similarity
• LLM Systems
• SemEval-2026
• Aspect Decomposition
• Schema-Constrained Prompting
• Weak-Signal Gating

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Characterizing Narrative Content in Web-scale LLM Pretraining Data — Narrative structure in web-scale LLM pretraining data is measurable and unevenly distributed, impacting model reasoning.
• SemEval-2026 Task 4: Narrative Story Similarity and Narrative Representation Learning — The NSNRL task operationalizes narrative similarity for evaluation and identifies areas for automated system improvement.
• Multi-View Attention Multiple-Instance Learning Enhanced by LLM Reasoning for Cognitive Distortion Detection — Combining LLMs with MIL and ELB components improves cognitive distortion detection by enhancing interpretability and reasoning.
• Measuring the sensitivity of LLM-based structured extraction to prompt, model, and schema choices in clinical discharge summaries — LLM extraction sensitivity varies significantly with schema design and model choice, especially on absence-versus-silence distinctions.
• Narrative Knowledge Weaver: Narrative-Centric Retrieval-Augmented Reasoning for Long-Form Text Understanding — Narrative Knowledge Weaver (NKW) enhances RAG by modeling evolving story worlds through structured narrative assets and multi-channel evidence assembly.
• Detecting HIV-Related Stigma in Clinical Narratives Using Large Language Models — LLMs can effectively detect HIV-related stigma in clinical notes, with encoder models outperforming generative models in zero-shot settings.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.