Duluth at SemEval-2026 Task 4: A Hybrid Approach to Narrative Similarity using Bi-Encoder Embeddings with Cross-Encoder Tie breaking using Learned Weights

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

The "Duluth" system, developed by Maxwell Bevers, Aidan Carlson, and Ted Pedersen, competed in SemEval-2026 Task 4 on Narrative Similarity, achieving 58.5% accuracy and placing 39th overall. This hybrid approach first decomposes stories into four narrative components: theme, plot, emotion, and outcome. Each component is then encoded using a bi-encoder, specifically all-mpnet-base-v2. Cosine similarities from these encodings are combined through a learned pairwise ranking model. For candidate stories with similarity scores falling within a small margin of error, a cross-encoder, ms-marcoMiniLM-L-6-v2, acts as a tie-breaker. Error analysis indicated the system faces challenges with complex themes, stories featuring multiple protagonists, and contrasting outcomes.

Key takeaway

For NLP Engineers developing narrative understanding systems, this hybrid approach offers a structured method for similarity tasks. You should consider decomposing narratives into components like theme and plot, using bi-encoders for initial scoring, and reserving cross-encoders for tie-breaking. Be aware that such systems may struggle with highly complex themes or multiple protagonists, requiring further refinement or specialized handling for nuanced story structures.

Key insights

A hybrid bi-encoder and cross-encoder system for narrative similarity achieved 58.5% accuracy on SemEval-2026 Task 4, struggling with complexity.

Principles

• Decompose narratives into core components.
• Use bi-encoders for initial similarity scoring.
• Employ cross-encoders for tie-breaking.

Method

Decompose stories into theme, plot, emotion, and outcome. Encode components with all-mpnet-base-v2 and combine cosine similarities. Use ms-marcoMiniLM-L-6-v2 as a tie-breaker when scores are within a small margin.

In practice

• Apply all-mpnet-base-v2 for component embeddings.
• Integrate ms-marcoMiniLM-L-6-v2 for score tie-breaking.
• Decompose complex texts into simpler elements.

Topics

• Narrative Similarity
• Bi-encoder Embeddings
• Cross-encoder Models
• SemEval-2026
• Text Decomposition
• Ranking Models

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• SemEval-2026 Task 4: Narrative Story Similarity and Narrative Representation Learning — The NSNRL task operationalizes narrative similarity for evaluation and identifies areas for automated system improvement.
• ttda704 at SemEval-2026 Task 4: Modeling Narrative Structures via Pseudonymization and Multi-View Sentence Alignment — Using contrastive learning with fine-tuned sentence transformers and multi-view alignment effectively models narrative structures and similarity.
• Hybrid Search and Re-Ranking in Production RAG — Combining hybrid search, cross-encoder re-ranking, and metadata filtering significantly improves RAG system accuracy for enterprise knowledge.
• Narrative Knowledge Weaver: Narrative-Centric Retrieval-Augmented Reasoning for Long-Form Text Understanding — Narrative Knowledge Weaver (NKW) enhances RAG by modeling evolving story worlds through structured narrative assets and multi-channel evidence assembly.
• One Single Hub Text Breaks CLIP: Identifying Vulnerabilities in Cross-Modal Encoders via Hubness — Hubness in cross-modal encoders creates vulnerabilities where single texts achieve high, spurious similarity across many images.
• Automated Essay Scoring for Brazilian Portuguese. Evidence from Cross-Prompt Evaluation of ENEM Essays — Cross-prompt AES for Brazilian Portuguese reveals distinct model strengths across essay traits and context availability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.