Team HITS at SemEval-2026 Task 4:Enhancing narrative text embedding model training with hard negatives generation and self-distillation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, short

Summary

Team HITS participated in SemEval-2026 Task 4, presenting a method to enhance narrative text embedding model training. Their approach involves two key stages. First, they utilize the Qwen2.5-32B-Instruct model to generate "hard negatives" derived from three distinct narrative dimensions. Following this, they train a Qwen3-Embedding-8B model. This training incorporates a multi-negative contrastive objective, a technique designed to improve the model's ability to differentiate between similar and dissimilar text pairs. Additionally, the training process employs self-distillation, a method where a model learns from its own predictions or from a more powerful version of itself, further refining the embedding quality for narrative texts. This work was presented at the 20th International Workshop on Semantic Evaluation in July 2026.

Key takeaway

For Machine Learning Engineers developing narrative text embedding models, consider integrating advanced training techniques to boost performance. You should explore generating hard negatives using large language models like Qwen2.5-32B-Instruct to create challenging training examples. Furthermore, applying a multi-negative contrastive objective combined with self-distillation on models such as Qwen3-Embedding-8B can significantly refine embedding quality and differentiation for complex narrative data.

Key insights

Hard negative generation and self-distillation enhance narrative text embedding model training.

Principles

• Hard negatives improve contrastive learning.
• Self-distillation refines embedding models.
• Multi-negative objective boosts differentiation.

Method

Generate hard negatives using Qwen2.5-32B-Instruct from three narrative dimensions. Train Qwen3-Embedding-8B with a multi-negative contrastive objective and self-distillation.

In practice

• Use Qwen2.5-32B-Instruct for negative sampling.
• Apply multi-negative contrastive loss.
• Implement self-distillation for refinement.

Topics

• Narrative Text Embeddings
• Hard Negative Generation
• Self-Distillation
• Qwen2.5-32B-Instruct
• Qwen3-Embedding-8B
• Contrastive Learning

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers — Finetuning multimodal embedding models on domain-specific data significantly boosts performance over general-purpose models.
• Hugging Face Journal Club: Embarrassingly Simple Self-Distillation Improves Code Generation — Self-distillation via temperature-tuned SFT improves model performance by refining token distribution.
• Algorithm Selection with Zero Domain Knowledge via Text Embeddings — Pretrained text embeddings can effectively replace hand-crafted features for domain-agnostic algorithm selection.
• ttda704 at SemEval-2026 Task 4: Modeling Narrative Structures via Pseudonymization and Multi-View Sentence Alignment — Using contrastive learning with fine-tuned sentence transformers and multi-view alignment effectively models narrative structures and similarity.
• The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models — Multi-teacher prompt distillation significantly improves VLM student performance, particularly under domain shift, through complementary teacher supervision.
• Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillation — Pedagogically-inspired knowledge distillation systematically improves student model performance and efficiency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.