Closing the Gap at CRAC 2026: Two-Stage Adaptation for LLM-Based Multilingual Coreference Resolution

2026-05-19 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing, Software Development & Engineering · Depth: Expert, extended

Summary

A system submitted to the LLM track of the CRAC 2026 shared task achieved the top rank in its track and third overall, with an average CoNLL F1 score of 74.32 on the official test set. The system is based on the Gemma-3-27b model, fine-tuned using a two-stage adaptation strategy: a multilingual base adapter followed by dataset-specific adapters. Key design choices included representing mention spans by their headword using a minimal XML-inspired format with local reindexing, and iteratively annotating documents. This approach proved effective across various languages, document lengths, and annotation guidelines, significantly closing the performance gap between LLM-based systems and traditional coreference resolution pipelines. The research also highlighted the importance of addressing annotation heterogeneity across datasets.

Key takeaway

For AI Engineers developing multilingual coreference resolution systems, adopting a two-stage fine-tuning approach with a strong base model like Gemma-3-27b and specialized adapters for diverse datasets is crucial. Your team should also prioritize optimizing input formats, such as minimal XML headword representation, and implement local reindexing to manage coreference IDs effectively, as these strategies significantly enhance performance and address annotation inconsistencies.

Key insights

Two-stage fine-tuning and optimized input formats enable LLMs to achieve state-of-the-art multilingual coreference resolution.

Principles

• Simplify mention representation to improve LLM performance.
• Address dataset-specific annotation inconsistencies via specialized adapters.
• Iterative processing with local reindexing enhances long-range coreference.

Method

The system fine-tunes Gemma-3-27b using QLoRA with a two-stage strategy: a multilingual base adapter, then dataset-specific continual SFT. It employs a minimal XML headword format, local reindexing, and an iterative, on-the-fly cleaning process.

In practice

• Use 4-bit QLoRA for efficient LLM fine-tuning.
• Implement headword-only mention representation for coreference tasks.
• Apply local reindexing to manage coreference IDs in long contexts.

Topics

• Multilingual Coreference Resolution
• LLM Fine-tuning
• Gemma-3 Model
• Two-Stage Adaptation
• QLoRA

Code references

• lattice-8094/coref-llm
• ondfa/text2text-coref

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Multilingual Coreference Resolution via Cycle-Consistent Machine Translation — Coreference resolution in low-resource languages can be significantly improved by cycle-consistent machine translation data augmentation.
• System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5 — Domain-specific LoRA fine-tuning on Qwen2.5-14B with CCPoetry-49K significantly enhances classical Chinese poetry understanding and translation.
• AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task — Adapting AlignAtt for decoder-only LLMs enables simultaneous speech translation by re-engineering attention mechanisms.
• Enhancing Multilingual Reasoning via Steerable Model Merging — Dynamically modulating source model contributions resolves conflicts in multilingual LLM merging for improved reasoning.
• Evaluating Cross-lingual Knowledge Consistency in Code-Mixed vis-a-vis Indian Languages using IndicKLAR — Code-mixing inputs significantly improves large language model knowledge recall in Indian languages, nearly matching English performance.
• A NATO-accredited organization published a report concerning language-based disparities in LLM performance (Published April 25, 2026) — LLM performance varies significantly by language, but adaptation strategies can bridge the gap.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.