PortNLP at CRAC 2026: QLoRA Fine-Tuning with Bounded Entity Registry for Multilingual Coreference Resolution

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

PortNLP's submission to the CRAC 2026 Shared Task on Multilingual Coreference Resolution (LLM track) details a system that fine-tuned Qwen 3 14B using QLoRA. This system was trained on CorefUD 1.4 gold annotations across 27 corpora spanning 19 languages, achieving 68.69 CoNLL F1. Documents are processed in 500-700 character chunks, utilizing a bounded rolling context of 500 characters of recent annotated text and a scored entity registry tracking up to 30 active entities via a frequency-times-recency decay formula. The approach incorporates data augmentation and language-aware sampling to manage typological and data-size diversity. Probing experiments revealed that coreference signal is concentrated in attention value projections rather than MLP modules, with the strongest readout at the earliest transformer layer.

Key takeaway

For NLP Engineers aiming to improve multilingual coreference resolution, consider fine-tuning large language models like Qwen 3 14B with QLoRA. Your system should incorporate a bounded rolling context and a scored entity registry to manage long-range dependencies effectively. Additionally, explore data augmentation and language-aware sampling to address data diversity, potentially replicating the 68.69 CoNLL F1 performance. Probing attention value projections might also reveal crucial task-specific signals for further optimization.

Key insights

QLoRA fine-tuning with a bounded entity registry effectively enhances multilingual coreference resolution in large language models.

Principles

• Coreference signal concentrates in attention value projections.
• Earliest transformer layers show strongest coreference readout.
• Bounded entity registries improve context for coreference.

Method

Fine-tune Qwen 3 14B with QLoRA on CorefUD 1.4. Process documents in 500-700 character chunks using a bounded rolling context and a scored entity registry (30 active entities, frequency-times-recency decay). Apply data augmentation and language-aware sampling.

In practice

• Implement QLoRA for efficient LLM fine-tuning.
• Design context windows with dynamic entity tracking.
• Probe attention layers for task-specific signal.

Topics

• QLoRA
• Coreference Resolution
• Multilingual NLP
• Large Language Models
• Fine-tuning
• Entity Resolution
• Qwen 3

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Closing the Gap at CRAC 2026: Two-Stage Adaptation for LLM-Based Multilingual Coreference Resolution — Two-stage fine-tuning and optimized input formats enable LLMs to achieve state-of-the-art multilingual coreference resolution.
• [P] QLoRA Fine-Tuning of Qwen2.5-1.5B for CEFR English Proficiency Classification (A1–C2) [P] — QLoRA fine-tuning of Qwen2.5-1.5B effectively classifies CEFR English proficiency, despite synthetic data limitations.
• Building an LLM fine-tuning Dataset — Building multi-speaker Reddit datasets for LLM fine-tuning is complex but achievable, with efficient QLoRA methods requiring minimal samples.
• Automated IEP Generation from Traditional Chinese Parent-Teacher Interviews via Corpus-Grounded Feature Diffusion — Corpus-Grounded Feature Diffusion enables high-performance, privacy-preserving automated IEP generation in Traditional Chinese despite low data resources.
• Multilingual Coreference Resolution via Cycle-Consistent Machine Translation — Coreference resolution in low-resource languages can be significantly improved by cycle-consistent machine translation data augmentation.
• QLoRA Explained: The Memory Compression Breakthrough — QLoRA enables fine-tuning large language models on consumer GPUs through efficient memory compression.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.