LLM-INSTRUCT at UZH Shared Task 2026: Constraint-Aware Retrieval and Selective Debate for Paragraph-Level Argument Mining

2026-04-16 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing, Software Development & Engineering · Depth: Expert, extended

Summary

The LLM-Instruct system won the UZH Shared Task at ArgMining 2026 for paragraph-level argument mining in UN and UNESCO resolutions. This task required classifying paragraph types, predicting a subset of 141 official tags, and identifying directed relations under a strict JSON schema, using only open-weight models up to 8B parameters. LLM-Instruct framed this as constrained structured prediction, employing metadata-aware dense retrieval to narrow candidate tag space, constrained decoding with per-dimension caps, and a three-agent debate for uncertain cases, followed by schema validation. It achieved 1st overall rank, including 1st in F1 and 5th in LLM-as-a-Judge. Development improved Task 1b Micro-F1 from 35.83% to 40.08%. The core finding is that reducing the decision space before generation significantly enhances both accuracy and submission robustness.

Key takeaway

For Machine Learning Engineers designing LLM-based systems for structured prediction with strict output constraints, prioritize explicit control layers. Implement pre-generation steps like metadata-aware retrieval to narrow the decision space. Apply constrained decoding with output caps and integrate robust schema validation. Consider selective multi-agent debate for high-uncertainty cases; this significantly improves accuracy and submission robustness.

Key insights

Constraining LLM output space via retrieval and validation significantly improves structured prediction accuracy and robustness.

Principles

• Reduce decision space before generation.
• Metadata-aware prototypes enhance tag retrieval.
• Admissibility rules improve output reliability.

Method

Classify paragraph type heuristically, then use metadata-aware dense retrieval to narrow tag candidates. Apply constrained decoding with caps and project to retrieved sets. Escalate uncertain cases to a three-agent debate, then validate output schema.

In practice

• Use dense retrieval with metadata-aware prototypes.
• Implement global and per-dimension output caps.
• Employ multi-agent debate for high-uncertainty cases.

Topics

• Argument Mining
• Constrained Decoding
• Large Language Models
• Dense Retrieval
• Multi-agent Debate
• Structured Prediction

Code references

• LLM-Instruct-at-UZH-Shared-Task-2026/Method

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• LLM-INSTRUCT at UZH Shared Task 2026: Constraint-Aware Retrieval and Selective Debate for Paragraph-Level Argument Mining — The LLM-INSTRUCT system combines retrieval, constrained decoding, and multi-agent debate for accurate, constrained argument mining.
• CAF-Gen: A Multi-Agent System for Enriching Argumentation Structures — Multi-agent LLM systems with iterative Creator-Reviewer pipelines reliably enrich shallow argument structures into complex formal models.
• Argchestrators at UZH Shared Task 2026: Efficient Argument Mining in UN Resolutions: A Sub-8B Pipeline using Agentic Debate and Heuristic Retrieval — Efficient argument mining in UN resolutions is achievable with a sub-8B parameter hybrid LLM pipeline using deterministic rules and graph extraction.
• From Argument Components to Graphs: A Multi-Agent Debate with Confidence Gating for Argument Relations — Multi-agent debate with confidence gating improves training-free argument relation identification and offers interpretability.
• The State of Information Retrieval in 2026 — Information retrieval has evolved into a reasoning problem, integrating LLMs and agentic search for complex, context-aware query processing.
• Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space — DLCMs improve LLM efficiency by shifting computation from tokens to a compressed, concept-based reasoning space.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.