Paradise at SemEval-2026 Task 12: Leveraging Instruction-Tuned Large Language Models with Chain-of-Thought Prompting for Abductive Event Reasoning

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Paradise, a system developed for SemEval-2026 Task 12: Abductive Event Reasoning, identifies plausible direct causes for real-world English events using retrieved contextual documents. The system employs Qwen2.5-7B-Instruct, a 7-billion-parameter instruction-tuned language model, combined with carefully engineered chain-of-thought prompting. Notably, Paradise requires no task-specific fine-tuning or training-data supervision, with prompt components selected using a development set. It achieved a score of 0.79 on the official 612-instance test set by integrating explicit causal-inference rules, 4,000-character document context windows, and greedy decoding. Analysis revealed that conservative prediction patterns, specifically 87.1% single-label and 36.9% Option D, effectively exploited the asymmetric scoring metric. Ablation studies confirmed significant contributions from document context (+6.4 points), chain-of-thought reasoning (+5.3 points), and explicit causal rules (+3.1 points) to development performance.

Key takeaway

For NLP Engineers developing abductive reasoning systems, this work demonstrates that instruction-tuned LLMs like Qwen2.5-7B-Instruct, when combined with sophisticated chain-of-thought prompting and explicit causal rules, can achieve strong performance (0.79 F1) without task-specific fine-tuning. You should prioritize prompt engineering and consider integrating external knowledge sources and large context windows to enhance your model's reasoning capabilities and potentially exploit scoring metrics.

Key insights

Paradise leverages instruction-tuned LLMs with CoT prompting and causal rules for abductive event reasoning, achieving 0.79 on SemEval-2026 Task 12.

Principles

• Instruction-tuned LLMs excel with engineered prompting.
• Chain-of-thought improves complex reasoning tasks.
• Explicit causal rules enhance abductive reasoning.

Method

Paradise integrates Qwen2.5-7B-Instruct with chain-of-thought prompting, 4,000-character document context, explicit causal-inference rules, and greedy decoding for abductive event reasoning.

In practice

• Apply CoT prompting to instruction-tuned LLMs.
• Incorporate external causal rules for reasoning tasks.
• Use large context windows for document-based reasoning.

Topics

• Abductive Event Reasoning
• Instruction-Tuned LLMs
• Chain-of-Thought Prompting
• Qwen2.5-7B-Instruct
• Causal Inference
• Prompt Engineering

Code references

• DhruvGoyal404/semeval2026-task12

Best for: Research Scientist, AI Scientist, NLP Engineer, Prompt Engineer

Related on AIssential

• AILS-NTUA at SemEval-2026 Task 12: Graph-Based Retrieval and Reflective Prompting for Abductive Event Reasoning — AILS-NTUA's three-stage system won SemEval 2026 Task 12 for Abductive Event Reasoning, achieving a first-place rank with an accuracy score of 0.95.
• Why We Think — Allocating more test-time compute via methods like Chain-of-Thought significantly boosts LLM reasoning and problem-solving capabilities.
• ttda704 at SemEval-2026 Task 6: Structured Chain-of-Thought Prompting for Political Evasion Detection — Structured Chain-of-Thought prompting significantly improves political evasion detection over fine-tuning large language models.
• Narration-of-Thought: Inference-Time Scaffolding for Defeasible Ethical Reasoning in Large Language Models — Narration-of-Thought (NoT) enhances LLM ethical reasoning by structuring chain-of-thought to explicitly address stakeholders and uncertainties.
• Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structures — LLMs' apparent faithfulness to intermediate reasoning structures is fragile; they often fail to update predictions after causal intervention.
• JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew — A synthetic-organic pipeline personalizes LLMs for individual judges, outperforming baselines in low-resource settings.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.