HCMUSDroneBoys at SemEval-2026 Task 11: Asymmetric Counterfactual Debiasing and Rank-Sensitive Logical Invariance Adaptation for Syllogistic Reasoning

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

The HCMUSDroneBoys system, developed for SemEval-2026 Task 11, Subtask 1, addresses the challenge of binary classification for syllogistic validity in English. Its primary goal is to overcome the "content effect," where language models mistakenly equate formal logical validity with an argument's semantic plausibility. The system integrates three distinct techniques: Structure-Disentangled Prompting (SDP) for breaking down syllogisms into premise-conclusion triples with a logic-first template; Asymmetric Counterfactual Debiasing (ACD), a data augmentation method that generates only valid-to-invalid counterfactual pairs to prevent label noise; and Rank-Sensitive Logical Invariance Adaptation (RLIA), which increases QLoRA adapter rank to enable simultaneous learning of classification and suppression of content-correlated shortcuts. Built upon Qwen2.5-14B-Instruct, this approach achieved a perfect Combined Score of 100.0 on the SemEval-2026 Task 11 Subtask 1 benchmark.

Key takeaway

For Machine Learning Engineers developing language models for formal reasoning tasks, this work highlights a critical approach to mitigate the "content effect." You should consider integrating Structure-Disentangled Prompting, Asymmetric Counterfactual Debiasing, and Rank-Sensitive Logical Invariance Adaptation. These techniques, demonstrated on Qwen2.5-14B-Instruct, can significantly improve your model's ability to distinguish logical validity from semantic plausibility, potentially achieving perfect scores on similar benchmarks. Adjusting QLoRA adapter rank is crucial for complex logical learning.

Key insights

A novel system combines prompting, data augmentation, and adapter rank adjustment to achieve perfect syllogistic validity classification.

Principles

• Language models struggle with logical validity vs. semantic plausibility.
• Asymmetric data augmentation can reduce label noise.
• Increase adapter rank for complex, multi-objective learning.

Method

Syllogisms are broken into premise-conclusion triples with logic-first prompts. Data augmentation generates valid-to-invalid counterfactuals. QLoRA adapter rank is increased to learn classification and suppress content-correlated shortcuts.

In practice

• Implement logic-first prompting for formal reasoning.
• Employ asymmetric counterfactuals for data augmentation.
• Optimize QLoRA adapter rank for logical invariance.

Topics

• Syllogistic Reasoning
• Language Models
• Counterfactual Debiasing
• QLoRA Adapters
• Prompt Engineering
• SemEval-2026

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• Counterfactual Reasoning for Fine-Grained Evidence Disentanglement in VideoQA — Counterfactual reasoning can explicitly disentangle causal visual evidence from confounders in VideoQA for more reliable systems.
• MODF-SIR: A Multi-agent Omni-modal Distilled Framework for Social Intelligence Reasoning — A multi-agent, omni-modal framework uses distillation and TTA with LoRA for superior social intelligence reasoning.
• FregeLogic at SemEval 2026 Task 11: A Hybrid Neuro-Symbolic Architecture for Content-Robust Syllogistic Validity Prediction — Hybrid neuro-symbolic systems can enhance syllogistic validity prediction by mitigating content bias.
• Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization — Preference alignment, specifically DPO, effectively balances validity and minimality in multilingual counterfactual explanation generation for LLMs.
• On Asymmetric Optimization of Reasoning and Perception in Vision-Language Model Post-Training — VLM post-training creates a perception-reasoning asymmetry due to token imbalance (SFT) or reward coupling (RL), hindering end-to-end performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.