YNU-HPCC at SemEval-2026 Task 11: Mitigating Content Effects in Syllogistic Reasoning with Qwen2-1.5B-Instruct and XLM-RoBERTa-Large for English and Multilingual TasksMultilingual Tasks

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

YNU-HPCC's system for SemEval-2026 Task 11 addresses the challenge of mitigating content effects in syllogistic reasoning, where large language models often confuse logical validity with semantic plausibility. The team proposed a novel rule- and template-based symbolic data augmentation framework, which generates logic-preserving synthetic data through lexical rules. This framework was used to fine-tune the Qwen2-1.5B-Instruct model and instruction-tune the XLM-RoBERTa-large model. Their approach achieved significant success, ranking 1st in Task 1 with a perfect overall score of 100 and 6th in Task 3 with a score of 56.97. The code for their system is publicly available.

Key takeaway

For NLP engineers developing reasoning systems, you should consider integrating symbolic data augmentation to explicitly control for content effects. This method, demonstrated by YNU-HPCC's success at SemEval-2026 Task 11, can significantly improve the logical validity of models like Qwen2-1.5B-Instruct and XLM-RoBERTa-large, preventing them from conflating logical correctness with semantic plausibility in complex tasks.

Key insights

Symbolic data augmentation effectively mitigates content effects in LLM syllogistic reasoning, improving logical validity.

Principles

• LLMs conflate logical validity with semantic plausibility.
• Explicit bias control is crucial beyond standard fine-tuning.

Method

A rule- and template-based symbolic data augmentation framework generates logic-preserving synthetic data for fine-tuning Qwen2-1.5B-Instruct and instruction-tuning XLM-RoBERTa-large.

In practice

• Fine-tune Qwen2-1.5B-Instruct with augmented data.
• Instruction-tune XLM-RoBERTa-large for multilingual tasks.
• Utilize public code for bias mitigation in reasoning.

Topics

• Syllogistic Reasoning
• Content Effects
• Data Augmentation
• Qwen2-1.5B-Instruct
• XLM-RoBERTa-large
• Bias Mitigation

Code references

• YNU-HPCC/semeval-2026-task11

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning — RA-RFT teaches LMs analogical reasoning by training a retriever on reasoning benefit and fine-tuning with retrieved demonstrations.
• Qwen-3.5 PLUS Outperforms MoE 397B-A17B — Qwen 3.5 Plus demonstrates strong causal reasoning but can generate invalid optimizations; the base Qwen 3.5 struggles with complex validation.
• Reasoning Quality Emerges Early: Data Curation for Reasoning Models — Reasoning quality in LLMs can be effectively improved by curating SFT data based on early token loss patterns, significantly boosting efficiency.
• FregeLogic at SemEval 2026 Task 11: A Hybrid Neuro-Symbolic Architecture for Content-Robust Syllogistic Validity Prediction — Hybrid neuro-symbolic systems can enhance syllogistic validity prediction by mitigating content bias.
• The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Mathematical Reasoning — High-quality Oracle data can impair SLM mathematical reasoning due to distributional drift, despite high reward scores.
• Enhancing Science Classroom Discourse Analysis through Joint Multi-Task Learning for Reasoning-Component Classification — Automated discourse analysis using LLM augmentation improves utterance classification and reveals effective teaching patterns.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.