HABIBTAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective Optimization

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, medium

Summary

The HABIBTAZ system for SemEval-2026 Task 11 addresses the challenge of disentangling formal logic from content, a task where Large Language Models (LLMs) often show bias towards real-world plausibility. This system utilizes mDeBERTa-v3 networks, fine-tuned on a synthetic, rule-based dataset of syllogistic schemes to bypass the semantic noise typically found in LLM-augmented data. Its training pipeline employs a multi-objective loss function, combining Adaptive Group Distributionally Robust Optimization (DRO), a scheduled differentiable bias penalty, and KL-Divergence consistency regularization to explicitly separate plausibility from logical structure. The system achieved #1 ranks and perfect Ranking Scores (100.0) with 0.00% bias and 100.0% accuracy on Subtask 1 (English), Subtask 2 (Noisy English), and Subtask 3 (Multilingual). For the complex Subtask 4 (Noisy Multilingual), it secured the 6th rank with 89.06% Accuracy and F1-score, a limited 2.89% Bias, and a 37.78 Ranking Score. The dataset generation engine and codebase are publicly available.

Key takeaway

For NLP engineers developing robust reasoning systems, you should consider synthetic, rule-based datasets to mitigate content effects and real-world plausibility bias in LLMs. Implement multi-objective loss functions, including Adaptive Group DRO and bias penalties, to explicitly decouple logical structure from content. This approach, demonstrated by HABIBTAZ's top SemEval-2026 Task 11 performance, offers a path to significantly improve formal reasoning accuracy and reduce bias across multiple languages.

Key insights

Synthetic training and multi-objective optimization can effectively disentangle formal logic from content in LLMs, reducing real-world plausibility bias.

Principles

• Content effects compromise LLM formal reasoning.
• Synthetic data avoids semantic noise in training.
• Multi-objective loss decouples plausibility from logic.

Method

The system fine-tunes mDeBERTa-v3 networks on a synthetic, rule-based syllogistic dataset. It uses a multi-objective loss function combining Adaptive Group DRO, a scheduled differentiable bias penalty, and KL-Divergence consistency regularization.

In practice

• Use synthetic data for logic tasks.
• Implement multi-objective loss for bias reduction.
• Leverage mDeBERTa-v3 for multilingual reasoning.

Topics

• Formal Logic Reasoning
• Large Language Models
• Synthetic Data Training
• Multi-Objective Optimization
• mDeBERTa-v3
• Semantic Evaluation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Synthetic Contrastive Reasoning for Multi-Table Q&A — Synthetic contrastive reasoning traces significantly improve multi-table Q&A performance in LLMs.
• Harmonizing Multi-Objective LLM Unlearning via Unified Domain Representation and Bidirectional Logit Distillation — Multi-objective LLM unlearning requires harmonizing data representation and employing bidirectional distillation for balanced efficacy and robustness.
• Efficient Financial Language Understanding via Distillation with Synthetic Data — Distillation with synthetic data, guided by clustering real examples, enables efficient financial NLP in low-resource settings.
• Designing a Synthetic Data Pipeline for Persian LLM Fine Tuning — Data quality and engineering are paramount for LLM performance, especially in low-resource languages.
• Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification — Merging LLM-extracted logical patterns with linguistic cues significantly improves automated fallacy classification and generalization.
• JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew — A synthetic-organic pipeline personalizes LLMs for individual judges, outperforming baselines in low-resource settings.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.