ModusPonens at SemEval-2026 Task 11: Breaking the Plausibility Trap in LLMs via Conflict-Aware Ensembling

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

ModusPonens' submission to SemEval-2026 Task 11 addresses the "belief bias" in Large Language Models (LLMs), where formal logical validity is confused with real-world plausibility. The team found that standard neuro-symbolic interventions like Structural Chain-of-Thought (CoT) and Nonsense Augmentation degraded performance in low-resource settings due to an "abstraction penalty." Their proposed solution is a Conflict-Aware Logit Ensemble, which fine-tunes two Qwen-2.5-14B models: a standard "Believer" and a bias-hardened "Skeptic" trained on oversampled conflict data. Ensembling their logits via soft-voting achieved a Total Content Effect (TCE) of 3.21, an overall accuracy of 94.27%, and a Combined Score of 39.09, demonstrating a Pareto-optimal balance.

Key takeaway

For AI Scientists and Machine Learning Engineers tackling logical reasoning tasks, be aware that standard neuro-symbolic interventions might degrade performance in low-resource settings. Instead, consider implementing a Conflict-Aware Logit Ensemble by fine-tuning distinct "Believer" and "Skeptic" models. This approach can achieve a Pareto-optimal balance between reducing belief bias and maintaining high accuracy in your LLM applications.

Key insights

Ensembling specialized LLMs with conflict-aware logits mitigates belief bias.

Principles

• LLMs often struggle to separate logical validity from real-world plausibility.
• Standard neuro-symbolic interventions can degrade performance in low-resource regimes.
• Conflict-aware ensembling balances heuristic and logical reasoning.

Method

Fine-tune a standard "Believer" LLM and a bias-hardened "Skeptic" LLM on oversampled conflict data, then ensemble their logits via soft-voting.

In practice

• Fine-tune Qwen-2.5-14B for specific reasoning roles.
• Oversample conflict data to harden models against bias.
• Implement soft-voting for logit ensembling.

Topics

• Large Language Models
• Belief Bias
• Neuro-symbolic AI
• Ensemble Methods
• Qwen-2.5-14B
• SemEval-2026 Task 11

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference — MACR resolves LLM knowledge conflicts by explicitly assessing confidence in sources and employing multi-agent reasoning, moving beyond binary reliability assumptions.
• LLMs believe false statements even after explicit warnings that they're false — LLMs prioritize statistical patterns in training data over explicit negation, leading to "belief implantation."
• Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning — CoT-PoT ensembling improves LLM reasoning accuracy and efficiency by leveraging diverse, complementary reasoning modalities.
• Human Mind Reconstruction w/ AI (ToM, UserHarness) — Decoupling epistemic structure from LLM generation significantly boosts reasoning accuracy and efficiency, even for smaller models.
• Identifying High-Confidence Social Biases in LLMs for Trustworthy Conversational Tutoring Agents — LLMs in tutoring agents exhibit overconfident social biases, challenging detection in conversational contexts.
• The malleable mind: context accumulation drives LLM’s belief drift — LLMs exhibit systematic belief drift from accumulated context, impacting reliability and potentially diverging stated beliefs from behavior.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.