RvH-40 at SemEval-2026 Task 11: Disentangling Reasoning from Belief through Symbolic Abstraction

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

RvH-40, a submission for SemEval-2026 Task 11, addresses the "belief bias" in Large Language Models (LLMs) that hinders syllogistic reasoning. This bias causes semantic world knowledge to override formal logical structures. The research investigates the gap between an LLM's inherent logical capabilities and its performance on natural language. By applying symbolic transformations, specifically variable and pseudoword substitution, the study demonstrates that models like Qwen2.5-14B possess strong latent reasoning skills. A "logic alignment" strategy, utilizing Low-Rank Adaptation (LoRA), was proposed to bridge this performance gap. The final model achieved a 97.92% accuracy on the validation set and 96.34% on the official hidden test set, successfully eliminating content bias and maintaining robust generalization across abstract formats.

Key takeaway

For Machine Learning Engineers developing LLM applications requiring precise logical inference, this research highlights a critical method to overcome "belief bias." Your models, like Qwen2.5-14B, likely possess stronger inherent reasoning than their natural language performance suggests. Consider implementing symbolic abstraction techniques, such as variable and pseudoword substitution, combined with LoRA-based "logic alignment" to significantly improve syllogistic reasoning accuracy and generalization, as demonstrated by the 96.34% test set accuracy.

Key insights

LLMs possess strong latent reasoning skills suppressed by linguistic content, which symbolic abstraction and LoRA can reveal.

Principles

• Belief bias hinders LLM syllogistic reasoning.
• Symbolic abstraction reveals latent logic.
• Content bias can be eliminated.

Method

The approach involves symbolic transformations, specifically variable and pseudoword substitution, followed by a "logic alignment" strategy using Low-Rank Adaptation (LoRA) to fine-tune LLMs.

In practice

• Apply pseudoword substitution for logic tasks.
• Use LoRA for "logic alignment."
• Test LLMs with abstract reasoning formats.

Topics

• Large Language Models
• Syllogistic Reasoning
• Belief Bias
• Symbolic Abstraction
• Low-Rank Adaptation
• SemEval-2026 Task 11

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• BREAKING: LLM “reasoning” continues to be deeply flawed — LLMs, even those marketed for reasoning, continue to exhibit fundamental and widespread reasoning failures.
• LoRi: Low-Rank Distillation for Implicit Reasoning — Hidden-state reasoning trajectories exhibit low-rank structure, enabling efficient distillation for implicit reasoning.
• LLMs believe false statements even after explicit warnings that they're false — LLMs prioritize statistical patterns in training data over explicit negation, leading to "belief implantation."
• eMoT: evolving Memory-of-Thought via Symbolic Anchoring and Memory Corrosion — eMoT stabilizes LLM multi-step reasoning by evolving dynamic memory, symbolic anchoring, and consistency refinement.
• More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models — Longer reasoning trajectories in LLMs increase position bias, a critical factor for evaluation reliability.
• Human Mind Reconstruction w/ AI (ToM, UserHarness) — Decoupling epistemic structure from LLM generation significantly boosts reasoning accuracy and efficiency, even for smaller models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.