Thiyaga6851 at SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models using Neuro-Symbolic Mapping

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Thiyaga6851's system, submitted for SemEval-2026 Task 11 Subtask 1, evaluates the formal validity of English syllogisms while aiming to minimize semantic plausibility effects. This system employs a hybrid neuro-symbolic pipeline that distinctly separates natural-language abstraction from logical inference. It functions by mapping each syllogism into categorical propositions using a combination of template rules and a learned parser, followed by explicit role mapping for major, minor, and middle terms. For structurally complete abstractions, a Venn-style satisfiability solver determines validity; otherwise, instances are directed to a learned fallback classifier. The official submission achieved an accuracy of 71.73%, a Total Content Effect of 11.84, and a Combined Score of 20.19, placing 41st. Analysis revealed that symbolic inference performs reliably on well-formed abstractions, with most errors stemming from paraphrase, multiword terms, and unstable term alignment.

Key takeaway

For NLP Engineers developing systems for formal reasoning from natural language, you should consider a neuro-symbolic approach to disentangle content from logical structure. Your focus should be on robust natural language abstraction, specifically improving paraphrase and multiword term handling, as these are critical error sources. Ensure stable term alignment to maximize the reliability of symbolic inference components. This strategy can significantly enhance the formal validity assessment of complex linguistic inputs.

Key insights

A hybrid neuro-symbolic pipeline effectively separates natural language abstraction from logical inference for formal syllogism validity.

Principles

• Separate language abstraction from logic.
• Symbolic inference needs well-formed inputs.
• Explicit term mapping reduces content effects.

Method

A hybrid neuro-symbolic pipeline maps syllogisms to categorical propositions using template rules and a learned parser, followed by explicit term role mapping. Validity is checked by a Venn solver or a learned fallback classifier.

In practice

• Improve paraphrase handling.
• Address multiword term challenges.
• Stabilize term alignment processes.

Topics

• SemEval-2026
• Neuro-Symbolic AI
• Syllogistic Reasoning
• Formal Logic
• Natural Language Abstraction
• Large Language Models

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• FregeLogic at SemEval 2026 Task 11: A Hybrid Neuro-Symbolic Architecture for Content-Robust Syllogistic Validity Prediction — Hybrid neuro-symbolic systems can enhance syllogistic validity prediction by mitigating content bias.
• Neurosymbolic Repo-level Code Localization — Existing code localization benchmarks exhibit a "Keyword Shortcut" bias, hindering genuine structural reasoning in models.
• Robust Dual-Signal Fusion: Hybrid Neuro-Symbolic Gating with Compressed Chain-of-Thought Refinement for Irony Detection in Social Media Texts — Hybrid neuro-symbolic fusion with compressed Chain-of-Thought effectively overcomes LLM literalism for zero-shot irony detection.
• Forget LLM: MIT's New RLM (Phase Shift in AI) — MIT's Recursive Language Model (RLM) uses a neurosymbolic approach to overcome LLM context rot and enhance reasoning.
• Grounding vs. Compositionality: On the Non-Complementarity of Reasoning in Neuro-Symbolic Systems — Symbol grounding alone is insufficient for compositional generalization; explicit reasoning objectives are required.
• FOL2NS: Generating Natural Sentences from First-Order Logic — FOL2NS is a neurosymbolic framework translating complex first-order logic into natural language, addressing data scarcity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.