UFAL-CUNI at SemEval-2026 Task 11: An Efficient Modular Neuro-symbolic Method for Syllogistic Reasoning

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

UFAL-CUNI submitted an efficient modular neuro-symbolic system to SemEval-2026 Task 11, focused on disentangling content and formal reasoning in large language models. This approach combines a symbolic prover with small reasoning LLMs, specifically 4B parameters. The system integrates an LLM-based parser that translates natural language syllogisms into a first-order logic (FOL) representation, an automated theorem prover, and optional modules for machine translation and symbolic premise retrieval. It achieved competitive accuracy and a relatively low content effect across most subtasks. Ablation studies demonstrated that this neuro-symbolic method surpasses LLM-based zero-shot baselines within its parameter size range, although it revealed limitations in the multilingual capabilities of smaller LLMs. The paper also discusses the task's primary ranking metric and its inherent limitations.

Key takeaway

For NLP Engineers developing reasoning systems, this work suggests that combining small LLMs (e.g., 4B parameters) with symbolic provers can significantly improve performance over zero-shot baselines. You should consider modular neuro-symbolic architectures, particularly an LLM-based parser for natural language to first-order logic translation, to enhance formal reasoning and reduce content effects. This approach offers a path to competitive accuracy without relying solely on massive models.

Key insights

The UFAL-CUNI system combines neuro-symbolic methods with small LLMs for efficient syllogistic reasoning, outperforming zero-shot baselines.

Principles

• Neuro-symbolic integration enhances reasoning.
• Small LLMs benefit from symbolic parsing.
• Content effect can be reduced modularly.

Method

Translate natural language syllogisms to FOL via an LLM-based parser, then use an automated theorem prover, optionally adding machine translation or symbolic premise retrieval.

In practice

• Implement LLM-based NL-to-FOL parsers.
• Combine small LLMs with symbolic provers.
• Evaluate content effect in reasoning tasks.

Topics

• Neuro-symbolic AI
• Syllogistic Reasoning
• Large Language Models
• First-Order Logic
• SemEval-2026
• Automated Theorem Proving

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Absolute Truths in AI: Neuro-Symbolic AI — LLMs excel at hypothesis generation, but external symbolic solvers are essential for deductive logical truth.
• FregeLogic at SemEval 2026 Task 11: A Hybrid Neuro-Symbolic Architecture for Content-Robust Syllogistic Validity Prediction — Hybrid neuro-symbolic systems can enhance syllogistic validity prediction by mitigating content bias.
• Forget LLM: MIT's New RLM (Phase Shift in AI) — MIT's Recursive Language Model (RLM) uses a neurosymbolic approach to overcome LLM context rot and enhance reasoning.
• Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery — AI for mathematical reasoning is a rapidly evolving frontier, integrating diverse techniques for verified discovery.
• From Natural Language to Executable Narsese: A Neuro-Symbolic Benchmark and Pipeline for Reasoning with NARS — Neuro-symbolic systems can enhance LLM reasoning by translating natural language into executable symbolic structures for validation.
• Grounding vs. Compositionality: On the Non-Complementarity of Reasoning in Neuro-Symbolic Systems — Symbol grounding alone is insufficient for compositional generalization; explicit reasoning objectives are required.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.