YNU-NLP at SemEval-2026 Task 11: A Neuro-Symbolic Approach with Reflexion Mechanism Disentangling Content and Formal Reasoning in Language Models

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

YNU-NLP's system for SemEval-2026 Task 11 addresses the "Content Effect" in language models, where models prioritize real-world plausibility over logical validity. The team developed a hybrid neuro-symbolic framework built upon the Qwen2.5-14B architecture, integrating multi-task instruction tuning with a robust neuro-symbolic pipeline. A key innovation is the Reflexion mechanism, which couples with formal verification by parsing natural language arguments into First-Order Logic (FOL) for validation by the Z3 Theorem Prover. The system includes an iterative self-correction module to rectify parsing anomalies automatically. This approach successfully disentangles content from formal reasoning, achieving 1st place in Subtasks 1 and 2, 2nd in Subtask 4, and 9th in Subtask 3, demonstrating its effectiveness in improving logical validity.

Key takeaway

For NLP Engineers focused on improving language model logical reasoning, this neuro-symbolic approach offers a robust solution to the "Content Effect." You should consider integrating formal verification using First-Order Logic and theorem provers like Z3 into your pipelines. This method, combined with a Reflexion mechanism for self-correction, can significantly enhance your models' ability to decouple logical validity from real-world plausibility, as demonstrated by its top rankings in SemEval-2026 Task 11.

Key insights

A neuro-symbolic framework with Reflexion and FOL verification effectively disentangles content from formal reasoning in LMs.

Principles

• Probabilistic biases in LMs hinder content-reasoning decoupling.
• Hybrid neuro-symbolic approaches enhance logical validity.
• Iterative self-correction improves formal parsing accuracy.

Method

A hybrid neuro-symbolic framework based on Qwen2.5-14B integrates multi-task instruction tuning, parses natural language to FOL, and uses Z3 Theorem Prover with Reflexion for verification and self-correction.

In practice

• Apply FOL parsing and Z3 for logical validity checks.
• Integrate Reflexion for iterative self-correction in parsing.
• Combine multi-task tuning with neuro-symbolic pipelines.

Topics

• Neuro-Symbolic AI
• Reflexion Mechanism
• First-Order Logic
• Z3 Theorem Prover
• Language Model Reasoning
• SemEval-2026

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• AXIOM: A Trust-First Neuro-Symbolic Execution Architecture for Verifiable Mathematical Reasoning — AXIOM uses an LM to canonicalize math problems for a CAS, ensuring verifiable, high-trust reasoning with a dynamic improvement cycle.
• NeRD: Neuro-Symbolic Rule Distillation for Efficient Ontology-Grounded Chain-of-Thought in Medical Image Diagnosis — NeRD distills neuro-symbolic rules for efficient, ontology-grounded, and clinically plausible reasoning in medical image diagnosis, avoiding manual rule crafting.
• Neurosymbolic Learning for Inference-Time Argumentation — ITA uses neurosymbolic learning and formal argumentation to provide faithful, explainable, ternary claim verification.
• Logical Phase Transitions: Understanding Collapse in LLM Logical Reasoning — LLM logical reasoning collapses abruptly at critical complexity thresholds, mirroring physical phase transitions.
• System 3 AI: No Humans Needed — System 3 AI integrates formal logic and verifiable code into LLMs for autonomous, complex reasoning beyond human feedback.
• FOL2NS: Generating Natural Sentences from First-Order Logic — FOL2NS is a neurosymbolic framework translating complex first-order logic into natural language, addressing data scarcity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.