Investigating Syntactic Biases in Multilingual Transformers with RC Attachment Ambiguities in Italian and English

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

An investigation explored whether monolingual and multilingual Large Language Models (LLMs) exhibit human-like preferences when processing relative clause attachment ambiguities in Italian and English. The study also examined if these preferences could be influenced by lexical factors, specifically the type of verb or noun in the matrix clause, which are known to affect syntactic and semantic relations. The findings reveal that LLM behavior varies inconsistently across different models and languages. This research underscores the critical importance of utilizing subtle syntactic contrasts to thoroughly evaluate these models' ability to accurately align with human linguistic preferences.

Key takeaway

For NLP engineers evaluating or fine-tuning Large Language Models for complex linguistic tasks, you should recognize that LLM syntactic preferences can be inconsistent across languages and models. Your evaluation strategies must incorporate subtle syntactic contrasts, like relative clause attachment ambiguities and lexical factor modulation, to accurately assess human-like linguistic alignment and build more robust systems.

Key insights

LLMs exhibit inconsistent human-like syntactic preferences, underscoring the need for nuanced linguistic evaluation.

Principles

• LLM behavior varies inconsistently across models and languages.
• Subtle syntactic contrasts are crucial for evaluating human-like alignment.

Method

Investigating monolingual and multilingual LLMs' preferences for relative clause attachment ambiguities, modulated by lexical factors (verb/noun type).

In practice

• Use relative clause ambiguities to test LLM syntactic understanding.
• Incorporate lexical factors to modulate syntactic preference tests.

Topics

• Multilingual Transformers
• Syntactic Biases
• Relative Clause Attachment
• Large Language Models
• Italian Language
• English Language
• Lexical Factors

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Cross-Lingual Response Consistency in Large Language Models: An ILR-Informed Evaluation of Claude Across Six Languages — ILR-informed expert judgment reveals interpretable, domain-dependent cross-lingual variation in LLM outputs.
• Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processing — LLMs struggle with deep metaphor comprehension, often misinterpreting concepts and relying on lexical associations over context or full syntactic understanding.
• Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints? — LLMs show shared, item-sensitive filler-gap dependency mechanisms, but require more data than humans.
• Differences in Typological Alignment in Language Models' Treatment of Differential Argument Marking — LMs show human-like markedness preferences but not object preference in differential argument marking.
• Extending the Contact Hypothesis: Cross-Linguistic Evaluation of Religion and Nationality Bias When Prompting LLMs in German and Icelandic — LLMs exhibit religion and nationality biases, requiring cross-linguistic evaluation for fairness.
• Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency — LLMs can self-align cultural knowledge across languages by identifying consistent responses and transferring them to weaker language contexts.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.