Compositional Meaning Representations in LLMs: a Critical Review of Probing Studies

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A critical review of 24 probing studies examines how Large Language Models (LLMs) represent compositional meaning. The review proposes a taxonomy of probing tasks, categorizing them into four tiers based on linguistic primitives: lexical semantics, the syntax–semantics interface, propositional semantics, and discourse and pragmatics. Findings indicate LLMs robustly encode lexical information but show less consistent sensitivity to structural relations within sentences. Performance on tasks requiring propositional content, speech acts, or pragmatic inference is unsatisfactory. The analysis emphasizes the need for a clearer theoretical grounding of what probing tasks truly measure and how they can illuminate the compositional pathways within current language models.

Key takeaway

For NLP Engineers designing or evaluating Large Language Models, you should recognize that current LLMs exhibit significant weaknesses in propositional content, speech acts, and pragmatic inference. Your evaluation strategies must extend beyond lexical and syntactic understanding to truly assess compositional capabilities. Consider adopting a multi-tiered probing approach, as outlined, to identify specific representational gaps and guide future model improvements.

Key insights

LLMs struggle with higher-level compositional semantics despite robust lexical encoding, highlighting probing limitations.

Principles

• LLMs robustly encode lexical information.
• Probing tasks require clearer theoretical grounding.
• LLM representational evidence shows a gradient of capabilities.

Method

A taxonomy of probing tasks is proposed, distinguishing four tiers: lexical semantics, syntax–semantics interface, propositional semantics, and discourse and pragmatics, based on linguistic primitives.

In practice

• Evaluate LLMs using a tiered probing taxonomy.
• Focus LLM development on propositional and pragmatic inference.

Topics

• Large Language Models
• Compositional Semantics
• Probing Studies
• Lexical Semantics
• Propositional Semantics
• Pragmatic Inference

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Rhetorical Questions in LLM Representations: A Linear Probing Study — LLMs represent rhetorical questions heterogeneously, with multiple linear directions capturing distinct rhetorical cues.
• Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processing — LLMs struggle with deep metaphor comprehension, often misinterpreting concepts and relying on lexical associations over context or full syntactic understanding.
• Experiments or Outcomes? Probing Scientific Feasibility in Large Language Models — Outcome evidence is more reliable for LLM scientific feasibility assessment than experiment descriptions alone.
• My Deep Dive into Large Language Models: An Architectural Journey — The shift to LLMs redefines NLP through scalable Transformer architectures, enabling versatile language tasks and requiring advanced customization.
• Hybrid Human-LLM Corpus Construction and LLM Evaluation for the Caused-Motion Construction — LLMs struggle with complex linguistic constructions like CMC, necessitating novel corpus creation methods.
• Impact of enriched meaning representations for language generation in dialogue tasks: A comprehensive exploration of the relevance of tasks, corpora and metrics — Enriched Meaning Representations improve NLG quality, especially for complex tasks and small, variable datasets.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.