Can language models process linguistic deference?

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Advanced, quick

Summary

A study investigated language models' ability to process Korean subject honorifics, which are linguistic forms encoding respect based on social status. The research aimed to determine if these models capture the socio-pragmatic constraints governing honorific use or merely rely on surface co-occurrence patterns. Evaluating a set of language models, findings revealed a systematic dissociation: models successfully detected surface morphosyntactic mismatches, treating unacceptable honorific constructions as less expected. However, the models consistently favored overt honorific marking irrespective of the subject's social status. This suggests that current language models primarily rely on surface heuristics rather than genuine pragmatic knowledge, indicating they have not fully acquired the complex socio-pragmatic rules underlying honorific use, even after extensive training on Korean text.

Key takeaway

For NLP Engineers developing or deploying language models for languages with complex honorific systems like Korean, you should recognize that current models may lack genuine socio-pragmatic understanding. Your evaluation metrics must go beyond morphosyntactic correctness to assess true contextual awareness. Consider augmenting training data with explicit social context or developing specialized modules to encode deference rules, as models currently default to surface-level heuristics.

Key insights

Language models struggle with socio-pragmatic nuances of honorifics, relying on surface patterns over genuine understanding.

Principles

• LMs detect morphosyntactic mismatches in honorifics.
• LMs favor overt honorifics regardless of social status.
• Surface heuristics often override pragmatic knowledge.

Method

The study evaluated language models by assessing their processing of Korean subject honorifics, specifically testing their ability to capture socio-pragmatic constraints versus surface co-occurrence patterns.

In practice

• Test LMs for pragmatic understanding beyond syntax.
• Focus training on socio-pragmatic context.

Topics

• Language Models
• Korean Honorifics
• Socio-pragmatics
• NLP Evaluation
• Morphosyntax
• Linguistic Deference

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Disparities In Negation Understanding Across Languages In Vision-Language Models — VLMs show affirmation bias, with negation understanding varying significantly across diverse languages and models.
• Korean Culture into LLM Alignment: Toward Cultural Coherence — LLM cultural alignment requires defining culturally coherent responses, not just suppressing harmful ones.
• From Comments to Trends: Studi NLP dan Computational Linguistics dalam Mendeteksi Topik Viral dari… — NLP and computational linguistics are crucial for detecting social media trends from vast, informal, and diverse comment data.
• Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem — LLMs align with human judgments on the proviso problem but use shallow pattern matching, not deep pragmatic reasoning.
• A Multilingual Paradigm for AI Created by All, for All — Multilingual AI evaluations must be culturally nuanced, context-specific, and independently verified to ensure equitable global AI performance.
• Scripts Through Time: A Survey of the Evolving Role of Transliteration in NLP — Transliteration bridges the "script barrier" in NLP, enhancing cross-lingual transfer and efficiency, especially for low-resource languages.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.