The American Palimpsest: Quantifying South Asian English Dialect Erasure in LLMs

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

Research quantifies South Asian English (SAsE) dialect erasure in Large Language Models, specifically using Llama 3.3 70B. A 500-sentence diagnostic benchmark, comprising 320 lexical and 180 syntactic markers, revealed significant suppression. Standard grammar correction retained only 26.0% of SAsE markers (31.2% lexical; 16.7% syntactic), while formalization was more destructive, retaining just 14.0%. Lexical Americanization occurred in 56.2% of correction cases and 59.4% of formalization cases. Implementing a simple dialect-aware prompt dramatically increased retention to 92.0% and reduced lexical Americanization to 6.2%, though some function-word phenomena remained resistant. A stress test showed even stronger suppression, with only 6.7% retention. The study frames dialect erasure within representational-harm and cultural-competence frameworks, offering a replicable auditing protocol.

Key takeaway

For NLP Engineers developing writing assistants for global users, standard grammar correction and formalization in LLMs like Llama 3.3 70B significantly erase South Asian English markers, leading to Americanization. You should implement dialect-aware prompting to drastically improve dialect retention (up to 92.0%) and reduce Americanization to 6.2%. Prioritize auditing your systems for representational harm and cultural competence to ensure equitable language support.

Key insights

LLMs, like Llama 3.3 70B, erase South Asian English dialects, but dialect-aware prompting significantly improves retention.

Principles

• LLM writing assistance can suppress postcolonial varieties.
• Dialect erasure aligns with representational harm.
• Auditing LLMs requires replicable protocols.

Method

Quantify dialect erasure using a 500-sentence diagnostic benchmark with lexical and syntactic markers, then test prompt-based mitigation.

In practice

• Implement dialect-aware prompts for SAsE users.
• Audit writing assistants for dialect retention.
• Develop benchmarks for other English varieties.

Topics

• Large Language Models
• South Asian English
• Dialect Erasure
• Prompt Engineering
• Cultural Competence
• Representational Harm

Best for: AI Engineer, Machine Learning Engineer, Research Scientist, AI Scientist, NLP Engineer, AI Ethicist

Related on AIssential

• Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs — Multilingual LLMs exhibit implicit US-centric and population-driven biases, exacerbated by instruction tuning.
• Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communities — Integrating culturally grounded narratives via RAG enhances LLM sensitivity to minority perspectives in content moderation.
• How I Improved Speech-to-Text Accuracy — A two-pass LLM post-processing method effectively corrects common speech-to-text transcription errors.
• ROMPAR: Morphological Completion and Demographic Unlearning for Romanian-Accented Speech Recognition — A multi-task adversarial framework with LLM-guided decoding improves ASR for accented speech by reducing demographic bias and completing morphology.
• Analysing LLMs for spelling normalization of 18th century Portuguese — Specialized LLMs significantly outperform multilingual models for historical language normalization.
• Can AI Save India’s Dying Languages? | Building a Tulu LLM Without Training Data Ft. Prathamesh — Structured prompting and custom romanization enable LLMs to generate low-resource languages without extensive training data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.