The 1.6 Billion People Large Language Models Still Can’t Understand

2026-06-20 · Source: NLP on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Intermediate, quick

Summary

Large language models (LLMs) like GPT-5 exhibit significant limitations in understanding and reasoning in many non-English languages, impacting approximately 1.6 billion people globally. The author, while developing EverestQ, observed that these models not only perform poorly but often become "confidently wrong" when processing queries in languages such as Bhojpuri, spoken by over 50 million people, Nepali, and Maithili. This issue was discovered during a debugging session that evolved into extensive research, revealing that the problem extends beyond simply a lack of training data. The article highlights that while data scarcity is a factor, it represents a symptom rather than the root cause of LLMs' inability to reliably function in a substantial portion of the world's languages.

Key takeaway

For NLP Engineers and AI Scientists developing global LLM applications, recognize that current models exhibit critical failures in non-English languages, particularly those with fewer resources. Your evaluation metrics must extend beyond standard benchmarks to identify "confidently wrong" outputs, not just reduced performance. This necessitates a deeper investigation into linguistic biases and data representation beyond simple data scarcity, ensuring your models are genuinely effective for the 1.6 billion people currently underserved.

Key insights

Large language models fail profoundly in many non-English languages, going beyond mere performance degradation to confident incorrectness.

Principles

• LLM multilingual failure is deeper than just missing data.
• Confident incorrectness is a distinct failure mode in low-resource languages.

Topics

• Large Language Models
• Multilingual AI
• Low-Resource Languages
• Linguistic Bias
• Model Evaluation
• GPT-5

Best for: Research Scientist, AI Engineer, Machine Learning Engineer, AI Scientist, NLP Engineer, AI Ethicist

Related on AIssential

• Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research — LLMs offer transformative potential for low-resource language research in humanities, despite significant data and ethical challenges.
• Human Evaluation of Large Language Models: How Good is Hugging Face’s BLOOM? — Human evaluation is crucial for assessing large language models' real-world performance beyond flawed academic benchmarks.
• MADE: Beyond Scoring via a Multilingual Agentic Diagnosing Engine for Fine-Grained Evaluation Insights — Multilingual LLM evaluation requires fine-grained, agentic diagnosis to transform scores into actionable insights, addressing cultural and deployment realities.
• Rethinking Cross-lingual Gaps from a Statistical Viewpoint — LLM cross-lingual gaps stem from response variance, not knowledge barriers, and are reducible via variance control.
• Study: AI chatbots provide less-accurate information to vulnerable users — AI chatbots underperform for vulnerable users, providing less accurate information and exhibiting biased behaviors.
• Linguistic Symptoms: Augmented Generation by Retrieval and Reasoning in LLMs under Portuguese-English Variation in Medical Contexts — Linguistic variation significantly impacts smaller LLMs in medical contexts, with larger models showing greater cross-language robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NLP on Medium.