Deer, Deities, and Dancing: Culturally Biased LLM Hallucination in Low-Resource Wixárika Translation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

A study by Henry Gagnier and Ashwin Kirubakaran, presented at AmericasNLP 2026, reveals that large language models struggle significantly with low-resource polysynthetic languages like Wixárika. Evaluating GPT-4o-mini, Gemma~3~27B, Llama~3.3~70B, and NLLB-200 on Spanish$\leftrightarrow$Wixárika translation, all models scored below 3 BLEU and 21 chrF, rendering them unusable. Qualitative analysis showed LLMs frequently ignore source content, instead generating fluent hallucinations. Spanish translations often included indigenous cultural stereotypes such as deer, deities, rain dance, and shamans, irrespective of input. Conversely, Wixárika outputs were repetitive and morphologically implausible. While few-shot prompting improved Gemma and Llama substantially, GPT-4o-mini showed no benefit. These findings highlight LLMs' inability to handle polysynthetic morphology and their tendency to exoticize Indigenous culture.

Key takeaway

For NLP Engineers developing translation systems for low-resource Indigenous languages, you must recognize that current LLMs like GPT-4o-mini and Llama~3.3~70B are fundamentally inadequate. Your focus should shift from general-purpose LLMs to specialized, morphological-aware modeling strategies. Prioritize investing in increased resource creation for these languages. This prevents perpetuating cultural stereotypes and ensures accurate, safe translation outputs.

Key insights

LLMs fail low-resource polysynthetic language translation, exhibiting culturally biased hallucinations and morphological implausibility.

Principles

• LLMs exoticize Indigenous culture.
• Polysynthetic morphology challenges LLMs.
• Few-shot prompting offers model-dependent gains.

Method

The study evaluated LLMs on Spanish$\leftrightarrow$Wixárika translation using zero-shot and 5-shot prompting, assessing performance via BLEU, chrF, and qualitative analysis of output errors.

In practice

• Prioritize morphological-aware modeling.
• Increase Indigenous language resource creation.
• Qualitatively analyze LLM outputs for bias.

Topics

• LLM Hallucination
• Wixárika Language
• Low-Resource NLP
• Polysynthetic Languages
• Cultural Stereotypes
• Machine Translation Bias

Best for: Research Scientist, AI Scientist, NLP Engineer, AI Ethicist

Related on AIssential

• A multilingual hallucination benchmark: MultiWikiQHalluA — Multilingual hallucination benchmarks reveal higher rates in smaller models and lower-resource languages.
• Bridging Cultural Gaps in Automated Translation of Brazilian Expressions: A Study on Cultural Adaptation — Automated translation systems struggle with cultural adaptation, even with advanced LLMs, requiring human oversight.
• IndoBias: A Dual Track Culturally Grounded Benchmark for LLMs Bias Evaluation in Indonesian Languages — Culturally-grounded benchmarks are crucial for evaluating LLM bias in diverse, multilingual contexts like Indonesia.
• Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research — LLMs offer transformative potential for low-resource language research in humanities, despite significant data and ethical challenges.
• When Similar Means Different: Evaluating LLMs on Arabic--Hebrew Cognates — Current LLMs struggle with cross-lingual semantic disambiguation, especially false friends and loanwords, due to over-reliance on surface forms.
• A Comparison of Methods to Bias Translation Toward Portuguese Variants — Biasing MT towards minority language variants is achievable through various lifecycle interventions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.