LLM-Adapted Colombian Spanish Lexicography: Proficiency Control, Hallucination, and Cultural Distortion

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Language Learning & Cultural Education · Depth: Expert, quick

Summary

An evaluation assessed the capability of three 7-8B instruction-tuned open-source LLMs—Llama 3.1, Qwen2.5, and Mistral—to generate proficiency-graded English adaptations for 8,252 entries from the Diccionario de colombianismos (DiCol). Using structured zero-shot prompts targeting Beginner, Intermediate, and Advanced CEFR bands, the study found significant issues. Automated metrics revealed that Intermediate-level targeting collapsed, with 73-83% of outputs classified as Advanced by vocabulary (χ² > 705, p < .001), and Advanced outputs expanded 4.9-8.2 times the source length. Expert annotation of a 360-entry sample (κ = 0.61-0.68) identified hallucination in 19% of entries, predominantly in the Advanced condition (81%, χ² = 86.6, p < .001). This hallucination, associated with higher expansion (U = 16,662, p < .001, r = 0.68), manifested as generic elaboration, Colombia-stereotyping, and pragmatic polarity inversion, a pattern termed "algorithmic domestication".

Key takeaway

For NLP Engineers developing localized language learning resources, you should exercise caution when relying on open-source LLMs for proficiency-graded content. Your systems must incorporate robust expert review and automated checks for output expansion, as models like Llama 3.1, Qwen2.5, and Mistral can introduce significant hallucination and cultural distortion, especially at advanced proficiency levels. Prioritize human oversight to maintain accuracy and cultural fidelity.

Key insights

LLMs struggle with nuanced proficiency grading and cultural preservation in lexicographic adaptation, exhibiting "algorithmic domestication."

Principles

• LLM output expansion correlates with increased hallucination.
• Proficiency-graded translation requires precise control beyond current LLM capabilities.
• Cultural context is vulnerable to distortion in LLM adaptations.

Method

Structured zero-shot prompts with 7-8B instruction-tuned LLMs generated CEFR-banded translations, followed by automated vocabulary classification and expert annotation for hallucination and cultural preservation.

In practice

• Validate LLM-generated proficiency-graded content with expert review.
• Monitor output expansion as an indicator of potential hallucination.
• Implement specific cultural preservation checks for localized content.

Topics

• LLM Lexicography
• Proficiency Grading
• Cultural Distortion
• Hallucination Detection
• Zero-Shot Prompting
• Colombian Spanish

Best for: Research Scientist, AI Scientist, NLP Engineer, AI Student

Related on AIssential

• The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs — LLMs often access local cultural knowledge better in local languages, despite lower raw accuracy due to proficiency gaps.
• AI-assisted cultural heritage dissemination: Comparing NMT and glossary-augmented LLM translation in rock art documents — Glossary-augmented LLMs significantly improve terminology accuracy in specialized translation without sacrificing overall quality.
• Bridging Cultural Gaps in Automated Translation of Brazilian Expressions: A Study on Cultural Adaptation — Automated translation systems struggle with cultural adaptation, even with advanced LLMs, requiring human oversight.
• Linguistic Issues in Computational Linguistics for Non-English Languages — English-centric LLM design creates systematic linguistic weaknesses for non-English languages across multiple levels.
• Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning — Multilingual code-switching instruction tuning improves LLM performance beyond bilingual settings.
• Textual Inference in Portuguese: Comparing Language Models — LLMs struggle with logic-sensitive semantic reasoning in Portuguese, highlighting the need for specialized benchmarks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.