Can AI Save India’s Dying Languages? | Building a Tulu LLM Without Training Data Ft. Prathamesh

2026-03-13 · Source: AIM Network · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, long

Summary

Pratamesh, an AI researcher, developed a novel method for generating grammatically correct Tulu language using Large Language Models (LLMs) without extensive training data. Tulu, an ancient Dravidian language spoken by approximately 2 million people predominantly in coastal Southern India, is currently diminishing. Existing LLMs struggle with low-resource languages like Tulu due to statistical dominance of high-resource languages in training data and "vocabulary contamination" where similar root words with languages like Kannada lead to incorrect outputs. Pratamesh's solution employs a structured prompting approach, utilizing a less than 2,800-token prompt across five layers: identity, negative constraints, grammar documentation, few-shot examples, and self-verification. This method also incorporates a custom romanization scheme, reducing tokenization from 3.2 to 1.4 tokens per word, which improves tokenization efficiency and helps the model distinguish Tulu from Kannada, thereby preserving language boundaries and enabling better application of grammar rules.

Key takeaway

For NLP Engineers working with low-resource languages, consider adopting structured prompting techniques instead of relying solely on data-intensive training or fine-tuning. Your approach should include defining an LLM's identity, providing negative constraints, documenting grammar rules, offering few-shot examples, and implementing self-verification. Additionally, explore custom romanization schemes to optimize tokenization efficiency and reduce linguistic contamination, which can significantly improve model performance and enable broader language support.

Key insights

Structured prompting and custom romanization enable LLMs to generate low-resource languages without extensive training data.

Principles

• Statistical dominance hinders low-resource language generation.
• Vocabulary contamination confuses LLMs with related languages.
• Tokenization efficiency correlates with contamination reduction.

Method

A five-layer structured prompt (identity, negative constraints, grammar, few-shot examples, self-verification) combined with custom romanization scheme allows LLMs to learn and generate low-resource languages effectively.

In practice

• Apply structured prompting for low-resource language tasks.
• Develop custom romanization for improved tokenization.
• Use negative examples to prevent vocabulary contamination.

Topics

• Low-Resource Languages
• Large Language Models
• Structured Prompting
• Tokenization Efficiency
• Language Preservation

Best for: NLP Engineer, AI Scientist, Research Scientist, AI Engineer, Machine Learning Engineer, AI Researcher

Related on AIssential

• TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages — African language prompting reduces LLM refusal, but reveals comprehension and judge reliability issues in low-resource contexts.
• Label-Aware Pseudo-Training Sample Generation for Text Classification — A new data augmentation method uses LLMs to subtly alter sentences, improving NLP performance with limited data.
• We Don’t Have an AI Problem. We Have a Context Problem. (And MCP Exposes It) — Unstructured context, not model limitations, is the primary bottleneck in scaling and maintaining LLM-based systems.
• Modular Monolingual Adaptation using Pretrained Language Models — Modular PLM adaptation, freezing embeddings while tuning the rest, improves low-resource language performance.
• Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research — LLMs offer transformative potential for low-resource language research in humanities, despite significant data and ethical challenges.
• Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification — In-context learning for Turkish LVCs is highly prompt-sensitive, with few-shot demonstrations improving LLM calibration to match supervised baselines.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AIM Network.