Andha-Dhun: A First Look at Audio Descriptions in Hindi

2026-04-06 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

The "Andha-Dhun" project presents the first systematic study of Audio Descriptions (ADs) in Hindi, addressing a critical gap for India's Blind and Low Vision (BLV) audiences following mandates from the Central Board of Film Certification. Researchers introduce the Andha-Dhun dataset, comprising 5,870 human-authored Hindi AD sentences from 8 full-length movies. They investigate two automatic generation approaches: directly creating Hindi ADs from English dense video descriptions using Hindi-capable LLMs like Nemotron-4-Mini-Hindi-4B-Instruct, and translating English ADs into Hindi via models such as IndicTrans2. Evaluation using perplexity and LLM-as-a-judge metrics reveals that human-authored ADs exhibit greater linguistic diversity. While the direct generation approach with Nemotron-4-Mini-Hindi-4B-Instruct performs best among automatic methods, all current systems fall significantly short of human quality, particularly in adapting culture-specific items, where human ADs resolve 42.5% of CSIs compared to machine translation's 10%.

Key takeaway

For NLP Engineers developing audio description systems for Indian languages, you must prioritize cultural adaptation over literal translation. Your models should be trained or fine-tuned to resolve culture-specific items, as direct machine translation of English ADs significantly underperforms human-authored quality in this regard. Consider employing Hindi-capable LLMs for direct generation from dense descriptions, as this approach shows more promise than translating existing English ADs.

Key insights

Effective Hindi Audio Descriptions prioritize cultural adaptation over literal translation for Indian Blind and Low Vision audiences.

Principles

• AD translation must align with target audience's cultural context.
• Direct Hindi AD generation can surpass English AD translation.
• Linguistic diversity is a hallmark of high-quality ADs.

Method

Generate Hindi ADs by distilling MLLM-produced English dense descriptions with a Hindi-capable LLM, or by translating English ADs. Evaluate using perplexity and LLM-as-a-judge metrics.

In practice

• Employ Hindi-capable LLMs for direct AD generation.
• Adapt cultural references in ADs for target audience.
• Avoid direct machine translation of English ADs.

Topics

• Audio Description
• Hindi Language Processing
• Cultural Adaptation
• LLM-based Generation
• Machine Translation Evaluation
• Andha-Dhun Dataset

Code references

• katha-ai/AndhaDhun-HindiAD

Best for: Research Scientist, AI Scientist, NLP Engineer, AI Ethicist

Related on AIssential

• Bridging Cultural Gaps in Automated Translation of Brazilian Expressions: A Study on Cultural Adaptation — Automated translation systems struggle with cultural adaptation, even with advanced LLMs, requiring human oversight.
• Cascaded Modular or End-to-End? : An Investigation on Speech-to-Speech Translation Task for Dravidian Languages — Cascaded Modular systems consistently outperform End-to-end models for low-resource Dravidian speech-to-speech translation.
• UrduMMLU: A Massive Multitask Benchmark for Urdu Language Understanding — Native, culturally-grounded benchmarks reveal significant LLM knowledge gaps in non-English, humanities-focused content.
• From Sinhala to Dhivehi: Cross-Lingual Transfer Learning for Low-Resource Speech Recognition — Cross-lingual transfer from linguistically related languages significantly improves low-resource ASR, but adaptation strategy and decoding configuration are equally critical.
• A Dataset for Probing Translationese Preferences in English-to-Swedish Translation — Language models often prefer literal "translationese" over idiomatic phrasing, especially when exposed to the source language.
• AI-assisted cultural heritage dissemination: Comparing NMT and glossary-augmented LLM translation in rock art documents — Glossary-augmented LLMs significantly improve terminology accuracy in specialized translation without sacrificing overall quality.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.