Culturally-Aware Image Captioning for Guaraní with Multimodal Prompting: IUHoosiers at AmericasNLP 2026

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

The IUHoosiers system from Indiana University secured first place in the AmericasNLP 2026 shared task for Guaraní image captioning. This system generates culturally grounded captions for severely underresourced indigenous languages, demanding both cultural awareness and linguistic accuracy. Instead of fine-tuning, IUHoosiers uses inference-time knowledge injection. It retrieves relevant Guaraní grammatical and cultural resources via BM25. These resources are then injected into a large vision language model's prompt alongside the image. This enables language-specific grounding without any parameter updates. The system outperformed all other participants, achieving 24.67 chrF++ in automatic evaluation and 3.45/5 in human evaluation.

Key takeaway

For NLP Engineers developing culturally-aware systems or optimizing model deployment, IUHoosiers offers a powerful alternative to fine-tuning. You should explore inference-time knowledge injection. Retrieve relevant cultural and linguistic resources using BM25, then integrate them directly into your vision language model's prompts. This method achieved first place for Guaraní. It provides strong performance without costly parameter updates, offering a scalable path for diverse language support.

Key insights

Inference-time knowledge injection via multimodal prompting effectively grounds image captions in underresourced indigenous languages.

Principles

• Inference-time knowledge injection grounds LLMs.
• BM25 effectively retrieves cultural resources.

Method

Retrieve Guaraní grammatical and cultural resources using BM25, then inject them into a large vision language model's prompt alongside the image for language-specific grounding.

In practice

• Use BM25 for context retrieval in low-resource NLP.
• Inject external knowledge into prompts for cultural grounding.

Topics

• Image Captioning
• Guaraní Language
• Multimodal Prompting
• Low-Resource NLP
• Knowledge Injection
• BM25

Best for: Research Scientist, AI Scientist, NLP Engineer, Prompt Engineer

Related on AIssential

• Annotation Challenges in Low-Resource African Languages — High-quality NLP for African languages demands culturally-attuned annotation, robust QA, and fair, community-centric practices.
• AFRILANGTUTOR: Advancing Language Tutoring and Culture Education in Low-Resource Languages with Large Language Models — Dictionary-based synthetic data generation and combined SFT+DPO significantly enhance LLM tutoring for low-resource African languages.
• Scripts Through Time: A Survey of the Evolving Role of Transliteration in NLP — Transliteration bridges the "script barrier" in NLP, enhancing cross-lingual transfer and efficiency, especially for low-resource languages.
• Māori Text-to-Speech Model Spurns Big Tech’s Values — Indigenous communities can develop high-fidelity AI language models prioritizing data sovereignty and community ownership, offering an alternative to big tech.
• AfriSUD: A Dependency Treebank Collection for Evaluating Models on African Languages — Models exhibit a significant "syntax gap" in processing African languages, struggling with their structural diversity.
• Multilingual OCR-Aware Fine-Tuning and Prompt-Guided Chain-of-Thought Reasoning for Multimodal Large Language Models — OCR-aware data curation and fine-tuning significantly improve MLLM robustness for multilingual text in degraded images.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.