JerinWarriors@DravidianLangTech 2026: A Two-Stream Cross-Attention Approach for Prompt Recovery in Telugu

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

JerinWarriors@DravidianLangTech 2026 introduced a Two-Stream Cross-Attention architecture for the Telugu Prompt-Style Recovery Shared Task at DravidianLangTech @ ACL 2026. This novel approach addresses the difficulty of identifying detailed sentence structures and annotation cues in Telugu, a low-resource and morphologically rich language where traditional Multi Layer Perceptrons (MLP) perform poorly. The architecture incorporates a shared MuRIL encoder to analyze the relationship between an original transcript and its style-shifted version, thereby improving the MLP's capacity to differentiate styles and detect subtle variations. The proposed model effectively manages the signal dilution of individual labels. It secured 2nd place among 13 teams, achieving a Macro F1-score of 0.2588 on the test set. Researchers determined that local transformation is the primary factor for successful style recovery in this task, and they have released their implementation on GitHub.

Key takeaway

For NLP Engineers developing solutions for low-resource languages like Telugu, particularly in prompt recovery or style transfer tasks, you should investigate Two-Stream Cross-Attention architectures. This approach, which uses a shared MuRIL encoder, significantly enhances style distinction and handles signal dilution better than standard MLPs. Your team can improve model performance by focusing on local transformations, as they are key drivers for effective style recovery. The released implementation offers a valuable starting point for your research.

Key insights

A Two-Stream Cross-Attention architecture improves prompt recovery in low-resource Telugu by using a shared MuRIL encoder.

Principles

• Low-resource, morphologically rich languages challenge standard MLPs.
• Local transformation drives style recovery in prompt tasks.
• Cross-attention with shared encoders enhances style distinction.

Method

A Two-Stream Cross-Attention architecture uses a shared MuRIL encoder to relate original and style-shifted transcripts, aiding an MLP in distinguishing styles and recovering prompts in Telugu.

In practice

• Apply cross-attention for low-resource NLP.
• Utilize MuRIL encoders for language style analysis.
• Focus on local transformations for prompt recovery.

Topics

• Two-Stream Cross-Attention
• Prompt Recovery
• Telugu NLP
• Low-Resource Languages
• MuRIL Encoder
• Dravidian Languages

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Evaluating Cross-lingual Knowledge Consistency in Code-Mixed vis-a-vis Indian Languages using IndicKLAR — Code-mixing inputs significantly improves large language model knowledge recall in Indian languages, nearly matching English performance.
• Hybrid Diffusion Transformer for Instruction-Guided Audio Editing via Rectified Flow — The Hybrid Diffusion Transformer uses a two-stage coarse-to-fine strategy for efficient, accurate instruction-guided audio editing.
• Bytes Speak All Languages: Cross-Script Name Retrieval via Contrastive Learning — A byte-level transformer encoder effectively solves cross-script phonetic name retrieval by learning universal byte sequence mappings.
• DREAM: Extending Vision-Language Models with Dual-Objective Encoding for Cross-Modal Retrieval — Dual-objective encoding and hierarchical attention significantly enhance cross-modal video retrieval performance.
• Scripts Through Time: A Survey of the Evolving Role of Transliteration in NLP — Transliteration bridges the "script barrier" in NLP, enhancing cross-lingual transfer and efficiency, especially for low-resource languages.
• Neural Speaker Diarization via Multilingual Training: Evaluation on Low-Resource Nepali-Hindi Speech — Multilingual training with Perceiver-based attractors significantly improves speaker diarization for low-resource languages like Nepali-Hindi.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.