Team_One@DravidianLangTech 2026: A Gated Multimodal Architecture for Multi-Level Stance and Target Detection in Malayalam Political Memes

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

A novel gated multimodal architecture is proposed for multi-level stance and target detection in Malayalam political memes, addressing challenges in low-resource and highly imbalanced datasets. This framework, developed for the DravidianLangTech 2026 Shared Task, processes a Malayalam dataset of 500 samples with a significant 95.4:4.6 stance imbalance. It integrates bilingual OCR, a Vision Transformer (ViT), and IndicBERT to learn complementary visual and textual features. A gated fusion mechanism effectively combines these multimodal features, while asymmetric loss weighting and post-training threshold optimization are employed to mitigate extreme class imbalance. The methodology achieved a Weighted F1-score of 0.9535 for stance detection and 0.5283 for target identification, demonstrating robust performance under realistic multimodal constraints.

Key takeaway

For NLP Engineers developing multimodal classification systems, especially with low-resource languages or imbalanced data, consider adopting a gated multimodal architecture. You should integrate specialized components like bilingual OCR, Vision Transformers, and IndicBERT for robust feature extraction. Implement asymmetric loss weighting and post-training threshold optimization to effectively manage extreme class imbalances, improving both stance and target detection accuracy in challenging real-world scenarios.

Key insights

A gated multimodal architecture effectively detects stance and targets in imbalanced, low-resource Malayalam political memes.

Principles

• Multimodal fusion enhances complex content analysis.
• Asymmetric loss addresses extreme class imbalance.
• Gated mechanisms optimize feature integration.

Method

The proposed method integrates bilingual OCR, Vision Transformer (ViT), and IndicBERT for visual and textual representations. It uses a gated fusion mechanism, asymmetric loss weighting, and post-training threshold optimization.

In practice

• Apply bilingual OCR for low-resource text.
• Combine ViT and IndicBERT for multimodal input.
• Use asymmetric loss for imbalanced datasets.

Topics

• Multimodal AI
• Stance Detection
• Target Detection
• Malayalam NLP
• Political Memes
• Class Imbalance

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• TeamHerald@CHIPSAL 2026: Hate Speech Detection and Sentiment Analysis of Nepali Memes using Transformer-based Architectures and Ensemble Learning — Ensemble strategies for meme analysis perform differently for binary vs. multi-class tasks, requiring task-specific aggregation.
• IHUBERT: Vector-Based Semantic Deduplication and Domain-Balanced Pretraining for Persian Resources — Vector-based semantic deduplication and domain-balanced pretraining significantly enhance Persian language model performance across diverse NLU tasks.
• Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection — AI-text detectability is non-monotonic, influenced by edit operations and revision history, not just AI content proportion.
• Zero-shot Tweet-Level Stance Detection Enhanced by External Knowledge and Reflective Chain-of-Thought Reasoning — KIRP improves zero-shot tweet stance detection by integrating external knowledge, entity reorganization, and reflective Chain-of-Thought reasoning.
• KITE: A Tri-Modal Transformer Integrating Text, Images, and Knowledge Graphs for Fake News Detection — KITE integrates text, images, and knowledge graphs via a tri-modal transformer for enhanced fake news detection and interpretability.
• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.