SigJBS@LT-EDI 2026: QLoRA-Tuned Homophobic and Transphobic Counter Narrative Generation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

The SigJBS team's approach to the LT-EDI@ACL 2026 challenge focuses on generating counter-narratives for homophobic and transphobic comments, particularly in multilingual and low-resource environments. Their strategy involved fine-tuning the Gemma 3 12B model using QLoRA, incorporating techniques like data rebalancing and alternative input methods for Tamil. The research found that task-specific fine-tuning with native-script Tamil inputs yielded more stable and higher-quality outputs compared to large few-shot prompts or transliteration-based inputs. On the official leaderboard, their system achieved second place in English with an overall score of 86.35% and sixth place in Tamil with 63.77%. These results underscore the effectiveness of targeted fine-tuning for this sensitive task, while also highlighting the persistent difficulties in low-resource counter-narrative generation.

Key takeaway

For NLP Engineers developing hate speech counter-narrative systems, you should prioritize task-specific QLoRA fine-tuning on models like Gemma 3 12B. This approach, especially when using native-script inputs for low-resource languages, significantly improves output quality and stability compared to few-shot prompting or transliteration. Consider implementing data rebalancing to address dataset imbalances, which is crucial for achieving competitive performance in challenging multilingual environments.

Key insights

QLoRA-tuned Gemma 3 12B with native-script input excels at generating counter-narratives in low-resource settings, outperforming few-shot methods.

Principles

• Task-specific fine-tuning improves output quality.
• Native-script input enhances model stability.
• Data rebalancing addresses imbalance issues.

Method

Fine-tune Gemma 3 12B with QLoRA, employing data rebalancing and native-script inputs for low-resource languages like Tamil, to generate counter-narratives.

In practice

• Apply QLoRA fine-tuning for specific NLP tasks.
• Prioritize native-script data for low-resource languages.
• Rebalance datasets to mitigate performance issues.

Topics

• Counter-Narrative Generation
• QLoRA Fine-tuning
• Gemma 3 12B
• Low-Resource NLP
• Hate Speech Detection
• Multilingual NLP

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• Assisted Counterspeech Writing at the Crossroads of Hate Speech and Misinformation — LLMs can assist counterspeech against co-occurring hate and misinformation, but expert refinement and mixed knowledge strategies are crucial.
• KIT's Submission to Cross-Lingual Voice Cloning in IWSLT 2026 — Explicit language tags and lexical matching significantly improve cross-lingual voice cloning performance and reduce accent leakage.
• Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian — QLoRA fine-tuning significantly boosts cross-lingual relation extraction performance for low-resource languages, outperforming prompt-only LLMs.
• CoCoGEC: Counterfactual Generation for Robust Grammatical Error Correction — Training GEC models with context-altered counterfactuals significantly enhances their robustness against contextual perturbations.
• A Multitask Transformer for Offensive Language Detection and Target Identification in HateBR — Multitask learning improves offensive language detection consistency but may reduce fine-grained target attribution.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.