IReL_IIT(BHU)@LTEDI 2026: Fine-Tuning Instruction-Tuned Transformers for Gender-Inclusive Rewriting and Counterfactual Bias Mitigation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing, Responsible AI · Depth: Expert, quick

Summary

IReL_IIT(BHU)'s submissions to the LT-EDI@ACL 2026 Shared Task on Gender Inclusive Language Generation utilized fine-tuned instruction-tuned encoder-decoder models for controlled text rewriting. The research addressed two independent subtasks: Gender-Inclusive Language Generation (Subtask A) and Counterfactual Generation (Subtask B). For Subtask A, focusing on the English dataset, the team achieved an average score of 43.7917, evaluated on rubrics including Gender Assumption, Gender Neutrality, and Quality Relevance. In Subtask B, which involved generating counterfactual sentences, an average score of 82.6241 was obtained, based on Politeness and Respectful, Contextual Counter-Narrative Coherence, and Quality and Relevance metrics. The experiments demonstrated that full fine-tuning of instruction-tuned transformers effectively produces gender-neutral sentences and counterfactuals for biased ones, especially when each subtask is optimized with its own specific training data.

Key takeaway

For NLP Engineers developing bias mitigation strategies, you should consider full fine-tuning of instruction-tuned transformers. This approach effectively generates gender-neutral language and counterfactuals, as demonstrated by scores of 43.7917 and 82.6241 respectively in the LT-EDI@ACL 2026 task. Optimizing models independently for each specific subtask, such as gender-inclusive rewriting or counterfactual generation, can significantly enhance performance in controlled text rewriting applications.

Key insights

Full fine-tuning instruction-tuned transformers effectively generates gender-neutral and counterfactual text when optimized per subtask.

Principles

• Independent subtask optimization improves results.
• Controlled text rewriting reduces gender bias.
• Instruction-tuned transformers are effective for bias mitigation.

Method

Fine-tuning instruction-tuned encoder-decoder models independently for each subtask (Gender-Inclusive Language Generation and Counterfactual Generation) on respective training datasets.

In practice

• Apply full fine-tuning for specific bias mitigation tasks.
• Optimize models on subtask-specific datasets.

Topics

• Gender Bias Mitigation
• Instruction-Tuned Transformers
• Counterfactual Generation
• Fine-Tuning
• Text Rewriting

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing — Gender bias from LLM writing assistants transfers to human-produced text, particularly suppressing agency in female-targeted writing.
• Greater accessibility can amplify discrimination in generative AI — Voice interfaces for LLMs introduce new discrimination pathways via paralinguistic cues, amplifying biases beyond text.
• MORPHOGEN: A Multilingual Benchmark for Evaluating Gender-Aware Morphological Generation — MORPHOGEN evaluates LLM gender-aware morphological generation across French, Arabic, and Hindi, revealing performance gaps and biases.
• Short-form Text Rewriting with Phi Silica — Targeted adaptation of Small Language Models (SLMs) can significantly improve their performance for precision-critical short-form text rewriting tasks.
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction — Fine-tuned AI text detectors amplify a pretrained typicality axis, not construct a new AI-vs-human boundary.
• A Systematic Analysis of Biases in Large Language Models — LLMs, despite alignment efforts, inherit and perpetuate diverse biases from human-generated training data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.