A Multitask Transformer for Offensive Language Detection and Target Identification in HateBR

2026-04-12 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

A Multitask Learning (MTL) approach, employing a shared BERTimbau encoder, has been developed for offensive language detection and target identification within the HateBR dataset. This MTL architecture simultaneously predicts binary offensiveness, ordinal severity, and specific hate speech targets, moving beyond traditional binary classification. Experiments show the MTL model improves the Matthews Correlation Coefficient for offensive detection from 0.80 to 0.82 compared to Single-Task baselines. Furthermore, joint training ensures hierarchical consistency, achieving a 0% target-inconsistency rate where no comment is labeled "Non-offensive" yet assigned a hate target. However, the model exhibits negative transfer in the fine-grained multilabel target task, with Micro-F1 dropping from 0.59 to 0.42, indicating a trade-off between logical consistency and target attribution, especially under extreme class imbalance.

Key takeaway

For research scientists developing hate speech detection systems, you should consider implementing Multitask Learning (MTL) with shared encoders to enhance overall offensive language detection performance and ensure hierarchical consistency. Be aware that while MTL improves primary detection, you may observe reduced performance on fine-grained, multilabel target identification tasks, especially with imbalanced datasets. Evaluate the specific needs of your application to balance consistency with detailed target attribution.

Key insights

Multitask learning improves offensive language detection consistency but may reduce fine-grained target attribution.

Principles

• Toxicity is hierarchical, not binary.
• Joint training can enforce logical consistency.

Method

A Multitask Learning (MTL) approach uses a shared BERTimbau encoder to predict binary offensiveness, ordinal severity, and hate speech targets simultaneously on the HateBR dataset.

In practice

• Use MTL for improved offensive detection.
• Consider trade-offs for fine-grained target tasks.

Topics

• Multitask Learning
• Offensive Language Detection
• Target Identification
• BERTimbau
• HateBR Dataset

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Multi-Stage Training for Abusive Comment Detection in Indic Languages — Multi-stage training with language preprocessing and model ensembles improves abusive comment detection while minimizing false positives.
• Comparison of Modern Multilingual Text Embedding Techniques for Hate Speech Detection Task — Supervised models with multilingual embeddings and gradient boosted trees excel at hate speech detection.
• Probing Multimodal Large Language Models on Cognitive Biases in Chinese Short-Video Misinformation — MLLMs struggle with short-video misinformation due to cognitive biases like authority and popularity effects, despite multimodal reasoning capabilities.
• Muted: Multilingual Targeted Offensive Speech Identification and Visualization — Muted identifies targeted offensive speech using transformer models and spaCy, requiring no fine-tuning for toxic span detection.
• Socially Responsible and Explainable Automated Fact-Checking and Hate Speech Detection — Explainable NLP methods and high-quality datasets can jointly optimize performance and interpretability for Portuguese hate speech and fact-checking.
• TeamHerald@CHIPSAL 2026: Hate Speech Detection and Sentiment Analysis of Nepali Memes using Transformer-based Architectures and Ensemble Learning — Ensemble strategies for meme analysis perform differently for binary vs. multi-class tasks, requiring task-specific aggregation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.