CuriousVectors@LT-EDI 2026: Detection of Homophobic and Transphobic Memes on Social Media Using a Hybrid Multimodal Approach

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

The paper "CuriousVectors@LT-EDI 2026" introduces a hybrid multimodal approach designed to detect homophobic and transphobic memes circulating on social media platforms. This method directly addresses the growing issue of abusive and harmful content, specifically targeting the LGBT+ community through visual and textual meme formats. The proposed technique integrates both image and text information processing. To mitigate the challenge of limited data within the LT-EDI 2026 challenge dataset, the models underwent pre-fine-tuning on a similar external dataset named PrideMM. This multimodal strategy achieved notable results in the shared task, securing the 8th rank for English language memes with a Macro F1-score of 0.24, and the 6th rank for Chinese language memes, yielding a Macro F1-score of 0.57.

Key takeaway

For NLP Engineers and AI Scientists developing hate speech detection systems for social media, you should prioritize multimodal approaches, especially when dealing with meme content. Your models will benefit significantly from pre-fine-tuning on similar, larger datasets like PrideMM to address data scarcity challenges. This strategy can improve detection accuracy for nuanced content like homophobic and transphobic memes, enabling more effective content moderation.

Key insights

Hybrid multimodal AI detects homophobic/transphobic memes by pre-fine-tuning on similar datasets to overcome data scarcity.

Principles

• Multimodal analysis is crucial for meme content.
• Pre-fine-tuning on related datasets improves performance.
• Limited data requires strategic model initialization.

Method

A hybrid multimodal technique processes image and text data. Models are pre-fine-tuned on the PrideMM dataset before training on the LT-EDI 2026 challenge meme dataset.

In practice

• Use multimodal models for meme analysis.
• Pre-fine-tune on external datasets like PrideMM.
• Combine image and text features for detection.

Topics

• Homophobic Memes
• Transphobic Memes
• Multimodal AI
• Hate Speech Detection
• Social Media Content Moderation
• LT-EDI 2026 Challenge
• PrideMM Dataset

Best for: Research Scientist, AI Scientist, NLP Engineer, Computer Vision Engineer

Related on AIssential

• I Know What You Meme, Even If it Emerged Today: Understanding Evolving Memes through Open-World Knowledge Acquisition — Meme understanding requires dynamic, open-world knowledge acquisition beyond static model parameters.
• Simulating Hate Speech Cascades with Multi-LLM Agents: Empirical Grounding, Modeling Fidelity, and Intervention Strategies — Multi-LLM agents can faithfully simulate implicit hate speech cascades by conditioning on user profiles, community, and content.
• Memes-as-Replies: Can Models Select Humorous Manga Panel Responses? — LLMs show promise in understanding social cues for humor, but struggle with visual information and subtle wit in meme reply selection.
• Evaluation of Conversational Agents: Understanding Culture, Context and Environment in Emotion Detection — Accurate and ethical emotion detection in conversational AI requires models that account for cultural and geographical context, as demonstrated by a new model.
• CATCH-ME if you RAG: a dataset of Contextually Annotated multi-Turn Counterspeech against Hate and Misinformation Exchanges — CATCH-ME provides a multilingual, multi-turn, knowledge-grounded dataset for countering intertwined hate speech and misinformation.
• Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes — Reinforcement learning with CoT supervision significantly improves MLLM performance and explainability for hateful meme detection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.