STEMTOX: From Collaborative Tags to Fine-Grained Toxic Meme Detection via Entropy-Guided Multi-Task Learning

2026-07-23 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

STEMTOX introduces a novel dataset of 6,300 real-world meme-based posts and a generative AI-based tag generation framework to enhance toxic meme detection. The dataset features a two-stage human annotation process, classifying memes as toxic/normal, then further into hateful, dangerous, or offensive, achieving Fleiss' κ scores of 0.8176 and 0.8197 respectively. It is enriched with auxiliary metadata, including socially relevant tags. The proposed tag generation module, leveraging Google Search API and Google Lens, automatically produces context-aware tags for untagged memes. Experimental results demonstrate that incorporating these contextual tags significantly boosts the performance of state-of-the-art Vision Language Models (VLMs) in toxicity detection tasks. Specifically, combining image and predicted tag similarity (I₋im⊕pt) consistently outperformed other few-shot selection strategies, establishing a scalable foundation for improved multimodal content moderation.

Key takeaway

For content moderation teams developing or refining toxic meme detection systems, you should prioritize integrating contextual tag information. Traditional VLM approaches often underperform due to memes' nuanced nature; however, incorporating AI-generated or human-annotated tags significantly boosts accuracy. Consider implementing a tag generation module and leveraging combined image and tag embeddings for few-shot learning to build more robust and context-aware moderation pipelines.

Key insights

Contextual tags, whether human-annotated or AI-generated, significantly improve Vision Language Model performance in toxic meme detection.

Principles

• Multi-stage annotation improves quality for subjective content tasks.
• Socially grounded tags enhance VLM understanding of meme context.
• Combining visual and tag embeddings optimizes few-shot learning.

Method

A two-step generative AI framework first creates an intermediate meme summary using metadata and Google Lens context, then extracts keywords from it as tags.

In practice

• Use Google Lens and Search API to enrich meme context for VLMs.
• Employ a two-stage annotation pipeline for fine-grained content labeling.
• Combine image and predicted tag embeddings for superior few-shot VLM performance.

Topics

• Toxic Meme Detection
• Multimodal Content Moderation
• Vision Language Models
• Dataset Annotation
• Tag Generation
• Few-shot Learning

Code references

• huggingface/peft

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer, AI Ethicist

Related on AIssential

• Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memes — MAR-12 uses multi-perspective VLM analysis to detect and explain harmful humor in complex memes.
• LilyMeme@EEUCA 2026: Multimodal Vaccine Meme Stance Detection with Task-Adapted MemeCLIP and Complementary Ensembling — Task-adapted MemeCLIP with noise-aware training and ensembling significantly improves vaccine meme stance detection.
• CuriousVectors@LT-EDI 2026: Detection of Homophobic and Transphobic Memes on Social Media Using a Hybrid Multimodal Approach — Hybrid multimodal AI detects homophobic/transphobic memes by pre-fine-tuning on similar datasets to overcome data scarcity.
• Holy $#!t: Are popular toxicity models simply profanity detectors? — Toxicity models often misclassify positive profanity due to inadequate training data and contextual understanding.
• EVL-MCoT: Enhanced Vision-Language Multi-CoT for Harmful Meme Detection — EVL-MCoT uses multi-perspective Chain-of-Thought and fine-grained vision-language fusion for robust harmful meme detection.
• Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection — LLM annotation at scale matches human aggregate performance at lower cost, but with distinct error profiles.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.