Linus@EEUCA 2026: Multimodal and Text-Only Approaches to Vaccine-Critical Meme Detection.

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

Acharya, Saud, and Regmi participated in the EEUCA 2026 VaxMeme Shared Task, focusing on classifying Twitter-based vaccine memes into anti-vaccine, neutral, or pro-vaccine categories. They utilized the VaxMeme dataset, comprising 8,195 train, 1,024 validation, and 1,025 test samples. The team explored two primary architecture families: multimodal hybrids combining CLIP ViT-B/32 for images with BERT-based models like BERT-base-uncased for text, using a late fusion strategy; and text-only approaches employing various pre-trained models such as BERT-base-uncased, RoBERTa-base, and Deberta-v3-base. Both families incorporated a three-layer feed-forward network with GELU activation and were optimized using techniques like AdamW and OneCycleLR. Surprisingly, the text-only BERT-base-uncased model achieved the highest performance with a macro-F1 of 0.8102, outperforming the multimodal CLIP + BERT-base hybrid model, which scored 0.7603.

Key takeaway

For Machine Learning Engineers developing social media content moderation systems, particularly for vaccine-related misinformation, you should prioritize text-only models. The text-only BERT-base-uncased model achieved a macro-F1 of 0.8102, outperforming multimodal approaches like CLIP + BERT-base. This suggests that focusing on robust text analysis can yield superior results, potentially simplifying architecture and reducing computational overhead compared to complex multimodal fusion. Consider evaluating BERT-base-uncased as a strong baseline for similar classification tasks.

Key insights

Text-only BERT-base-uncased models surprisingly outperform multimodal approaches for vaccine meme classification.

Principles

• Text-only models can surpass multimodal for specific meme classification.
• Late fusion of image and text features is a viable multimodal strategy.
• Class-weighted loss and learning rate schedulers aid optimization.

Method

Classify Twitter vaccine memes using either multimodal (CLIP ViT-B/32 + BERT-based, late fusion) or text-only (BERT-base-uncased, RoBERTa-base, etc.) models. Optimize with AdamW, OneCycleLR, and early stopping.

In practice

• Prioritize text-only BERT-base-uncased for similar meme classification tasks.
• Evaluate multimodal fusion strategies like L2-normalized feature concatenation.
• Implement AdamW and OneCycleLR for robust model training.

Topics

• Vaccine-Critical Content
• Meme Detection
• BERT-base-uncased
• Multimodal Classification
• Text-Only Models
• Social Media Analysis

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• I Know What You Meme, Even If it Emerged Today: Understanding Evolving Memes through Open-World Knowledge Acquisition — Meme understanding requires dynamic, open-world knowledge acquisition beyond static model parameters.
• Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings — LLM unembedding matrices encode an "edge spectrum" subspace biasing embeddings towards frequent, uninformative tokens, which EmbedFilter removes.
• Severity-Aware Curriculum Learning with Multi-Model Response Selection for Medical Text Generation — A severity-aware multi-model framework using curriculum learning and response selection significantly improves medical text generation across case complexities.
• Comparing BERT Sentence-Pair Classification and Few-Shot LLM Prompting for Detecting Threat and Solution Framing in German Climate News — Fine-tuned BERT models outperform few-shot LLMs for German climate news framing detection, especially with sentence context.
• OpenAI CLIP: The Model That Learnt Zero-Shot Image Recognition Using Text — CLIP connects vision and language through contrastive learning on internet-scale image-text pairs, enabling zero-shot image classification.
• Multimodal foundation models exploit text to make medical image predictions — Multimodal medical AI models heavily rely on text, which can both enhance and degrade image-based diagnostic accuracy.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.