Assisted Counterspeech Writing at the Crossroads of Hate Speech and Misinformation

2026-05-21 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

A new study investigates using Large Language Models (LLMs) to assist in writing counterspeech (CS) for online content that combines both hate speech and misinformation, a problem previously addressed separately. Researchers tested three knowledge-driven generation strategies: prompting an LLM with fact-checkers' guidelines, with NGOs' guidelines, or a mixed approach combining both. While LLMs generated adequate counterspeech in 40% of cases, expert revisions by 23 specialists significantly enhanced naturalness, exhaustiveness, and adherence to guidelines. The mixed strategy, which integrated guidelines and documents from both fact-checkers and NGOs, proved most effective in crowdsourcing evaluations, demonstrating strong factual correction alongside stereotype mitigation and empathetic engagement. The study also released a dataset of hateful and misinformed claims with expert-verified counterspeech and supporting knowledge.

Key takeaway

For NLP Engineers developing content moderation tools, you should integrate diverse knowledge sources like fact-checker and NGO guidelines when training or prompting LLMs for counterspeech generation. Your systems will benefit significantly from a human-in-the-loop approach, as expert revision substantially improves the naturalness and effectiveness of generated responses. Consider utilizing the released dataset to fine-tune models for nuanced hate speech and misinformation contexts.

Key insights

LLMs can assist counterspeech against co-occurring hate and misinformation, but expert refinement and mixed knowledge strategies are crucial.

Principles

• Combining diverse knowledge sources improves LLM output.
• Expert human review is vital for quality counterspeech.
• Counterspeech needs factual, empathetic, and mitigating elements.

Method

LLMs were prompted with fact-checker guidelines, NGO guidelines, or a mixed set to generate counterspeech for hateful and misinformed claims, then expert-revised and evaluated.

In practice

• Integrate fact-checking and NGO guidelines for LLM prompts.
• Implement expert human-in-the-loop review for CS.
• Utilize the released dataset for model training.

Topics

• Counterspeech Generation
• Large Language Models
• Hate Speech Detection
• Misinformation Combatting
• Expert-in-the-Loop AI
• Dataset Release

Best for: Research Scientist, AI Scientist, NLP Engineer, AI Ethicist

Related on AIssential

• Assisted Counterspeech Writing at the Crossroads of Hate Speech and Misinformation — LLM-assisted counterspeech for combined hate and misinformation is most effective with a mixed, knowledge-driven strategy and expert human post-editing.
• Explain the Flag: Contextualizing Hate Speech Beyond Censorship — A hybrid LLM and vocabulary approach improves hate speech detection and explanation across multiple languages.
• Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval — Fact-checking without retrieval leverages LLM internal representations for robust, scalable claim verification.
• Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection — LLM labels can match human-level F1 scores for hostility detection but exhibit distinct error profiles.
• Socially Responsible and Explainable Automated Fact-Checking and Hate Speech Detection — Explainable NLP methods and high-quality datasets can jointly optimize performance and interpretability for Portuguese hate speech and fact-checking.
• Do we really need to detect LLM-generated text? — Masked word prediction by LLMs can distinguish AI-generated text from human text without explicit training.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.