Toxic Subword Pruning for Dialogue Response Generation on Large Language Models

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

The ToxPrune algorithm offers a novel approach to mitigate toxic content generation in large language models, particularly addressing "already-toxic" models or scenarios where model weights are accessible. Unlike prior research focused on defending against jailbreak prompts on safe models, ToxPrune operates by pruning subwords associated with toxic words from the Byte Pair Encoding (BPE) in trained LLMs. This method surprisingly demonstrates usefulness in preventing toxic output, contrasting with earlier findings that BPE pruning was detrimental to machine translation. ToxPrune simultaneously improves the NSFW-3B model on dialogue response generation and enhances diversity in the official Llama-3.1-6B. Both automatic results and human evaluations confirm its efficacy for remediating toxic LLMs and improving non-toxic LLMs in dialogue response tasks.

Key takeaway

For machine learning engineers deploying or fine-tuning large language models, consider integrating ToxPrune to address potential toxicity issues, especially with models that may already exhibit undesirable behaviors or when you have access to model weights. This method not only remediates toxic outputs but also improves response diversity, as demonstrated on Llama-3.1-6B. Implementing ToxPrune could enhance your model's safety and performance in dialogue generation tasks.

Key insights

ToxPrune effectively mitigates LLM toxicity by pruning subwords from BPE, surprisingly enhancing diversity and dialogue generation.

Principles

• Subword pruning from BPE can prevent LLM toxicity.
• BPE pruning, contrary to prior work, can be beneficial.

Method

ToxPrune prunes subwords contained by toxic words from the Byte Pair Encoding (BPE) within already trained large language models.

In practice

• Remediate existing toxic LLMs.
• Improve non-toxic LLMs for dialogue generation.
• Enhance LLM response diversity.

Topics

• Large Language Models
• Toxicity Mitigation
• Subword Pruning
• Byte Pair Encoding
• Dialogue Response Generation
• Llama-3.1-6B

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Detoxification for LLM: From Dataset Itself — Detoxifying LLM pretraining datasets directly at the source fundamentally reduces inherent model toxicity while preserving semantics.
• Building a Context Pruning Pipeline for Long-Running Agents — Efficiently manage LLM context for long-running agents by dynamically pruning conversation history based on semantic relevance.
• From Prompt Risk to Response Risk: Paired Analysis of Safety Behavior of Large Language Model — LLM safety analysis benefits from paired prompt-to-response severity transitions and joint relevance assessment.
• LEAF: Growing Trees Without Branching for Speech-Aware Large Language Model Post-Training — LEAF is a retrospective tree-based RL method for speech-aware LLM post-training that refines credit assignment.
• From Hazard Functions to Language Space: Cox-Supervised Distillation of Survival Risk into a Large Language Model — Large language models can internalize and predict survival risk from structured clinical data via text-based distillation.
• Where Do Large Language Models Fail on Competitive Programming? A Taxonomy of Failures by Algorithm Type and Difficulty Rating — Chain-of-Thought prompting can degrade LLM code generation performance and instruction adherence in competitive programming.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.