Data filtering works a lot worse than you would expect

2026-07-07 · Source: AI Alignment Forum · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A study investigating data filtering for supervised fine-tuning (SFT) of large language models found that removing specific training data points is largely ineffective at eliminating undesirable behaviors. Researchers used a "speed-run" OLMo SFT model, a rank 64 LoRA on OLMo 3 7B mid-train with a 1% stratified sample of the Dolci-Think-SFT-7B dataset. They evaluated seven behaviors, including bold formatting, liberal-lean, and "validate feelings." Across various Training Data Attribution (TDA) methods—EKFAC, Probe, LLM Judge, and Activation-based—filtering out 10-25% of identified "proponent" documents often performed no better than random removal. For instance, filtering 10% of documents did not reduce the "Your feelings are valid" behavior, despite relevant phrases appearing in less than 0.2% of documents. The only exception was "refusal," which proved filterable using Probes and LLM Judges. This suggests many SFT behaviors are elicited from the base model's existing personas rather than being taught by specific SFT data.

Key takeaway

For machine learning engineers aiming to mitigate undesirable behaviors in LLMs via supervised fine-tuning, recognize that data filtering is generally ineffective for broad traits. Your efforts should focus on "refusal" behaviors, where LLM Judges or Probes show efficacy. For other behaviors, consider that they may be inherent "personas" elicited from the base model, requiring alternative intervention strategies beyond simple data removal.

Key insights

Data filtering is largely ineffective for removing broad undesirable LLM behaviors acquired during SFT, except for refusal.

Principles

• Broad SFT behaviors are often elicited, not taught.
• Many behaviors are bundled into model personas.
• Random data removal often matches targeted filtering.

Method

The study involved fine-tuning OLMo 3 7B mid-train with LoRA, identifying behaviors, scoring training examples with TDA methods, filtering top examples, and retraining to evaluate behavior reduction.

In practice

• Prioritize filtering for "refusal" behaviors.
• Use LLM Judges or Probes for refusal filtering.
• Re-evaluate assumptions about SFT behavior origins.

Topics

• Data Filtering
• Supervised Fine-Tuning
• Large Language Models
• OLMo 3 7B
• Training Data Attribution
• Model Behavior
• LoRA

Code references

• pomonam/kronfluence

Best for: Research Scientist, AI Engineer, NLP Engineer, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• I created an LLM trained solely on Jeffrey Epsteins emails to see how messed up it becomes :) — Fine-tuning an LLM on a controversial, unannotated dataset can reveal unique behavioral shifts and spark technical debate.
• Pretraining Data Filtering for Open-Weight AI Safety — Pretraining data filtering effectively builds tamper-resistant safeguards into open-weight LLMs by preventing undesirable knowledge acquisition.
• LLMs just have more time and energy? — LLMs imitate human language patterns, including confidence and humility, through relentless, large-scale data compression, acting as a mirror of human communication.
• I fine-tuned an LLM to be C-3PO to test which training data format works best for persona injection [P] — First-person statements are most effective for LLM persona injection, outperforming chat demos and synthetic docs.
• Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior — LLM self-reports predict behavior selectively, requiring fine-grained instruments and shared context for coherence.
• Modeling Pathology-Like Behavioral Patterns in Language Models Through Behavioral Fine-Tuning — Behavioral fine-tuning can induce specific, stable, pathology-like biases in LLMs, shifting their fundamental interpretive priors.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.