I created an LLM trained solely on Jeffrey Epsteins emails to see how messed up it becomes :)

2026-02-15 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Data Science & Analytics · Depth: Intermediate, quick

Summary

A developer created a language model (LLM) by fine-tuning TinyLLama-v0 using a dataset of Jeffrey Epstein's emails. The project, hosted on GitHub, aimed to explore the model's behavioral changes after exposure to this specific, controversial dataset. The fine-tuning process involved 100 epochs with a learning rate of 0.0004. Community discussion surrounding the project highlighted technical distinctions between training and fine-tuning, with some commenters clarifying that the model was likely subjected to "continued pre-training" rather than traditional fine-tuning due to the unannotated nature of the data. The project also generated humorous and speculative comments regarding the model's potential sentience and its mysterious deactivation.

Key takeaway

For AI Research Scientists exploring model behavior with niche datasets, consider the precise technical definition of your training approach. Clarifying whether you are fine-tuning or performing continued pre-training is crucial for accurate interpretation of results and community discourse. Be prepared for discussions on data provenance and its impact on model characteristics.

Key insights

Fine-tuning an LLM on a controversial, unannotated dataset can reveal unique behavioral shifts and spark technical debate.

Principles

• Data quality dictates model behavior.
• Fine-tuning alters an LLM's "vibe."

Method

The project involved fine-tuning TinyLLama-v0 for 100 epochs with a learning rate of 0.0004, using Jeffrey Epstein's emails as the dataset.

In practice

• Consider data source implications for model output.
• Distinguish between fine-tuning and continued pre-training.

Topics

• LLM Fine-tuning
• TinyLLama
• Dataset Curation
• AI Ethics
• Controversial AI Applications

Code references

• lachameleon/epstein-llm

Best for: AI Scientist, Research Scientist, AI Product Manager, AI Engineer, Machine Learning Engineer, AI Student

Related on AIssential

• Developing an LLM: Building, Training, Finetuning — LLM development involves building architecture, pre-training on vast datasets, and fine-tuning for specific tasks and behaviors.
• Modeling Pathology-Like Behavioral Patterns in Language Models Through Behavioral Fine-Tuning — Behavioral fine-tuning can induce specific, stable, pathology-like biases in LLMs, shifting their fundamental interpretive priors.
• I Fine-Tuned an LLM to Reply Like My Wife — 10 Years of Chats, One Adapter, and Results That Amazed… — Fine-tuning LLMs with personal, messy data on consumer hardware can yield surprising persona learning.
• Are LLMs safe? — LLM training data curation, especially via quality filters, embeds implicit biases that shape model behavior and capabilities.
• No, Pasting Data Into ChatGPT Does Not Train It — Prompting and RAG provide temporary context; fine-tuning permanently alters an LLM's internal parameters and latent space.
• Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost? — Fine-tuning LLMs improves personality questionnaire response stability but fails to induce accurate, multi-dimensional Big Five profiles from unguided text.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.