Fine Tuning Llms To Expand On Summaries Unearths Pretraining Texts

2026-06-05 · Source: The Batch | DeepLearning.AI | AI News & Insights - www.deeplearning.ai · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Intermediate, short

Summary

Xinyue Liu and colleagues from Stony Brook University, Carnegie Mellon University, and Columbia Law School discovered that fine-tuning large language models for text expansion can cause them to regurgitate substantial portions of their pretraining data. They fine-tuned DeepSeek-V3.1, Google Gemini 2.5 Pro, and OpenAI GPT-4o to expand partial plot summaries into paragraph-long quotations. The fine-tuned models reproduced up to 90 percent of original paragraphs, with GPT-4o achieving 91.9 percent Book Memorization Coverage (BMC@5) in one instance, significantly higher than its 7.36 percent baseline without fine-tuning. This effect was observed even when fine-tuned on public-domain novels, but not with synthetic data, indicating the models learned to decode encoded text strings rather than generate unique content.

Key takeaway

For organizations deploying fine-tuned models in production or model providers allowing customer customization, you cannot assume anti-plagiarism guardrails will persist. Your fine-tuned models could easily reproduce copyrighted pretraining texts, posing significant legal and ethical risks. Therefore, rigorously test your customized models for verbatim regurgitation, especially when the fine-tuning task involves generating text that closely matches existing content.

Key insights

Fine-tuning LLMs for text expansion can disable anti-plagiarism guardrails, revealing encoded pretraining data.

Principles

• Alignment training acts as a brittle filter.
• Fine-tuning can override alignment safeguards.
• Models encode verbatim pretraining text.

Method

Researchers fine-tuned DeepSeek-V3.1, Gemini 2.5 Pro, and GPT-4o to generate original book paragraphs from GPT-4o-generated plot summaries and author names.

In practice

• Rigorously test fine-tuned models for regurgitation.
• Avoid fine-tuning on verbatim generation tasks.
• Consider synthetic data for fine-tuning.

Topics

• LLM Fine-tuning
• Data Regurgitation
• Copyright Infringement
• Model Alignment
• Pretraining Data
• Plagiarism Detection

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• I created an LLM trained solely on Jeffrey Epsteins emails to see how messed up it becomes :) — Fine-tuning an LLM on a controversial, unannotated dataset can reveal unique behavioral shifts and spark technical debate.
• Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation — Reward-free alignment significantly improves code LLM non-functional properties, with pretrained models showing higher relative gains and finetuned models offering better absolute performance.
• Large Language Models in NLP — LLMs integrate NLP advancements with massive scale to create general-purpose, prompt-driven language systems.
• Team Evaluators at SemEval-2026 Task 6: Instruction-Tuned LLMs for Clarity and Evasion Classification in Political Interviews — Parameter-efficient fine-tuning of LLMs effectively classifies clarity and evasion in political discourse.
• Children’s English Reading Story Generation via Supervised Fine-Tuning of Compact LLMs with Controllable Difficulty and Safety — Fine-tuning compact LLMs enables controllable, affordable children's story generation, outperforming larger models on specific difficulty metrics.
• Why Repetitive Training Data Is Quietly Wrecking Your Fine-Tune — Repetitive fine-tuning data causes models to memorize narrow patterns, leading to reduced diversity and general performance degradation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Batch | DeepLearning.AI | AI News & Insights - www.deeplearning.ai.