Do Large Language Models Always Tell The Same Stories?

2026-06-15 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A recent investigation into the diversity of large language model (LLM)-generated stories reveals that these narratives are consistently more similar to each other than human-written stories. Researchers utilized a contrastive framework, a dataset of human-written stories and prompts from r/WritingPrompts, and collected narrative similarity judgments across 10 representative LLMs. Both human evaluations and three different automatic annotation methods confirmed this trend. The study found that frontier models, in particular, converge on a "mean" generic narrative, which approximates individual human stories but significantly lacks the collective diversity found in human authors. Furthermore, common mitigation strategies like negative prompting and temperature scaling were shown to be ineffective in meaningfully addressing this observed homogeneity.

Key takeaway

For NLP Engineers developing generative AI applications, you should critically evaluate the true narrative diversity of your LLM outputs. Relying solely on techniques like temperature scaling or negative prompting will likely not yield genuinely varied stories, potentially leading to repetitive user experiences or content. Consider integrating human oversight or exploring novel architectural approaches to ensure your generated content achieves the desired level of creative breadth.

Key insights

Large Language Models consistently produce narratives that are more homogeneous than human-written stories, even with common mitigation efforts.

Principles

• LLM narratives lack collective human diversity.
• Frontier models converge on generic stories.
• Standard diversity mitigations are ineffective.

Method

Researchers used a contrastive framework, human evaluations, and three automatic annotation methods to assess narrative similarity across 10 LLMs using r/WritingPrompts data.

In practice

• Evaluate LLM outputs for narrative uniqueness.
• Do not rely on temperature scaling for diversity.
• Consider human-in-the-loop for creative content.

Topics

• Large Language Models
• Narrative Generation
• Content Diversity
• Generative AI Evaluation
• Prompt Engineering
• Homogeneity

Best for: Research Scientist, AI Product Manager, AI Scientist, NLP Engineer, AI Student

Related on AIssential

• Automated Creativity Evaluation of Language Models Across Open-Ended Tasks — A new domain-agnostic framework automates large language model creativity evaluation across diverse open-ended tasks.
• AI is making journalistic language more repetitive and predictable – and it’s a problem for all of us — AI's pervasive use in journalism risks linguistic homogenization and decline, particularly via "model collapse" from training on synthetic text.
• StoryCoder: Narrative Reformulation for Structured Reasoning in LLM Code Generation — Narrative reformulation of code problems significantly improves LLM code generation by structuring context.
• Why Every AI Writing Tool I Tried Failed Me (And What They All Get Wrong) — AI writing tools fail at novel-length consistency due to context amnesia and a lack of structured, persistent world-building.
• Competition and Diversity in Generative AI — Competition among generative AI users can increase content diversity, but equilibrium remains less diverse than optimal.
• Large Language Models in NLP — LLMs integrate NLP advancements with massive scale to create general-purpose, prompt-driven language systems.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.