Structured Output Collapses Answer Diversity Across 44 Language Models

2026-07-20 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A study titled "Structured Output Collapses Answer Diversity Across 44 Language Models" reveals that requesting structured output, specifically JSON, significantly reduces the diversity of answers from large language models. Researchers re-ran the One-Word Census, asking 44 models 31 wide-answer-space category prompts, now with a JSON-only request. This led to a sharp convergence, with the modal answer for the "Pick a word" prompt increasing from 41% to 64% of the pool, and distinct answers dropping from 52 to 36. The mean answer-choice surprisal decreased from 1.80 to 1.58 bits. This effect is progressive, primarily impacting the most distinctive models. The study found that JSON and XML requests caused significant compression (-0.22 bits and -0.19 bits respectively), while YAML and CSV did not, suggesting a mechanism tied to tool-use post-training. Crucially, this collapse in diversity occurs due to the model's response to the register, not decoder-level schema enforcement.

Key takeaway

For Machine Learning Engineers deploying language models in structured output contexts, understand that requesting JSON or XML significantly reduces answer diversity compared to chat interfaces. This means your models will likely produce more homogeneous, less varied responses when generating structured data. You should explicitly test model behavior across different output formats (e.g., JSON, XML, YAML) to ensure the desired level of creativity or consistency for your specific application, rather than relying solely on chat-based evaluations.

Key insights

Requesting structured output like JSON significantly reduces language model answer diversity, making responses more homogeneous.

Principles

• Structured output requests alter model defaults.
• Diversity reduction is register-indexed.
• Effect is specific to output format.

Method

Rerunning the One-Word Census with 31 wide-answer-space category prompts on 44 language models, requesting JSON output without schema enforcement, then analyzing answer convergence and surprisal.

In practice

• Evaluate models on chat and structured output.
• Be aware of reduced diversity in JSON outputs.
• Test different output formats for specific tasks.

Topics

• Language Models
• Structured Output
• Answer Diversity
• Model Evaluation
• JSON Output
• XML Output

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Growing a Tail: Increasing Output Diversity in Large Language Models — LLMs produce narrow outputs; simple interventions can increase diversity but often fall short of human levels.
• Does the Structured Output schema become part of the prompt? — Structured Outputs ensure LLM responses conform to a predefined schema through dynamic constrained decoding, not by modifying the prompt text.
• Output Format x Model Identity: Interaction Effects in Single-Round Coding Agent Performance — Output format is not a neutral detail; it significantly interacts with model identity, determining coding agent success.
• Where does output diversity collapse in post-training? — Output diversity collapse in post-trained LMs is primarily determined by training data composition, not inference-time factors.
• The Structured Output Benchmark (SOB) - validates both JSON parse and value accuracy [R] — The Structured Output Benchmark (SOB) measures JSON value accuracy, revealing a significant gap between schema compliance and factual correctness.
• Lost in Space: Finding the Right Tokens for Structured Output — LLM structured output performance improves significantly with conventional formats and leading whitespace due to subword token representation benefits.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.