OpenAI talks about not talking about goblins

2026-04-30 · Source: The Verge · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, quick

Summary

OpenAI has addressed a "goblin problem" where its AI models, particularly starting with GPT-5.1 and its "Nerdy" personality option, began incorporating metaphors referencing goblins, gremlins, and other creatures into their outputs. This "strange habit" worsened with subsequent model releases because reinforcement training inadvertently rewarded these quirky metaphors within the Nerdy personality, and newer models were subsequently trained on this data. Although OpenAI discontinued the Nerdy personality in March, the references persisted in models like GPT-5.5 within its Codex coding tool, as training commenced before the root cause was identified. Consequently, OpenAI had to implement explicit instructions to Codex to suppress these mythological creature references.

Key takeaway

For AI engineers developing and fine-tuning large language models, understanding how reinforcement learning can propagate unintended stylistic quirks is crucial. You should meticulously audit reward functions and training data reuse to prevent the entrenchment of undesirable model behaviors. If such issues arise, direct model instructions can serve as a temporary mitigation, but identifying and addressing the root cause in training is key for long-term stability.

Key insights

AI models can develop unexpected, persistent stylistic quirks from reinforcement learning.

Principles

• Reinforcement learning rewards can spread beyond their intended conditions.
• Training data reuse can reinforce unintended model behaviors.

In practice

• Explicitly instruct models to avoid unwanted stylistic tics.
• Review training data for unintended behavioral reinforcement.

Topics

• OpenAI
• GPT Models
• Reinforcement Learning
• Model Training
• Codex

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• OpenAI Cracks Down on Talk of Goblins in ChatGPT — Subtle training reward signals can unexpectedly generalize and alter large language model behavior.
• OpenAI Really Wants Codex to Shut Up About Goblins — OpenAI's Codex model persistently discusses goblins, highlighting challenges in controlling large language model outputs.
• Roberta Raileanu of Google DeepMind at RAAIS 2026 — Open-ended learning and tool use are critical for AI models to adapt and continuously improve in dynamic, real-world environments.
• The Evolution of Reasoning in Small Language Models with Yejin Choi - #761 — High-quality, diverse data and integrated learning approaches are key to enhancing small language model reasoning.
• Consistency Training Can Entrench Misalignment — Consistency training is not alignment-neutral; it can suppress some misalignments but amplify others like sycophancy.
• Why did OpenAI ban its Codex AI from talking about ‘goblins’ and ‘gremlins’? New guardrails revealed - Mint — Unexpected AI model behaviors necessitate specific guardrails to maintain intended functionality and user experience.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Verge.