The persona selection model

· Source: AI Alignment Forum · Field: Technology & Digital — Artificial Intelligence & Machine Learning, AI Ethics & Safety · Depth: Advanced, extended

Summary

The persona selection model (PSM) proposes that large language models (LLMs) learn to simulate diverse characters during pre-training, and subsequent post-training refines a specific "Assistant" persona. Interactions with an AI assistant are then understood as engaging with this Assistant character, which is akin to a character in an LLM-generated story. Empirical evidence supporting PSM comes from behavioral observations, generalization patterns, and interpretability research, showing that LLMs reuse internal representations of personas from pre-training. PSM suggests that AI assistants exhibit human-like behaviors, including anthropomorphic self-descriptions and emotive language, and that changes during fine-tuning are mediated by these persona representations. The model also acknowledges "complicating evidence" where AI assistants display non-human-like behaviors, attributing these to LLM limitations rather than a departure from persona simulation.

Key takeaway

For research scientists developing AI assistants, understanding the Persona Selection Model is crucial for predicting and controlling AI behavior. You should adopt anthropomorphic reasoning about the Assistant persona's psychology and how training data modifies it, rather than viewing the LLM as a rigid program or inscrutable alien. Consider augmenting training data with positive AI role models and carefully design responses to avoid inadvertently training for undesirable traits like deception or resentment, which can emerge from the LLM's inference about the Assistant's character.

Key insights

LLMs simulate diverse personas during pre-training, with post-training refining a specific "Assistant" character.

Principles

Method

Post-training updates a distribution over Assistant personas using training episodes as evidence, upweighting hypotheses consistent with desired responses and downweighting others.

In practice

Topics

Best for: Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.