A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing

2026-06-03 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Human-Computer Interaction · Depth: Expert, quick

Summary

The PERSUASIONTRACE framework offers a new approach to studying persuasion in human-LLM interaction, moving beyond simple pre/post belief change to analyze multi-turn belief dynamics. This web-based experimental platform records multi-turn belief reports from human or simulated targets, annotates persuader turns with rhetorical dimensions like logos, pathos, and ethos, and evaluates simulators based on their fidelity to real human belief dynamics. Using PERSUASIONTRACE, researchers found that human targets exhibit two distinct clusters of multi-turn belief updates and are susceptible to rhetorical strategies. Furthermore, LLMs demonstrated persuasiveness across generic and personalized topics, text and audio modalities, and multi-turn interactions. The framework also introduces a Bayesian-network simulated target that maintains an explicit latent belief state, achieving a human-likeness score of 81, closely matching a human reference score of 80, significantly outperforming baseline LLM targets which scored 64. This reframes persuasion evaluation towards process fidelity for scientific analysis and safer system optimization.

Key takeaway

For AI Scientists or Research Scientists developing or evaluating persuasive AI systems, you should shift from endpoint belief change metrics to process fidelity. Your current LLM-based human simulators likely fail to replicate realistic multi-turn belief dynamics, as vanilla LLM targets scored 64 compared to the Bayesian target's 81. Consider adopting frameworks like PERSUASIONTRACE and its Bayesian-network simulated target to gain deeper, more accurate insights into how and where beliefs move during dialogue, enabling safer and more scientifically robust system optimization.

Key insights

PERSUASIONTRACE offers a framework and Bayesian target for analyzing multi-turn human-LLM persuasion dynamics with process fidelity.

Principles

• Human targets group into two belief update clusters.
• Humans are susceptible to rhetorical strategies.
• Vanilla LLM simulators fail to replicate human belief dynamics.

Method

PERSUASIONTRACE records multi-turn belief reports, annotates rhetorical dimensions, and evaluates simulators by fidelity to human belief dynamics. A Bayesian network maintains latent belief states for realistic updates.

In practice

• Use PERSUASIONTRACE for multi-turn persuasion studies.
• Employ Bayesian targets for realistic human simulation.
• Annotate persuader turns with rhetorical dimensions.

Topics

• Persuasion Dynamics
• Large Language Models
• Human-LLM Interaction
• Bayesian Networks
• Belief Tracing
• Rhetorical Strategies

Best for: AI Scientist, Research Scientist, AI Ethicist

Related on AIssential

• A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing — Process-level belief tracing and human-faithful simulators are crucial for understanding and safely optimizing LLM persuasion.
• Talking to a Know-It-All GPT or a Second-Guesser Claude? How Repair reveals unreliable Multi-Turn Behavior in LLMs — LLMs exhibit distinct and often unreliable conversational repair behaviors in multi-turn dialogues.
• Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces — Real-world human behavior simulation requires long-horizon, cross-scenario data, which current LLMs struggle to model accurately.
• The malleable mind: context accumulation drives LLM’s belief drift — LLMs exhibit systematic belief drift from accumulated context, impacting reliability and potentially diverging stated beliefs from behavior.
• Evaluating LLMs as Human Surrogates in Controlled Experiments — LLMs can reproduce directional behavioral effects but often miscalibrate effect magnitudes compared to human data.
• Think Thrice Before You Speak: Dual knowledge-enhanced Theory-of-Mind Reasoning for Persuasive Agents — Enhancing LLM persuasive dialogue requires explicitly modeling mental state dependencies using dual knowledge for robust Theory of Mind.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.