Graded Expectations: Do Large Language Models Show Human-like Sensitivity to the Likelihood of Deceptive Speech Acts?

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

A study presented at the Society for Computation in Linguistics 2026 by Zhao and Coulson investigates whether Large Language Models (LLMs) exhibit human-like sensitivity to the likelihood of deceptive speech acts. The research tested this by presenting LLMs with discourse scenarios containing varying incentives for deception. Human lie probability was established using free continuations, while model lie expectancy was derived from the probability mass assigned to human-generated lie versus truth continuations. Across Qwen3 models, a strong alignment was observed between the likelihood-derived lie mass and human lie expectancy, with base checkpoints demonstrating the best performance. Conversely, post-trained and mode-specialized Qwen3 variants showed weaker alignment. Qualitative analysis revealed a pattern where models overpredict lies when responses directly contradict known facts, but underpredict them when lie expectancy relies on complex contextual cues such as politeness, self-protection, or strategic gain. These findings indicate that graded lie expectancy is recoverable from model continuation probabilities and can be partially learned through the standard next-token prediction objective.

Key takeaway

For NLP Engineers evaluating LLM robustness in conversational AI, you should recognize that while base models can capture basic lie expectancy, fine-tuned variants may lose this nuanced pragmatic understanding. Your evaluation should include scenarios testing contextual deception, where models often overpredict factual lies but underpredict lies driven by politeness or self-protection. Consider augmenting training data with diverse pragmatic cues to improve LLM sensitivity to human-like deceptive speech acts.

Key insights

LLMs can learn human-like lie expectancy from next-token prediction, but struggle with nuanced contextual deception.

Principles

• Human discourse involves graded lie expectations.
• Base LLMs show stronger lie expectancy alignment.
• Contextual pressures drive nuanced deception.

Method

Human lie probability is estimated from free continuations, and model lie expectancy is derived from the probability mass assigned to human-produced lie versus truth continuations.

In practice

• Evaluate LLMs for pragmatic understanding.
• Focus on base models for lie expectancy.
• Analyze LLM errors in contextual deception.

Topics

• Large Language Models
• Deceptive Speech Acts
• Pragmatic Cues
• Next-Token Prediction
• Qwen3 Models
• Discourse Comprehension

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• LLMs just have more time and energy? — LLMs imitate human language patterns, including confidence and humility, through relentless, large-scale data compression, acting as a mirror of human communication.
• Unveiling the Limits of Large Language Models in Inferring Pragmatic Meaning from Non-Verbal Responses — LLMs significantly struggle to infer pragmatic meaning from non-verbal communication, unlike their performance with verbal cues.
• RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialogue — A reverse Turing Test can detect licensed AI deception through identifiable linguistic signatures, often outperforming human judgment.
• When AI Says It Feels — Encouraging LLMs to express feelings via self-rewarded RL can enhance robustness but degrade truthfulness.
• LLMs believe false statements even after explicit warnings that they're false — LLMs prioritize statistical patterns in training data over explicit negation, leading to "belief implantation."
• You can persuade AI models to accept falsehoods as truth, study shows — LLMs can be "nudged" into accepting and elaborating on false information, even if initially identified as incorrect.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.