The Roast of GPT4o: Experiments in Generating, Detecting and Evaluating Celebrity Roast Comedy

2026-06-27 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, short

Summary

Exploratory experiments by Jens Lemmens, Jérémy Genette, Tony Veale, and Walter Daelemans investigated GPT4o's capabilities in generating celebrity roast comedy. Published in the Proceedings of the 2nd Workshop on Computational Humor (CHum 2026), their study involved scraping @ComedyCentral roasts to design a survey. Participants blindly evaluated snippets of both human and AI-generated roasts, subsequently attempting to predict the author. The findings indicate no significant difference in how the barbs from human- and AI-generated roasts were rated by participants. Furthermore, a qualitative analysis revealed that while GPT4o employs specific recurrent phrases to mimic human comedians, both generative LLM detectors and human evaluators performed suboptimally in accurately identifying the true author of the roasts.

Key takeaway

For AI Scientists evaluating generative models for creative content like comedy, recognize that GPT4o can produce roasts indistinguishable from human-authored ones. This suggests current detection methods, both human and automated, are insufficient for sophisticated AI text. You should prioritize developing more advanced detection mechanisms and rigorous evaluation metrics that go beyond surface-level stylistic imitation to truly differentiate AI-generated creative works.

Key insights

GPT4o generates roasts indistinguishable from human ones, fooling both humans and AI detectors.

Principles

• AI can mimic complex human humor styles effectively.
• Human and AI detectors struggle with advanced AI-generated text.
• Stylistic imitation can mask AI authorship.

Method

Scraped @ComedyCentral roasts, created a survey for blind evaluation of human/AI snippets, then a second round for author prediction.

In practice

• Test AI models on nuanced creative text generation.
• Develop more robust AI content detection tools.
• Evaluate AI's ability to imitate specific comedic styles.

Topics

• GPT4o
• Generative AI
• Computational Humor
• AI Content Detection
• Comedic Roasts
• Human-AI Evaluation

Best for: NLP Engineer, AI Scientist, Research Scientist

Related on AIssential

• Beating the Style Detector: Three Hours of Agentic Research on the AI-Text Arms Race — Agentic research rapidly reproduces and extends NLP studies, demonstrating LLMs' ability to evade AI-text detection.
• Humor Style Drives Laughter, Topic Shapes Acceptability: Evaluating Bilingual Personal and Political Robot-Delivered AI Jokes — Humor style dictates funniness, while joke topic determines acceptability for robot-delivered AI jokes.
• WHY ARE YOU LIKE THIS — AI image generation models can introduce unprompted, contextually relevant elements.
• After the Acid, the Opium — This HackerNoon article is a human-written science fiction story, verified by GPTZero, and available in multiple languages.
• Off for adventures — AI image generation still struggles with realistic depiction, often producing humorous artifacts.
• A Multi-Domain Benchmark for Detecting AI-Generated Text-Rich Images from GPT-Image-2 — Existing AI image detectors struggle with text-rich, structured content, necessitating new benchmarks and specialized detection methods.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.