"That's AI Slop, You Bot!" Studying Accusations, Evidence, and Credibility in Online Discourse Towards LLM-Generated Comments

· Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Social Sciences & Behavioral Studies · Depth: Expert, quick

Summary

A study titled "That's AI Slop, You Bot!" analyzed 25 million comments from Hacker News and Reddit between 2023 and 2026 to understand reader responses to LLM-generated content. Researchers combined LLM judgment on 7,500 sampled accusations, sentiment trajectories, speech-act coding of 300 confirmed accusations, and a matched-control test. They found that the pejorative label "AI slop" increased over tenfold, now comprising 94 percent of negative mentions, with the tone shifting from mockery to gatekeeping. Crucially, the study revealed that linguistic features distinguishing AI from human text do not predict which human text is accused of being AI-generated. This suggests online accusations function as social gatekeeping for perceived authenticity rather than accurate AI detection, extending signaling theory to explain how inaccurate substitute signals can proliferate when expert detection is unavailable to the public. The research highlights distinct reader-side impacts of AI on writing.

Key takeaway

For AI Ethicists, Content Strategists, or Community Managers managing online communities or developing AI content policies, you should recognize that public accusations of "AI slop" are primarily social signals of perceived inauthenticity and gatekeeping, not reliable indicators of actual AI generation. Focus your efforts on fostering genuine engagement and transparent communication about content origins, rather than solely relying on technical detection tools to address community concerns about AI-generated text. This shift in understanding is crucial for maintaining credibility and trust.

Key insights

Online "AI slop" accusations function as social gatekeeping for perceived authenticity, not accurate AI detection.

Principles

Method

Researchers analyzed 25 million comments from Hacker News and Reddit (2023-2026), combining LLM judgment on 7,500 accusations, sentiment analysis, speech-act coding, and a matched-control test.

In practice

Topics

Best for: AI Scientist, Research Scientist, AI Ethicist

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.