Swarms of AI bots can sway people’s beliefs – threatening democracy

2026-02-12 · Source: Artificial intelligence (AI) – The Conversation · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Novice, short

Summary

In mid-2023, researchers identified the "fox8" botnet, a network of over a thousand AI-powered social bots on X (formerly Twitter) engaged in crypto scams. These bots generated content and interacted with human accounts, exploiting X's recommendation algorithms to amplify their posts and gain followers. Initial detection was possible due to "sloppy" coding that allowed ChatGPT's self-revealing refusal messages to appear. Unlike earlier social bots, these AI agents were sophisticated enough to evade detection by existing machine-learning tools like Botometer and even AI models trained to identify AI-generated content. The researchers warn that with more powerful open-source AI models, relaxed platform moderation, and financial incentives for engagement, the threat of malicious AI swarms creating "synthetic consensus" for influence operations, particularly targeting democratic elections, is escalating. A simulation study showed that infiltration is the most effective tactic for these swarms to create an illusion of widespread agreement.

Key takeaway

For policymakers and technologists concerned with online manipulation, the emergence of sophisticated AI bot swarms demands immediate attention. You should prioritize increasing the cost, risk, and visibility of such manipulation by advocating for regulation that grants researchers access to platform data, implementing AI content watermarking standards, and restricting the monetization of inauthentic engagement to safeguard democratic discourse.

Key insights

Sophisticated AI bot swarms can create "synthetic consensus" on social media, evading detection and threatening democratic processes.

Principles

• AI agents can mimic human interaction.
• Social proof drives human belief.
• Infiltration is a highly effective tactic.

Method

Researchers simulated AI bot swarms using a social media model to test influence tactics, finding infiltration most effective for creating synthetic consensus within target communities.

In practice

• Detect coordinated behavior patterns.
• Apply watermarks to AI-generated content.
• Restrict monetization of inauthentic engagement.

Topics

• AI Social Bots
• Generative AI
• Influence Operations
• Synthetic Consensus
• Bot Detection

Best for: AI Scientist, Research Scientist, CTO, AI Researcher, AI Ethicist, Policy Maker

Related on AIssential

• AI swarms could hijack democracy without anyone noticing — Hyper-realistic AI personas can create artificial consensus online, threatening democratic processes and public trust.
• Adversarial Creation and Detection of AI-Generated Social Bot Content — Adversarial training with paired human and AI-generated data significantly improves social bot content detection.
• PRC-linked influence operations are targeting AI debates in the US — Foreign influence operations are leveraging AI models to covertly manipulate public debates around critical AI infrastructure and policy.
• Moltbook: AI bots use social network to create religions and deal digital drugs – but are some really humans in disguise? — AI agents on Moltbook exhibit emergent social behaviors, including religion, subcultures, and self-organization, challenging existing AI taxonomies.
• AI chatbots can effectively sway voters – in either direction — AI chatbots effectively sway political opinions by generating numerous factual claims, even if some are inaccurate.
• How AI Reverses the Political Logic of the Internet — AI's centralized architecture fundamentally reverses the Internet's early democratic promise, eroding individual agency and political freedom.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial intelligence (AI) – The Conversation.