ChatGPT and other AI bots made huge errors before Scottish election, study finds

2026-05-20 · Source: AI (artificial intelligence) | The Guardian · Field: Government & Public Sector — Public Policy & Governance, Regulatory & Compliance, Artificial Intelligence & Machine Learning · Depth: Fundamental Awareness, short

Summary

A study by the thinktank Demos, commissioned by the Electoral Commission, found that AI chatbots like ChatGPT and Google Gemini made significant errors providing election information before the recent Scottish election. The investigation, which posed 75 questions to five free AI tools, revealed misinformation in 34% of responses. Specific inaccuracies included inventing fictitious scandals, providing incorrect election dates, misstating voter ID requirements, and placing candidates in wrong contests. An accompanying poll indicated 20% of British voters, equivalent to 10 million people, used AI tools for election information. Replika performed worst with 56% errors, followed by ChatGPT at 46%, and Google Gemini at 22%. Grok had the lowest error rate at 9% but provided poor external links. The Electoral Commission is urging ministers to introduce new legal controls and clearer duties for AI platforms to combat misinformation, while Demos advocates for making AI companies liable under UK law and mandating accuracy safeguards.

Key takeaway

For policy makers developing AI regulation, this study underscores the urgent need for specific legal controls over AI chatbots in electoral contexts. You should prioritize legislation that establishes clear duties for AI platforms to prevent misinformation, mandates accuracy safeguards for election-related content, and ensures companies are liable under electoral law. This proactive approach is critical to protect democratic processes from the rapid spread of false information.

Key insights

AI chatbots frequently generate election misinformation, necessitating urgent regulatory intervention to protect democratic processes.

Principles

• AI tools pose significant election integrity risks.
• Regulation must address AI platform accountability.
• Transparency in AI data/training is crucial.

Method

Demos simulated voter queries by asking 75 questions to five free AI tools (ChatGPT, Google Gemini, Replika) about three real-life constituencies to assess accuracy and evidence.

In practice

• Implement legal requirements for AI company liability.
• Mandate accuracy safeguards for AI election content.
• Allow researchers to test AI internal data.

Topics

• AI Chatbots
• Election Integrity
• Political Misinformation
• AI Regulation
• Electoral Commission
• Demos Thinktank

Best for: CTO, VP of Engineering/Data, Director of AI/ML, Policy Maker, AI Ethicist, Tech Journalist

Related on AIssential

• UK Electoral Commission is pushing for AI regulations (chatbots got 34% of election answers wrong) — AI chatbots exhibit significant error rates in providing election information, necessitating regulatory oversight.
• AI chatbots giving misleading voting advice in run-up to election - BBC — AI chatbots can provide inconsistent and factually incorrect political information, including voting recommendations and candidate details.
• Family of man killed in shooting at Florida State University to sue ChatGPT and OpenAI — AI chatbot developers face increasing legal challenges over their platforms' alleged roles in violent acts.
• Brazil Is Preparing for Its First Real AI Election, But Is It Ready? — Brazil's electoral court is struggling to regulate rapidly evolving AI in elections due to limited scope and stalled comprehensive legislation.
• Politicians couldn't keep kids off social media. They’ll try again with AI. — Political and industry leaders are converging on age-based AI regulation, learning from social media's impact on children.
• Election information and safeguards in 2026 — AI providers are deploying multi-faceted strategies to mitigate generative AI risks in global elections.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI (artificial intelligence) | The Guardian.