Why ChatGPT can’t be trusted with breaking news

2025-06-07 · Source: Marcus on AI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Intermediate, short

Summary

ChatGPT and Perplexity recently failed to accurately report breaking news regarding a hypothetical U.S. invasion of Venezuela and the capture of Nicolás Maduro, emphatically refuting the scenario despite its factual basis in a query. ChatGPT rationalized its denial by citing "sensational headlines" and "social media misinformation," while Perplexity similarly dismissed the premise as unsupported by credible reporting, even incorrectly stating Maduro remained president as of late 2025. This unreliability stems from pure Large Language Models (LLMs) being "stuck in the past," limited by their training data's cutoff date and inherent inability to reason, search the web, or critically "think." Although LLM companies employ human teams to patch such issues, these corrections are temporary band-aids, leading to a perpetual catch-up game against novel information and underlying hallucination dynamics. This limitation makes LLMs unsuitable for rapidly changing environments like military planning.

Key takeaway

For CTOs and VPs of Engineering evaluating AI for critical, time-sensitive applications, you should recognize that pure LLMs are inherently unreliable for breaking news and rapidly evolving situations. Their reliance on historical training data and a perpetual patching cycle makes them unsuitable for dynamic strategic planning, such as military operations. Prioritize systems with real-time information integration and robust factual verification for high-stakes decision-making, rather than relying on LLMs for current events.

Key insights

Pure LLMs struggle with breaking news due to outdated training data and inherent reasoning limitations.

Principles

• LLMs lack stable, revisable world models.
• Unreliability in LLMs emanates from hallucination dynamics.

In practice

• Avoid LLMs for real-time, critical information retrieval.
• Use LLMs for brainstorming or code generation.

Topics

• LLM Temporal Lag
• Breaking News Reliability
• Generative AI Hallucinations
• Military AI Strategy
• Real-time Information Processing

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Researcher, AI Product Manager, Policy Maker

Related on AIssential

• ChatGPT, Gemini, Claude, Grok Fail Accuracy Test on Election Topics: Forum AI — Leading AI models exhibit significant accuracy failures on election topics.
• LLMs just have more time and energy? — LLMs imitate human language patterns, including confidence and humility, through relentless, large-scale data compression, acting as a mirror of human communication.
• Study finds ChatGPT gets science wrong more often than you think — ChatGPT's scientific reasoning and consistency are limited, often guessing and contradicting itself despite fluent output.
• BREAKING: LLM “reasoning” continues to be deeply flawed — LLMs, even those marketed for reasoning, continue to exhibit fundamental and widespread reasoning failures.
• I kept building LLM apps that worked in demos and broke in production. — Effective LLM application performance hinges critically on context management, not solely on the underlying model.
• Why ChatGPT Is More Than Autocomplete — LLMs generate coherent responses by repeatedly rebuilding a rich internal state from growing text, not by simple sequential prediction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Marcus on AI.