Don’t trust Generative AI to do your taxes — and don’t trust it with people’s lives

2025-06-07 · Source: Marcus on AI · Field: Government & Public Sector — Digital Government & E-Government, Public Policy & Governance, Public Safety & Security · Depth: Fundamental Awareness, quick

Summary

A New York Times article by Stuart Thompson on March 5, 2026, highlights significant concerns regarding the application of artificial intelligence in critical governmental functions, specifically tax processing and military operations. The report indicates that AI systems, while potentially offering efficiency, are not yet reliable enough for tasks involving sensitive financial data or human lives. It points out that the IRS is exploring AI for tax refunds, despite widespread acknowledgment of AI's current limitations in accuracy and trustworthiness. Furthermore, the article criticizes the consideration of AI for military applications, such as drone operations, given its unsuitability for tax-related decisions. The author emphasizes that deploying AI in such high-stakes scenarios in 2026 is a serious misjudgment.

Key takeaway

For government agencies considering AI integration, particularly for critical functions like tax processing or defense, you must prioritize rigorous validation over perceived efficiency gains. Your teams should conduct thorough, independent audits of AI system accuracy and reliability before any deployment that impacts citizens' financial well-being or national security. Rushing AI into sensitive applications without proven trustworthiness introduces unacceptable risks.

Key insights

AI systems are currently unreliable for critical governmental functions like tax processing and military applications.

Principles

• AI reliability is paramount for sensitive applications.
• Untrustworthy AI should not manage human lives.

In practice

• Evaluate AI systems for accuracy before deployment.
• Avoid AI in high-stakes decision-making.

Topics

• AI Ethics
• AI Safety
• Government AI
• Tax Automation
• AI Deployment Risks

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Ethicist, Policy Maker, General Interest

Related on AIssential

• Governments Are Using AI To Draft Legislation. What Could Possibly Go Wrong? — AI is being adopted by governments worldwide to enhance legislative efficiency, but raises significant concerns about legitimacy and trust.
• Code Red for Humanity? — Unreliable Generative AI, if deployed in military decision-making, poses an immediate and catastrophic global risk.
• Is AI Trustworthy? — AI's biggest opportunity lies in building trust for high-stakes enterprise automation, not just in model intelligence.
• The risks of inviting AI into the heart of our economy, society and governance | Letters — The core risk of AI lies in its capacity to blur truth, erode provenance, and diminish human originality and empathy.
• British Police Built a Sprawling Crime-Prediction Machine. Some Results Couldn’t Be Trusted — Predictive policing models demand rigorous, continuous auditing and comprehensive bias testing for reliability and ethical deployment.
• AI Doesn’t Need to Be Right. It Only Needs to Sound Procedural — AI's procedural language masks uncertainty, enabling policy formation without visible accountability beyond mere hallucination.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Marcus on AI.