How AI can lead to false arrests and wrongful convictions

2026-05-11 · Source: Artificial intelligence (AI) – The Conversation · Field: Technology & Digital — Artificial Intelligence & Machine Learning, AI in Public Safety & Justice · Depth: Intermediate, long

Summary

On October 20, 2025, a 17-year-old student in Baltimore, Taki Allen, was handcuffed at gunpoint after an AI-enhanced surveillance camera misidentified a Doritos bag in his pocket as a gun. Similarly, on December 24, 2025, Angela Lipps, a Tennessee grandmother, was jailed for five months due to facial recognition software incorrectly linking her to fraud in North Dakota. These incidents highlight a critical issue: AI systems produce probabilistic predictions, which humans often misinterpret as certainties, leading to severe real-world consequences. Researchers studying the intersection of technology, law, and public administration emphasize that this shift from probabilistic prediction to operational certainty is common in AI policing tools, which are used in dozens of U.S. cities to score neighborhoods and route officers based on predicted risk.

Key takeaway

For CTOs and VPs of Engineering deploying AI systems, it is crucial to implement clear policies that mandate human verification for all probabilistic AI outputs, especially in high-stakes applications like law enforcement. Your teams must design systems that explicitly communicate uncertainty and educate end-users on the difference between statistical likelihoods and factual certainties to prevent misinterpretation and mitigate the risk of severe human error.

Key insights

AI systems generate probabilities, not certainties, a distinction often lost in human operational decisions.

Principles

• AI outputs are probabilistic, not factual.
• Confidence thresholds shape AI system alerts.
• Human interpretation of AI outputs is critical.

Method

AI systems use historical data to generate statistical risk scores or heat maps, which inform deployment decisions, often losing the underlying uncertainty.

In practice

• Design systems to admit uncertainty.
• Educate users on interpreting AI outputs.
• Calibrate diagnostic tools based on harm trade-offs.

Topics

• AI Policing
• Facial Recognition
• Predictive Policing
• Algorithmic Uncertainty
• Confidence Thresholds

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Ethicist, Legal Professional, Policy Maker

Related on AIssential

• Police AI chief admits crime-fighting tech will have bias but vows to tackle it | AI (artificial intelligence) - The Guardian — Police AI will contain bias, necessitating robust mitigation strategies and human oversight for fair outcomes.
• Man used AI to make false statements to shut down London nightclub, police say — AI-generated false complaints are an emerging threat to regulatory processes and public administration.
• The most important AI failure may be false confidence, not wrong answers — AI's greatest risk is confident action based on flawed data, not just wrong answers.
• As AI accuracy increases, humans may become increasingly dependent on algorithmic “proactive conclusions,” potentially leading to the atrophy of “trained judgment”,... — Proactive AI systems can influence human decisions, necessitating ethical governance to preserve agency and mitigate risks.
• AI Doesn’t Need to Be Right. It Only Needs to Sound Procedural — AI's procedural language masks uncertainty, enabling policy formation without visible accountability beyond mere hallucination.
• Wrongful Arrest Exposes Failures in One of the Oldest Police Face-Recognition Tools in the US — Inaccurate face recognition tools can lead to wrongful arrests and legal challenges, even with established systems.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial intelligence (AI) – The Conversation.