Carissa Véliz on the dangers of predictive AI

2026-05-28 · Source: Sifted · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, extended

Summary

Carissa Véliz, an AI ethicist and Oxford associate professor, discusses the inherent dangers of predictive AI, as detailed in her book "Prophecy: Prediction, Power, and the Fight for the Future". She argues that modern AI, much like ancient oracles, often serves to reinforce existing power structures by telling the powerful what they want to hear. Véliz highlights that digital technology's "two sins of design"—surveillance and prediction—lead to social control and conflict with democratic principles. She distinguishes between predicting things, which can be beneficial, and predicting people, which is problematic due to the potential for self-fulfilling prophecies and perpetuating historical biases like sexism and racism. The discussion also questions the efficacy of tech giants' self-regulation and emphasizes the critical need for ethical frameworks in AI, particularly in sensitive areas like the justice system, where current machine learning lacks due process and contestability.

Key takeaway

For AI Ethicists and Policy Makers evaluating predictive AI systems, recognize that these tools are not neutral; they embed historical biases and can create self-fulfilling prophecies, especially when applied to human decisions. Prioritize causal reasoning over mere correlation and demand contestability in systems affecting justice. You should advocate for regulations that ensure transparency and due process, moving beyond self-regulation to foster genuinely fair and accountable AI development.

Key insights

Predictive AI, designed for surveillance and prediction, risks social control and perpetuates historical biases, undermining democracy and fairness.

Principles

• Prediction about people influences outcomes, unlike predictions about things.
• Algorithms are conservative, perpetuating past biases from historical data.
• Fairness contexts demand avoiding predictions, which are never facts.

In practice

• Avoid past data for disruptive innovation to prevent perpetuating biases.
• Invest in individuals algorithms would never select for true innovation.
• Use local servers and controlled data environments for privacy.

Topics

• Predictive AI
• AI Ethics
• Algorithmic Bias
• Data Surveillance
• AI Regulation
• Due Process

Best for: AI Ethicist, Policy Maker, Consultant

Related on AIssential

• AI doesn’t create bias, it inherits it – how do we ensure fairness when it comes to automated decisions? — AI fairness is a complex, context-dependent, and evolving challenge, not a purely technical problem.
• As AI accuracy increases, humans may become increasingly dependent on algorithmic “proactive conclusions,” potentially leading to the atrophy of “trained judgment”,... — Proactive AI systems can influence human decisions, necessitating ethical governance to preserve agency and mitigate risks.
• The Internet of Probability — LLMs transform the internet into a probability machine, eroding truth and provenance through statistical generation and contaminated training data.
• Why AI predictions are so hard — Predicting AI's future is challenging due to LLM intelligence plateaus, public unpopularity, and regulatory confusion.
• Are humans being left behind in the artificial intelligence push? - Pittsburgh Post-Gazette — AI's rapid evolution necessitates independent auditing and human agency to mitigate bias and ensure transparency.
• Bias in the machine (edited) — Bias in AI systems stems from historical human prejudices embedded in training data and model design, leading to real-world harm.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Sifted.