Risks and Limitations of AI in the Life Sciences

2026-03-16 · Source: Answer.AI - Practical AI R&D – Answer.AI · Field: Science & Research — Life Sciences & Biology, Health & Medical Research, Research Methodology & Innovation · Depth: Intermediate, extended

Summary

Rachel Thomas, an AI ethics expert with a Masters in Microbiology-Immunology, highlights critical risks and limitations of AI in life sciences, emphasizing that confidence often outpaces scientific understanding. She notes an overreliance on existing data like patient records, arguing for more investment in new assays and biomarkers, citing AlphaFold's dependence on decades-old, curated datasets like the Protein Data Bank and CASP. Thomas details a Nature Communications paper that used 22 million enzymes to predict function, where a microbiologist later uncovered hundreds of errors, including 135 non-novel enzymes and implausible biological claims, underscoring the challenge of validating AI without deep domain expertise. She also discusses "data cascades" from poor data collection and communication, and how large, biased datasets, like the Zoe app's COVID tracking data, can lead to misleading research on conditions like Long COVID. The discussion stresses the need for fundamental changes, including slower, more deliberate development and robust cross-disciplinary collaboration.

Key takeaway

For Directors of AI/ML overseeing life science applications, you must prioritize rigorous data quality and deep domain expert integration over rapid deployment. Recognize that scaling biased or incomplete datasets, like those from underdiagnosed diseases, creates self-reinforcing errors. Fund foundational bench science and causal mechanism research, and ensure incentives reward thorough error-checking and data work, not just novel model development, to prevent costly and potentially harmful scientific misdirection.

Key insights

AI in life sciences is often undermined by poor data quality, insufficient domain expertise, and misaligned research incentives.

Principles

• Data quality and type fundamentally limit AI results.
• Deep domain expertise is crucial for AI model evaluation.
• Incentives must reward error-checking and data work.

In practice

• Invest in new assays and biomarkers, not just existing data.
• Implement "Data Sheets for Datasets" for data transparency.
• Integrate domain experts at all stages of AI development.

Topics

• Life Sciences AI
• AI Ethics
• Data Quality
• Domain Expertise
• Model Evaluation
• Research Incentives

Best for: AI Scientist, Research Scientist, AI Ethicist, Director of AI/ML

Related on AIssential

• Why AI can’t be trusted to write scientific reviews — Current AI tools are not ready for high-stakes scientific systematic reviews due to limitations in context, nuance, and transparency.
• AI can design and run thousands of lab experiments without human hands. Humanity isn’t ready for the new risks this brings to biology — AI is rapidly automating biological experimentation, accelerating discovery but also creating significant dual-use biosecurity risks.
• 17 AI Models Tested on REAL Scientific Research — Advanced AI models currently demonstrate limited capability in independent scientific discovery, even with curated data and extensive tools.
• F Cancer — AI's impact on cancer is limited by technosolutionism, market failures, and insufficient biological understanding.
• Dozens of AI disease-prediction models were trained on dubious data — AI models for disease prediction are being trained on potentially fabricated public datasets, risking patient harm.
• If you have enough data and enough compute, general learning machines that “discover” patterns tend to outperform systems where humans try to hard-code expertise. — General learning systems with vast data and compute outperform human-designed expert AI, a "bitter lesson" now impacting surgical AI.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Answer.AI - Practical AI R&D – Answer.AI.