Agentic systems for breast cancer treatment recommendations

2026-07-13 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Medical AI Applications · Depth: Expert, quick

Summary

Agentic large language model (LLM) systems were evaluated for generating breast cancer treatment recommendations across 72 real clinical cases spanning stages I to IV. Researchers utilized 1,147 case-specific rubrics, created through Asymmetric Information Rubric Generation (AIRG), to assess seven distinct pipelines, including single-LLM baselines, tool-augmented systems, and multi-agent architectures. The top-performing configuration, Claude Opus 4.8 with the D&C+SA pipeline, achieved a global score of 0.594 ± 0.025. While tool use and increased agent autonomy showed mixed effects on performance, significant variations were observed across clinical domains and disease stages. Oncologist-led error analysis identified persistent clinically relevant failures, such as incorrect or missing recommendations, flawed justifications, citation errors, outdated claims, and overconfidence, indicating these systems are not yet suitable for unsupervised clinical application.

Key takeaway

For AI Engineers developing clinical decision support systems, recognize that while agentic LLMs can generate relevant breast cancer recommendations, their current reliability is insufficient for unsupervised use. You must implement robust human oversight and rigorous, domain-specific validation, like the AIRG method, to mitigate persistent errors such as incorrect justifications and overconfidence. Prioritize safety and accuracy over full automation in sensitive medical applications.

Key insights

Agentic LLMs can generate relevant breast cancer recommendations but are not yet safe for unsupervised clinical use.

Principles

• LLM reliability in oncology planning is unclear.
• Agent autonomy effects are mixed.
• Performance varies by clinical domain.

Method

Evaluated agentic LLM systems using 72 real clinical cases and 1,147 AIRG-generated rubrics, comparing single-LLM, tool-augmented, and multi-agent pipelines.

In practice

• Use AIRG for robust LLM evaluation.
• Consider domain-specific performance variations.
• Prioritize human oversight for clinical LLM use.

Topics

• Agentic LLM Systems
• Breast Cancer Treatment
• Clinical Decision Support
• Large Language Models
• AIRG Evaluation
• Oncology AI

Best for: AI Scientist, Research Scientist, AI Engineer

Related on AIssential

• Agentic systems for breast cancer treatment recommendations — Agentic LLMs show promise for breast cancer recommendations but require expert oversight due to persistent clinical errors.
• Please don’t trust your chatbot for medical advice — LLMs are unreliable for medical advice, frequently generating misinformation with overconfidence and lacking clinical reasoning.
• An Agentic LLM-Based Framework for Population-Scale Mental Health Screening — An agentic LLM framework enhances mental health screening by ensuring robust, adaptable, and trustworthy AI pipelines.
• MedAct: Removing the Human Bottleneck in Benchmarking Clinical LLM Safety — MedAct provides a scalable, open-source method for action-level clinical LLM safety benchmarking using synthetically generated cases.
• Clinical decision support in hematological malignancies using a case-grounded AI agent — A case-grounded LLM agent significantly improves clinical decision support in hematological malignancies by integrating diverse knowledge sources.
• Agentic Radiotherapy Planning: Automating Outer-Loop Tuning via TextGrad — Agentic Radiotherapy Planning automates outer-loop tuning using TextGrad for enhanced medical treatment precision.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.