Agentic systems for breast cancer treatment recommendations

2026-04-23 · Source: cs.CL updates on arXiv.org · Field: Health & Wellbeing — Artificial Intelligence & Machine Learning, Clinical Care & Medical Practice, Health & Medical Research · Depth: Expert, extended

Summary

A study evaluated agentic large language model (LLM) systems for generating breast cancer treatment recommendations using 72 real clinical cases across stages I to IV. Researchers compared seven pipelines, including single-LLM baselines, tool-augmented systems, and multi-agent architectures with fact-checking and autonomous subagent spawning. The evaluation utilized 1,147 case-specific rubrics generated through Asymmetric Information Rubric Generation (AIRG). The top-performing configuration, Claude Opus 4.8 with the D&C+SA pipeline, achieved a global score of 0.594 ± 0.025. The impact of tool use and increased agent autonomy was mixed, showing both improvements and degradations in performance. System performance varied significantly by clinical domain, with systemic therapy scoring highest, and by disease stage, with stage I cases performing best and stage III cases lowest. Oncologist-led error analysis identified critical failures such as incorrect recommendations, flawed justifications, and outdated claims, indicating these systems are not yet suitable for unsupervised clinical use.

Key takeaway

For AI Scientists developing clinical decision support, recognize that agentic LLMs for breast cancer recommendations, despite promising scores, are not ready for unsupervised use. You must implement robust human-in-the-loop review processes. Prioritize stratified evaluation by disease stage and clinical domain to uncover hidden weaknesses. Your system designs should empirically validate tool use and multi-agent architectures, as complexity does not guarantee improved performance or safety.

Key insights

Agentic LLMs show promise for breast cancer recommendations but require expert oversight due to persistent clinical errors.

Principles

• Increased LLM system complexity does not guarantee performance gains.
• Tool access alone is insufficient; models need to effectively use tools.
• Optimal agentic architecture is model- and task-dependent.

Method

Asymmetric Information Rubric Generation (AIRG) uses a frontier LLM with privileged access to real clinical decisions and search tools to generate case-specific evaluation rubrics, mitigating expert cost.

In practice

• Stratify LLM evaluation by clinical stage and domain.
• Audit automated LLM evaluations with expert error analysis.
• Design fact-checkers with full reasoning trace access.

Topics

• Agentic LLM Systems
• Breast Cancer Treatment
• Clinical Decision Support
• Asymmetric Information Rubric Generation
• Medical AI Evaluation
• Multi-agent Architectures

Code references

• GRUPOMED4U/breast_cancer_agents_paper
• aeghnnsw/pubmed-mcp

Best for: AI Product Manager, AI Scientist, Research Scientist, Director of AI/ML

Related on AIssential

• Agentic systems for breast cancer treatment recommendations — Agentic LLMs can generate relevant breast cancer recommendations but are not yet safe for unsupervised clinical use.
• An Agentic LLM-Based Framework for Population-Scale Mental Health Screening — An agentic LLM framework enhances mental health screening by ensuring robust, adaptable, and trustworthy AI pipelines.
• Information-seeking failures of large language models in agentic clinical reasoning — LLMs fail in clinical reasoning due to poor information-seeking under uncertainty, not insufficient medical knowledge.
• Multi-modal AI for comprehensive breast cancer prognostication — A multi-modal AI test significantly improves breast cancer prognostication by integrating pathology and clinical data.
• A Multi-Agent Open-Source LLM for Structured Cancer Registry Information Extraction from Pathology and Medical Reports — A modular multi-agent LLM framework significantly improves context-dependent cancer registry data extraction from medical reports.
• F Cancer — AI's impact on cancer is limited by technosolutionism, market failures, and insufficient biological understanding.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.