Benchmarking Machine Learning Architectures for Antimicrobial Stewardship in Pediatric ICUs

2026-05-21 · Source: Machine Learning · Field: Health & Wellbeing — Clinical Care & Medical Practice, Medical Devices & Health Technology, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A systematic benchmarking study evaluated machine learning architectures for antimicrobial stewardship (AMS) intervention prediction in pediatric intensive care units (PICUs). Utilizing both a public dataset and a private institutional cohort, the research defined four proxy targets for reducing antibiotic exposure: intravenous-to-oral switching, de-escalation, discontinuation, and short-course therapy. The study compared tabular, sequence-based, and graph-based temporal models across multiple temporal resolutions. Key findings indicate that predictive performance is primarily influenced by target prevalence and dataset characteristics, rather than model complexity. While sequence models improved precision-recall over tabular approaches at a coarse 24-hour resolution, finer temporal modeling offered limited additional benefit. These gains, however, came with poorer calibration, as simpler tabular models provided more reliable probability estimates. Multi-task learning yielded only marginal improvements, suggesting minimal shared structure among stewardship targets. The findings emphasize the critical role of target design, temporal representation, and calibration in clinical machine learning for pediatric AMS.

Key takeaway

For Machine Learning Engineers developing decision support systems for pediatric antimicrobial stewardship, prioritize robust target design and understanding dataset characteristics over increasing model complexity. Your efforts should focus on ensuring model calibration, as simpler tabular models often provide more reliable probability estimates. While sequence models offer precision-recall improvements at a 24-hour resolution, finer temporal modeling yields diminishing returns, suggesting a practical sweet spot for temporal representation.

Key insights

Machine learning performance for pediatric antimicrobial stewardship is driven more by target design and data characteristics than model complexity.

Principles

• Predictive performance is primarily driven by target prevalence and dataset characteristics.
• Sequence models can improve precision-recall over tabular approaches at coarse temporal resolutions.
• Finer temporal modeling beyond 24-hour resolution provides limited additional benefit.

Method

A systematic benchmarking study compared tabular, sequence-based, and graph-based temporal models for four defined AMS proxy targets (IV-to-oral switching, de-escalation, discontinuation, short-course therapy) under a unified evaluation framework at multiple temporal resolutions.

In practice

• Prioritize target design and temporal representation when developing clinical ML systems.
• Consider sequence models for precision-recall gains at 24-hour resolution in AMS prediction.
• Opt for simpler tabular models when reliable probability calibration is critical.

Topics

• Antimicrobial Stewardship
• Pediatric ICUs
• Machine Learning Benchmarking
• Temporal Modeling
• Model Calibration
• Electronic Health Records

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• v333: Proceedings of CHIL 2026 — Machine learning, particularly LLMs, is rapidly expanding its utility across diverse clinical and administrative healthcare applications.
• Convolutional neural networks quantify antibiotic resistance in Mycobacterium tuberculosis with diagnostic grade accuracy and predict treatment response — CNNs can predict antibiotic MICs from genomic data with clinical accuracy, leveraging diverse biological features.
• Insulin4RL: Real-Time Insulin Management in the Intensive Care Unit for Offline Reinforcement Learning — Realistic offline reinforcement learning in healthcare requires datasets with naturally irregular clinical sampling to improve model generalizability.
• Beyond Classification: A Cough Regression Benchmark for Respiratory Acoustic Foundation Models — Respiratory acoustic foundation models can predict continuous health quantities from cough audio using regression, moving beyond classification.
• MADE: A Living Benchmark for Multi-Label Text Classification with Uncertainty Quantification of Medical Device Adverse Events — MADE is a living MLTC benchmark for medical device adverse events, emphasizing reliable uncertainty quantification.
• Language Models as Interfaces, Not Oracles: A Hybrid LLM-ML System for Pediatric Appendicitis — Hybrid LLM-ML systems improve clinical decision support by using LLMs for interface tasks and ML for stable prediction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.