How effective are ML algorithms in ICU mortality predictions?

Testing two machine learning algorithms — extra trees and gradient boosting — a team including Australian researchers set out to find out.

A variety of models are used by clinicians to estimate ICU mortality. These include the Acute Physiology and Chronic Health Evaluation (APACHE) or the Simplified Acute Physiology Score (SAPS). However, these tools are limited in their capacity to capture evolving patient conditions, take time to validate and need frequent recalibration.

With algorithms outperforming traditional scoring systems like APACHE and SAPS, the effectiveness of machine learning (ML) predictions in ICU settings is well-documented; however, the unexplainable results of MLs have hindered adoption. That’s why a team of international academics, including from Australian Catholic University (ACU) and Charles Darwin University (CDU), applied further analysis to explain predictions.

Extra trees (ET) and gradient boosting (GB) were two algorithms analysed and found to have accuracies of 98.33% and 98.23%, respectively, in predicting the mortality of ICU patients and then deciding what conditions affected mortality. To understand the key factors to mortality decisions — and how well the algorithms aligned with medical knowledge — ET’s results, with its higher accuracy, were also fed into explanation models, which found hypertension, tumours, endocrine disease, digestive disease and cardiovascular disease to be key factors in ET’s mortality predictions.

“These systems can assist clinicians in identifying high-risk patients who require urgent attention or targeted interventions,” said lead author and CDU Adjunct Professor Niusha Shafiabady, who is Head of Discipline for IT and the Director of Women in AI for Social Good lab at Australian Catholic University.

“Such systems enable continuous monitoring of at-risk patients, supporting proactive care and early intervention to prevent deterioration or adverse events,” Shafiabady added. “By embedding interpretable findings into clinical decision-support systems, this study supports the advancement of ML tools that are both accurate and clinically meaningful, ensuring they complement rather than complicate frontline healthcare delivery.”

The study — conducted alongside researchers from University of Technology Sydney, Western Sydney University, University of New England and Amirkabir University of Technology in Tehran — was published open access in BMJ Health & Care Informatics (doi: 10.1136/bmjhci-2024-101406). Using the models and algorithms with larger datasets and from varying healthcare settings are among the future avenues of research.

Image credit: iStock.com/Pixelimage