Clinical explainable machine learning models for early identification of patients at risk of hospital-acquired urinary tract infection

Published:March 31, 2023DOI:



      Machine learning (ML) models for early identification of patients at risk of hospital-acquired urinary tract infections (HA-UTI) may enable timely and targeted preventive and therapeutic strategies. However, clinicians are often challenged in the interpretation of the predictive outcomes provided by the ML models, which often reach different performances.


      To train ML models for predicting patients at risk of HA-UTI using available data from electronic health records at the time of hospital admission. We focused on the performance of different ML models and clinical explainability.


      This retrospective study investigated patient data representing 138.560 hospital admissions in the North Denmark Region from 01.01.2017 to 31.12.2018. We extracted 51 health socio-demographic and clinical features in a full dataset and used the χ2 test in addition to expert knowledge for feature selection, resulting in two reduced datasets. Seven different ML models were trained and compared between the three datasets. We applied the SHapley Additive exPlanation (SHAP) method to support population- and patient-level explainability.


      The best-performing ML model was a neural network based on the full dataset, reaching an area under the curve (AUC) of 0.758. The neural network was also the best-performing ML model based on the reduced datasets, reaching an AUC of 0.746. Clinical explainability was demonstrated with a SHAP summary- and forceplot.


      Within 24h of hospital admission, the ML models were able to identify patients at risk of developing HA-UTI, providing new opportunities to develop efficient strategies for the prevention of HA-UTI. Using SHAP, we demonstrate how risk predictions can be explained at individual patient level and for the patient population in general.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Hospital Infection
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. Saint S, Meddings JA, Calfee D, et al. Catheter-associated Urinary Tract Infection and the Medicare Rule Changes. 2009;150:877–884.

        • Saint S.
        Clinical and economic consequences of nosocomial catheter-related bacteriuria.
        Am J Infect Control. 2000; 28: 68-75
      2. ECDC, EMEA. The bacterial challenge: time to react. 2009.

        • Conway L.J.
        • Carter E.J.
        • Larson E.L.
        Risk Factors for Nosocomial Bacteremia Secondary to Urinary Catheter-Associated Bacteriuria: A Systematic Review.
        Urol Nurs. 2015; 35: 191-203
        • Zingg W.
        • Holmes A.
        • Dettenkofer M.
        • et al.
        Hospital organisation, management, and structure for prevention of health-care-associated infection: A systematic review and expert consensus.
        Lancet Infect Dis. 2015; 15: 212-224
        • World Health Organisation
        WHO guidelines on Hand Hygiene in Health Care.
        World Heal Organ. 2017; 30: 64
        • Andreessen L.
        • Wilde M.H.
        • Herendeen P.
        Preventing catheter-associated urinary tract infections in acute care: The bundle approach.
        J Nurs Care Qual. 2012; 27: 209-217
        • Gould I.M.
        Antibiotic policies to control hospital-acquired infection.
        J Antimicrob Chemother. 2008; 61: 763-765
        • Møller J.K.
        • Sørensen M.
        • Hardahl C.
        Prediction of risk of acquiring urinary tract infection during hospital stay based on machine-learning: A retrospective cohort study.
        PLoS One. 2021; 16: 1-16
        • Mitchell B.G.
        • Ferguson J.K.
        • Anderson M.
        • et al.
        Length of stay and mortality associated with healthcare-associated urinary tract infections: A multi-state model.
        J Hosp Infect. 2016; 93: 92-99
      3. Park JI, Bliss DZ, Chi C-L, et al. Knowledge Discovery With Machine Learning for HospitalAcquired Catheter-Associated Urinary Tract Infections. 2020;38:28–35.

        • Wiens J.
        • Saria S.
        • Sendak M.
        • et al.
        Do no harm: a roadmap for responsible machine learning for health care.
        Nat Med. 2019; 25
      4. Stenwig E, Salvi G, Rossi PS, et al. Comparative analysis of explainable machine learning prediction models for hospital mortality. 2022;1–14.

      5. Shaban-Nejad A, Michalowski M, Buckeridge DL, et al. Explainable AI in Healthcare and Medicine. 2020.

        • Vellido A.
        The importance of interpretability and visualization in machine learning for applications in medicine and health care.
        Neural Comput Appl. 2020; 32: 18069-18083
        • Stiglic G.
        • Kocbek P.
        • Fijacko N.
        • et al.
        Interpretability of machine learning-based prediction models in healthcare.
        Wiley Interdiscip Rev Data Min Knowl Discov. 2020; 10: 1-13
        • Ahmad M.A.
        • Teredesai A.
        • Eckert C.
        Interpretable machine learning in healthcare.
        Proc - 2018 IEEE Int Conf Healthc Informatics, ICHI 2018. 2018; : 447
        • Burkart N.
        • Huber M.F.
        A survey on the explainability of supervised machine learning.
        J Artif Intell Res. 2020; 70: 245-317
        • Molnar C.
        • Casalicchio G.
        • Bischl B.
        Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges.
        Commun Comput Inf Sci. 2020; 1323: 417-431
        • Deshmukh F.
        • Merchant S.S.
        Explainable Machine Learning Model for Predicting GI Bleed Mortality in the Intensive Care Unit.
        Am J Gastroenterol. 2020; 115: 1657-1668
        • Lundberg S.M.
        • Lee S.I.
        A unified approach to interpreting model predictions.
        Adv Neural Inf Process Syst. 2017; (2017-Decem): 4766-4775
        • Pollard T.J.
        • Johnson A.E.W.
        • Raffa J.D.
        • et al.
        The eICU collaborative research database, a freely available multi-center database for critical care research.
        Sci Data. 2018; 5: 1-13
        • Condell O.
        • Gubbels S.
        • Nielsen J.
        • et al.
        Automated surveillance system for hospital-acquired urinary tract infections in Denmark.
        J Hosp Infect. 2016; 93: 290-296
        • Gubbels S.
        The development of a national surveillance system for hospital-acquired infections in Denmark The Hospital Acquired Infections Database.
        HAIBA. 2016;
        • Sundararajan V.
        • Henderson T.
        • Perry C.
        • et al.
        New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality.
        J Clin Epidemiol. 2004; 57: 1288-1294
      6. Hansen BV. Acute admissions to internal medicine departments in Denmark - studies on admission rate , diagnosis , and prognosis - PhD Dissertation. 2014;

        • Christensen D.H.
        • Veres K.
        • Ording A.G.
        • et al.
        Risk of cancer in patients with thyroid disease and venous thromboembolism.
        Clin Epidemiol. 2018; 10: 907-915
        • Nielsen P.R.
        • Benros M.E.
        • Mortensen P.B.
        Hospital contacts with infection and risk of schizophrenia: A population-based cohort study with linkage of danish national registers.
        Schizophr Bull. 2014; 40: 1526-1532
        • Iguyon I.
        • Elisseeff A.
        An introduction to variable and feature selection.
        J Mach Learn Res. 2003; 3: 1157-1182
        • Pearson K.
        On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling.
        London, Edinburgh, Dublin Philos Mag J Sci. 1900; 50: 157-175
        • Pedregosa F.
        • Varoquaux G.
        • Thirion A.G.V.M.B.
        Scikit-learn: Machine Learning in Python.
        J ofMachine Learn Res. 2011; 12: 2825-2830
        • King C.
        • Garcia Alvarez L.
        • Holmes A.
        • et al.
        Risk factors for healthcare-associated urinary tract infection and their applications in surveillance using hospital administrative data: A systematic review.
        J Hosp Infect. 2012; 82: 219-226
        • Redder J.D.
        • Leth R.A.
        • Møller J.K.
        Analysing risk factors for urinary tract infection based on automated monitoring of hospital-acquired infection.
        J Hosp Infect. 2016; 92: 397-400
        • Aloush S.M.
        • Al Qadire M.
        • Assmairan K.
        • et al.
        Risk factors for hospital-acquired non-catheter-associated urinary tract infection.
        J Am Assoc Nurse Pract. 2019; 31: 747-751
        • Zhang Q.
        • Shi B.
        • Wu L.
        Characteristics and risk factors of urinary tract infection in patients with HBV-related acute-on-chronic liver failure: A retrospective study.
        Med (United States). 2022; 101E29913
        • Yoshida B.
        • Nguyen A.
        • Formanek B.
        • et al.
        Hypoalbuminemia and Elevated CRP are Risk Factors for Deep Infections and Urinary Tract Infections After Lumbar Spine Surgery in a Large Retrospective Patient Population.
        Glob Spine J. 2021; 13: 33-44
        • Rudin C.
        Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.
        Nat Mach Intell. 2019; 1: 206-215
      7. Breiman L. Classification and regression trees. Breiman L, editor. Boca Raton, Fla: Chapman & Hall; 1984.

        • Wiedermann C.J.
        Hypoalbuminemia as surrogate and culprit of infections.
        Int J Mol Sci. 2021; 22
        • Storme O.
        • Saucedo J.T.
        • Garcia-Mora A.
        • et al.
        Risk factors and predisposing conditions for urinary tract infection.
        Ther Adv Urol. 2019; 11: 19-28
        • Scherberich J.E.
        • Fünfstück R.
        • Naber K.G.
        Urinary tract infections in patients with renal insufficiency and dialysis - epidemiology, pathogenesis, clinical symptoms, diagnosis and treatment.
        GMS Infect Dis. 2021; 9: Doc07
        • Delgado A.
        • Cordero G.-G.E.
        • Marcos S.
        • et al.
        Influence of cognitive impairment on mortality, complications and functional outcome after hip fracture: Dementia as a risk factor for sepsis and urinary infection.
        Injury. 2020; 51: S19-S24
        • Wang M.C.
        • Tseng C.C.
        • Wu A.B.
        • et al.
        Bacterial characteristics and glycemic control in diabetic patients with Escherichia coli urinary tract infection.
        J Microbiol Immunol Infect. 2013; 46: 24-29
        • Lauritsen S.M.
        • Kristensen M.
        • Olsen M.V.
        • et al.
        Explainable artificial intelligence model to predict acute critical illness from electronic health records.
        Nat Commun. 2020; 11: 1-11
        • Ribeiro M.T.
        • Singh S.
        • Guestrin C.
        Why Should I Trust You?” Explaining the Predictions of Any Classifier.
        NAACL-HLT 2016 - 2016 Conf North Am Chapter Assoc Comput Linguist Hum Lang Technol Proc Demonstr Sess. 2016; : 97-101
        • Sips M.E.
        • Bonten M.J.M.
        • Van Mourik M.S.M.
        Automated surveillance of healthcare-associated infections: State of the art.
        Curr Opin Infect Dis. 2017; 30: 425-431