Using machine learning to predict paediatric 30-day unplanned hospital readmissions: a case-control retrospective analysis of medical records, including written discharge documentation
Huaqiong Zhou A B , Matthew A. Albrecht B , Pamela A. Roberts B , Paul Porter B C and Philip R. Della B D EA General Surgical Ward, Princess Margaret Hospital for Children, Perth, WA 6008, Australia.
B School of Nursing, Curtin University, GPO Box U 1987, Perth, WA 6845, Australia. Email address: h.zhou@curtin.edu.au; matthew.albrecht@curtin.edu.au; p.a.roberts@curtin.edu.au; paul.porter@curtin.edu.au
C Joondalup Health Campus, Joondalup, WA 6027, Australia.
D Visiting Professor, College of Nursing, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China.
E Corresponding author. Email: p.della@curtin.edu.au
Australian Health Review 45(3) 328-337 https://doi.org/10.1071/AH20062
Submitted: 14 April 2020 Accepted: 18 June 2020 Published: 12 April 2021
Journal Compilation © AHHA 2021 Open Access CC BY-NC-ND
Abstract
Objectives To assess whether adding clinical information and written discharge documentation variables improves prediction of paediatric 30-day same-hospital unplanned readmission compared with predictions based on administrative information alone.
Methods A retrospective matched case-control study audited the medical records of patients discharged from a tertiary paediatric hospital in Western Australia (WA) between January 2010 and December 2014. A random selection of 470 patients with unplanned readmissions (out of 3330) were matched to 470 patients without readmissions based on age, sex, and principal diagnosis at the index admission. Prediction utility of three groups of variables (administrative, administrative and clinical, and administrative, clinical and written discharge documentation) were assessed using standard logistic regression and machine learning.
Results Inclusion of written discharge documentation variables significantly improved prediction of readmission compared with models that used only administrative and/or clinical variables in standard logistic regression analysis (χ2 17 = 29.4, P = 0.03). Highest prediction accuracy was obtained using a gradient boosted tree model (C-statistic = 0.654), followed closely by random forest and elastic net modelling approaches. Variables highlighted as important for prediction included patients’ social history (legal custody or patient was under the care of the Department for Child Protection), languages spoken other than English, completeness of nursing admission and discharge planning documentation, and timing of issuing discharge summary.
Conclusions The variables of significant social history, low English language proficiency, incomplete discharge documentation, and delay in issuing the discharge summary add value to prediction models.
What is known about the topic? Despite written discharge documentation playing a critical role in the continuity of care for paediatric patients, limited research has examined its association with, and ability to predict, unplanned hospital readmissions. Machine learning approaches have been applied to various health conditions and demonstrated improved predictive accuracy. However, few published studies have used machine learning to predict paediatric readmissions.
What does this paper add? This paper presents the findings of the first known study in Australia to assess and report that written discharge documentation and clinical information improves unplanned rehospitalisation prediction accuracy in a paediatric cohort compared with administrative data alone. It is also the first known published study to use machine learning for the prediction of paediatric same-hospital unplanned readmission in Australia. The results show improved predictive performance of the machine learning approach compared with standard logistic regression.
What are the implications for practitioners? The identified social and written discharge documentation predictors could be translated into clinical practice through improved discharge planning and processes, to prevent paediatric 30-day all-cause same-hospital unplanned readmission. The predictors identified in this study include significant social history, low English language proficiency, incomplete discharge documentation, and delay in issuing the discharge summary.
Keywords: administrative data, clinical information, discharge planning, discharge summary, follow-up plan, machine learning, medical records, paediatric hospital readmissions, paediatric unplanned readmissions, retrospective analysis, social history, social predictors, written discharge documentation.
References
[1] Zhou H, Roberts PA, Dhaliwal SA, Della PR. Risk factors associated with paediatric unplanned hospital readmissions: a systematic review. BMJ Open 2019; 9 e020554| Risk factors associated with paediatric unplanned hospital readmissions: a systematic review.Crossref | GoogleScholarGoogle Scholar | 30696664PubMed |
[2] Wijlaars LP, Hardelid P, Woodman J, Allister J, Cheung R, Gilbert R. Who comes back with what: a retrospective databse study on reasons for emergency readmission to hospital in children and young people in England. Arch Dis Child 2016; 101 714–8.
| Who comes back with what: a retrospective databse study on reasons for emergency readmission to hospital in children and young people in England.Crossref | GoogleScholarGoogle Scholar | 27113555PubMed |
[3] Minhas SV, Chow I, Feldman DS, Bosco J, Otsuka NY. A predictive risk index for 30-day readmissions following surgical treatment of pediatric scoliosis. J Pediatr Orthop 2016; 36 187–92.
| A predictive risk index for 30-day readmissions following surgical treatment of pediatric scoliosis.Crossref | GoogleScholarGoogle Scholar | 25730378PubMed |
[4] Feng JY, Toomey SL, Zaslavsky AM, Nakamura MM, Schuster MA. Readmissions after pediatric mental health admissions. Pediatrics 2017; 140 e20171571
| Readmissions after pediatric mental health admissions.Crossref | GoogleScholarGoogle Scholar | 29101224PubMed |
[5] Topal E, Gucenmez OA, Harmanci K, Arga M, Derinoz O, Turktas I. Potential predictors of relapse after treatment of asthma exacerbations in children. Ann Allergy Asthma Immunol 2014; 112 361–4.
| Potential predictors of relapse after treatment of asthma exacerbations in children.Crossref | GoogleScholarGoogle Scholar | 24583137PubMed |
[6] Coller RJ, Klitzner TS, Lerner CF, Chung PJ. Predictors of 30-day readmission and association with primary care follow-up plans. J Pediatr 2013; 163 1027–33.
| Predictors of 30-day readmission and association with primary care follow-up plans.Crossref | GoogleScholarGoogle Scholar | 23706518PubMed |
[7] Choudhry AJ, Baghdadi YMK, Wagie AE, Habermann EB, Cullinane DC, Zielinski MD. Readability of discharge summaries: with what level of information are we dismissing our patients? Am J Surg 2016; 211 631–36.
| Readability of discharge summaries: with what level of information are we dismissing our patients?Crossref | GoogleScholarGoogle Scholar | 26794665PubMed |
[8] Coghlin DT, Leyenaar JK, Shen M, Bergert L, Engel R, Hershey D, Mallory L, Rassbach C, Woehrlen T, Cooperberg D. Pediatric discharge content: a multisite assessment of physician preferences and experiences. Hosp Pediatr 2014; 4 9–15.
| Pediatric discharge content: a multisite assessment of physician preferences and experiences.Crossref | GoogleScholarGoogle Scholar | 24435595PubMed |
[9] Olsen MR, Hellzen O, Skotnes LH, Enmarker I. Content of nursing discharge notes: associations with patient and transfer characteristics. Open Nurs J 2012; 2 277–87.
| Content of nursing discharge notes: associations with patient and transfer characteristics.Crossref | GoogleScholarGoogle Scholar |
[10] Artetxe A, Beristain A, Graña M. Predictive models for hospital readmission risk: a systematic review of methods. Comput Methods Programs Biomed 2018; 164 49–64.
| Predictive models for hospital readmission risk: a systematic review of methods.Crossref | GoogleScholarGoogle Scholar | 30195431PubMed |
[11] Jovanovic M, Radovanovic S, Vukicevic M, Pouke SV, Delibasic B. Building interpretable predictive models for pediatric hospital readmission using Tree-Lasso logistic regression. Artif Intell Med 2016; 72 12–21.
| Building interpretable predictive models for pediatric hospital readmission using Tree-Lasso logistic regression.Crossref | GoogleScholarGoogle Scholar | 27664505PubMed |
[12] Stiglic G, Wang F, Davey A, Obradovic Z. Pediatric readmission classification using stacked regularized logistic regression models. AMIA Annual Symp Proc 2014; 2014 1072–81.
[13] Wolff P, Grana M, Rios SA, Yarza MB. Machine learning readmission risk modeling: a pediatric case study. BioMed Res Int 2019; 2019 8532892
| Machine learning readmission risk modeling: a pediatric case study.Crossref | GoogleScholarGoogle Scholar | 31139655PubMed |
[14] Janjua MB, Reddy S, Samdani AF, Welch WC, Ozturk AK, Price AV, Weprin BE, Swift DM. Predictors of 90-day readmission in children undergoing spinal cord tumor surgery: a nationwide readmissions database analysis. World Neurosurg 2019; 127 e697–706.
| Predictors of 90-day readmission in children undergoing spinal cord tumor surgery: a nationwide readmissions database analysis.Crossref | GoogleScholarGoogle Scholar | 30947001PubMed |
[15] Wiens J, Shenoy E. Machine learning for healthcare: on the verge of a major shift in healthcare epidemiology. Clin Infect Dis 2018; 66 149–53.
| Machine learning for healthcare: on the verge of a major shift in healthcare epidemiology.Crossref | GoogleScholarGoogle Scholar | 29020316PubMed |
[16] Frizzell JD, Liang L, Schulte PJ, Yancy CW, Heidenreich PA, Hernandez AF, Bhatt DL, Fonarow GC, Laskey WK. Prediction of 30-day all-cause readmissions in patients hospitalized for heart failure: comparison of machine learning and other statistical approaches. JAMA Cardiol 2017; 2 204–209.
| Prediction of 30-day all-cause readmissions in patients hospitalized for heart failure: comparison of machine learning and other statistical approaches.Crossref | GoogleScholarGoogle Scholar | 27784047PubMed |
[17] Yang C, Delcher C, Shenkman E, Ranka S. Predicting 30-day all-cause readmissions from hospital inpatient discharge data. In 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), 14–16 September 2016, Munich, Germany. IEEE; 2016
[18] Golas SB, Shibahara T, Agboola S, Otaki H, Sato J, Nakae T, Hisamitsu T, Kojima G, Felsted J, Kakarmath S, Kvedar J, Jethwani K. A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform Decis Mak 2018; 18 44
| A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data.Crossref | GoogleScholarGoogle Scholar | 29929496PubMed |
[19] Zhou H, Della P, Porter P, Roberts P. Risk factors associated with 30-day all-cause unplanned hospital readmissions at a tertiary children’s hospital in Western Australia. J Paediatr Child Health 2020; 56 524–46.
| Risk factors associated with 30-day all-cause unplanned hospital readmissions at a tertiary children’s hospital in Western Australia.Crossref | GoogleScholarGoogle Scholar |
[20] Child and Adolescent Health Service. History and design: Princess Margaret Hospital. Available at: https://pch.health.wa.gov.au/About-us/History/Princess-Margaret-Hospital [verified 16 February 2021].
[21] Zhou H, Della P, Roberts P, Porter P, Dhaliwal S. A 5-year retrospective cohort study of unplanned readmissions in an Australian tertiary paediatric hospital. Aust Health Rev 2019; 43 662–71.
| A 5-year retrospective cohort study of unplanned readmissions in an Australian tertiary paediatric hospital.Crossref | GoogleScholarGoogle Scholar | 30369393PubMed |
[22] Blackwell M, Iacus S, King GP. G. cem: coarsened exact matching in Stata. Stata J 2009; 9 524–46.
| G. cem: coarsened exact matching in Stata.Crossref | GoogleScholarGoogle Scholar |
[23] Beck CE, Khambalia A, Parkin PC, Raina P, Macarthur C. Day of discharge and hospital readmission rates within 30 days in children: a population-based study. Paediatr Child Health 2006; 11 409–12.
| Day of discharge and hospital readmission rates within 30 days in children: a population-based study.Crossref | GoogleScholarGoogle Scholar | 19030310PubMed |
[24] Berry J, Hall DE, Kuo DZ, Cohen E, Agrawal R, Feudtner C, Hall M, Kueser J, Kaplan W, Neff J. Hospital utilization and characteristics of patients experiencing recurrent readmissions within children’s hospital. JAMA 2011; 305 682–90.
| Hospital utilization and characteristics of patients experiencing recurrent readmissions within children’s hospital.Crossref | GoogleScholarGoogle Scholar | 21325184PubMed |
[25] Schlesselman JJ, Stolley PD. Case-control studies: design, conduct, analysis. New York: Oxford University Press; 1982.
[26] Stekhoven DJ, Bühlmann P. MissForest–non-parametric missing value inputation for mixed-type data. Bioinformatics 2012; 28 112–8.
| MissForest–non-parametric missing value inputation for mixed-type data.Crossref | GoogleScholarGoogle Scholar | 22039212PubMed |
[27] R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical computing; 2018. Available at: https://www.R-project.org/ [verified 20 October 2020].
[28] Kuhn M. The caret package. 2019. Available at: https://cran.r-project.org/web/packages/caret/index.html [verified 12 Ocober 2020].
[29] Tibshirani R. Regression shrinkage and selection via the Lasso. JSTOR. Series B (Methodological) 1996; 58 267–88. [verified 12 October 2020] https://www.jstor.org/stable/2346178
[30] Zou H, Hastie T. Regularization and variable selection via the elastic net. JSTOR. Series B (Methodological) 2005; 67 301–20. [verified 12 October 2020] https://www.jstor.org/stable/3647580?seq=1
[31] Liaw A, Wiener M. Classification and regression by randomForest. R News 2002; 2/3 18–22. [verified 12 October 2020] https://cogns.northwestern.edu/cbmg/LiawAndWiener2002.pdf
[32] Chen T, He T. xgboost: eXtreme gradient boosting. Package version 1.2.0.1. 2020. Available at: https://cran.r-project.org/web/packages/xgboost/vignettes/xgboost.pdf [verified 1 May 2019].
[33] Feudtner C, Levin JE, Srivastava R, Goodman DM, Slonim AD, Sharma V, Shah SS, Pati S, Fargason C, Hall M. How well can hospital readmission be predicted in a cohort of hospitalized children? A retrospective, multicenter study. Pediatrics 2009; 123 286–93.
| How well can hospital readmission be predicted in a cohort of hospitalized children? A retrospective, multicenter study.Crossref | GoogleScholarGoogle Scholar | 19117894PubMed |
[34] Sacks JH, Kelleman M, McCracken C, Glanville M, Oster M. Pediatric cardiac readmissions: an opportunity for quality improvement? Congenit Heart Dis 2017; 12 282–8.
| Pediatric cardiac readmissions: an opportunity for quality improvement?Crossref | GoogleScholarGoogle Scholar | 27874252PubMed |
[35] Vo D, Zurakowski D, Faraoni D. Incidence and predictors of 30-day postoperative readmission in children. Pediatric Anaesth 2018; 28 63–70.
| Incidence and predictors of 30-day postoperative readmission in children.Crossref | GoogleScholarGoogle Scholar |
[36] Auger K, Davis M. Pediatric weekend admission and increased unplanned readmission rates. J Hosp Med 2015; 10 743–45.
| Pediatric weekend admission and increased unplanned readmission rates.Crossref | GoogleScholarGoogle Scholar | 26381150PubMed |
[37] Khan A, Nakamura MM, Zaslavsky AM, Jang J, Berry JG, Feng JY, Schuster MA. Same-hospital readmission rates as a measure of pediatric quality of care. JAMA Pediatr 2015; 169 905–12.
| Same-hospital readmission rates as a measure of pediatric quality of care.Crossref | GoogleScholarGoogle Scholar | 26237469PubMed |
[38] Sills MR, Hall M, Colvin JD, Macy ML, Cutler GJ, Bettenhausen JL, Morse RB, Auger KA, Raphael JL, Gottlieb LM, Fieldston ES, Shah SS. Association of social determinants with children’s hospitals’ preventable readmissions performance. JAMA Pediatr 2017; 170 350–8.
| Association of social determinants with children’s hospitals’ preventable readmissions performance.Crossref | GoogleScholarGoogle Scholar |
[39] Richards MK, Yanez D, Goldin AB, Grieb T, Murphy WM, Drugas GT. Factors associated with 30-day unplanned pediatric surgical readmission. Am J Surg 2016; 212 426–32.
| Factors associated with 30-day unplanned pediatric surgical readmission.Crossref | GoogleScholarGoogle Scholar | 26924805PubMed |
[40] Tommey S, Peltz A, Loren S, Tracy M, Williams K, Pengeroth L, Ste Marie A, Onorato S, Schuster MA. Potentially preventable 30-day hospital readmissions at a children’s hospital. Pediatr Neonatol 2016; 138 e20154182
| Potentially preventable 30-day hospital readmissions at a children’s hospital.Crossref | GoogleScholarGoogle Scholar |
[41] Heenan D, Birrell D. Hospital-based social work: challenges at the interface between health and social care. Br J Soc Work 2019; 49 1741–58.
| Hospital-based social work: challenges at the interface between health and social care.Crossref | GoogleScholarGoogle Scholar |
[42] Kornburger CK, Gibson C, Sadowski S, Maletta K, Klingbeil C. Using ‘teach-back’ to promote a safe transition from hospital to home: an evidence-based approach to improving the discharge process. J Pediatr Nurs 2013; 28 282–91.
| Using ‘teach-back’ to promote a safe transition from hospital to home: an evidence-based approach to improving the discharge process.Crossref | GoogleScholarGoogle Scholar |
[43] Hoyer EH, Odonkor CA, Bhatia SN, Leung C, Deutschendorf A, Brotman DJ. Association between days to complete inpatient discharge summaries with all-payer hospital readmissions in Maryland. J Hosp Med 2016; 11 393–400.
| Association between days to complete inpatient discharge summaries with all-payer hospital readmissions in Maryland.Crossref | GoogleScholarGoogle Scholar | 26913814PubMed |
