Developing key performance indicators for a tertiary children’s hospital network
Christopher Elliot A C D , Cheryl Mcullagh B , Michael Brydon B and Karen Zwi A CA Sydney Children’s Hospital Department of Community Child Health, Corner Barker and Avoca Streets, Randwick, NSW 2031, Australia.
B Executive Unit, Sydney Children’s Hospitals Network, Locked Bag 4001, Westmead, NSW 2145, Australia. Email: cheryl.mccullagh@health.nsw.gov.au; michael.brydon@health.nsw.gov.au
C School of Women’s and Children’s Health, University of New South Wales, Level 3, Executive Wing, Sydney Children’s Hospital, High Street, Randwick, NSW 2031, Australia. Email: karen.zwi@health.nsw.gov.au
D Corresponding author. Email: chriselliot@unsw.edu.au
Australian Health Review 42(5) 491-500 https://doi.org/10.1071/AH17263
Submitted: 15 November 2017 Accepted: 27 July 2018 Published: 20 August 2018
Abstract
Objective The aim of this study is to describe the experience of developing key performance indicators (KPIs) for Sydney Children’s Hospital Network (SCHN), the largest paediatric healthcare entity in Australia.
Methods Beginning with a published methodology, the process of developing KPIs involved five phases: (1) identification of potential KPIs referencing the organisational strategic plan and pre-existing internal and external documents; (2) consolidation into a pragmatic set; (3) analysis of potential KPIs against selection criteria; (4) mapping these back against the strategic plan and management structure; and (5) presentation to key stakeholders to ensure suitability and traction. Consistent with the strategic plan, a subset of indicators was selected to address quality of care for children from priority populations.
Results A pragmatic list of 60 mandated and 50 potential KPIs was created from the 328 new and 397 existing potentially relevant KPIs generated by the executive team. Of these, 20 KPIs were selected as the most important; 65% were process measures. The majority of mandated KPIs were process measures. Of the KPIs selected to highlight inequities, there were proportionately more outcome measures (44% outcome, 27% process). Less than one-third could currently be measured by the organisation and were thus aspirational.
Conclusion Developing a KPI suite requires substantial time, effort and organisational courage. A structured approach to performance measurement and improvement is needed to ensure a balanced suite of KPIs that can be expected to drive an organisation to improve child health outcomes. Future directions for SCHN include a systematic approach to implementation beyond the mandated KPIs, including KPIs that reflect equity and improved outcomes for priority populations, development of meaningful measures for the aspirational KPIs, adding structure KPIs and measurement of changes in child health outcomes related to the development of this KPI process.
What is known about the topic? Health services are increasingly required to demonstrate accountability through KPIs. There is a body of literature on both theoretical frameworks for measuring performance and a long list of possible measures, however developing a meaningful suite of KPIs remains a significant challenge for individual organisations.
What does this paper add? This paper describes lessons learned from the practical, pragmatic application of a published methodology to develop a suite of KPIs for the largest paediatric healthcare entity in Australia. It provides a select list of the highest-level KPIs selected by the organisation to stimulate further discussion among similar organisations in relation to KPI selection and implementation.
What are the implications for practitioners? Developing and implementing a suite of meaningful KPIs for a large organisation requires courage, an understanding of health informatics, stakeholder engagement, stamina and pragmatism. The process we describe can be replicated and/or modified as needed, with discussion of key lessons learned to help practitioners plan ahead.
Additional keywords: child, children, equity, hospitals, priority or vulnerable populations.
References
[1] Kavanagh PL, Adams WG, Wang CJ. Quality indicators and quality assessment in child health. Arch Dis Child 2009; 94 458–63.| Quality indicators and quality assessment in child health.Crossref | GoogleScholarGoogle Scholar |
[2] The Sydney Children’s Hospitals Network. Safer smarter stronger. Annual review 2016. 2016. Available at: https://www.schn.health.nsw.gov.au/files/attachments/schn_annual_review_2016.pdf [verified 31 July 2018].
[3] Porter ME. What is Value in Health Care? N Engl J Med 2010; 363 2477–81.
| What is Value in Health Care?Crossref | GoogleScholarGoogle Scholar |
[4] The Sydney Children’s Hospitals Network Strategic Plan 2012–2016. 2011. Available at: https://www.schn.health.nsw.gov.au/files/schn-strategic-plan-2011-2016.pdf [verified 31 July 2018].
[5] Australian Institute of Health and Welfare (AIHW). Australia’s health 2018. Australia’s Health Series no. 16. AUS 221. Canberra: AIHW; 2018.
[6] McDonald KM. Approach to improving quality: the role of quality measurement and a case study of the agency for healthcare research and quality pediatric quality indicators. Pediatr Clin North Am 2009; 56 815–29.
| Approach to improving quality: the role of quality measurement and a case study of the agency for healthcare research and quality pediatric quality indicators.Crossref | GoogleScholarGoogle Scholar |
[7] New South Wales Government Department of Health. Service agreement: an agreement between: Secretary, NSW Health and Sydney Children’s Hospitals Network for the period 1 July 2016–30 June 2017. 2016. Available at: https://www.schn.health.nsw.gov.au/files/attachments/service_agreement_201617_final_signed.pdf [verified 31 July 2018].
[8] NSW Ministry of Health. Schedule E key performance indicators and service measures for the 2016/17 service agreements version 1.01. 2016. Available at: http://www1.health.nsw.gov.au/pds/ActivePDSDocuments/IB2016_036.pdf [verified 31 July 2018].
[9] Donabedian A, Bashshur R. An introduction to quality assurance in health care. New York: Oxford University Press; 2002.
[10] Shonkoff JP, Garner AS, The Committee on Psychosocial Aspects of Child and Family Health Committee on Early Childhood, Adoption, and Dependent Care Section on Developmental and Behavioural Pediatrics Siegel BS, Dobbins MI, Earls MF, Garner AS, McGuinn L, Pascoe J, Wood DL. The Lifelong effects of early childhood adversity and toxic stress. Pediatrics 2012; 129
| The Lifelong effects of early childhood adversity and toxic stress.Crossref | GoogleScholarGoogle Scholar |
[11] Safford MM. Improving the quality of quality measurement; the tinkerer, the tailor and the candlestick maker. Med Care 2009; 47 375–7.
| Improving the quality of quality measurement; the tinkerer, the tailor and the candlestick maker.Crossref | GoogleScholarGoogle Scholar |
[12] Mannion R, Smith J. Hospital culture and clinical performance: where next? BMJ Qual Saf 2018; 27 179–81.
| Hospital culture and clinical performance: where next?Crossref | GoogleScholarGoogle Scholar |
[13] Mangione-Smith R, Schiff J, Dougherty D. Identifying children’s health care quality measures for Medicaid and CHIP: an evidence-informed, publically transparent expert process. Acad Pediatr 2011; 11 S11–21.
| Identifying children’s health care quality measures for Medicaid and CHIP: an evidence-informed, publically transparent expert process.Crossref | GoogleScholarGoogle Scholar |
[14] Kelley E, Hurst J. Health care quality indicators project: conceptual framework paper. OECD Health Working Papers No. 23. Paris: OECD Publishing; 2006.
[15] Buttigieg SC, Abela L, Pace A. Variables affecting hospital length of stay: a scoping review. J Health Organ Manag 2018; 32 463–93.
| Variables affecting hospital length of stay: a scoping review.Crossref | GoogleScholarGoogle Scholar |
[16] Australian Early Development Census 2016; https://www.aedc.gov.au/resources/detail/2015-aedc-national-report
| Crossref | GoogleScholarGoogle Scholar |
[17] Schechter MS. Benchmarking to improve the quality of cystic fibrosis care. Curr Opin Pulm Med 2012; 18 596–601.
| Benchmarking to improve the quality of cystic fibrosis care.Crossref | GoogleScholarGoogle Scholar |
[18] Stern M. The use of a cystic fibrosis patient registry to assess outcomes and improve cystic fibrosis care in Germany. Curr Opin Pulm Med 2011; 17 473–7.
[19] Kotter JP. A force for change: how leadership differs from management: New York: Free Press; 1990.
[20] Randolph G, Esporas M, Provost L, Massie S, Bundy DG. Model for improvement – part two: measurement and feedback for quality improvement efforts. Pediatr Clin North Am 2009; 56 779–98.
| Model for improvement – part two: measurement and feedback for quality improvement efforts.Crossref | GoogleScholarGoogle Scholar |
[21] Smith PC, Mossialos E, Papanicolas I. Performance measurement for health system improvement: experiences, challenges and prospects. World Health Organization; 2008. Available at: http://www.who.int/management/district/performance/PerformanceMeasurementHealthSystemImprovement2.pdf [verified 30 July 2018].
[22] Kemp L, Harris E, McMahon C, Matthey S, Vimpani G, Anderson T, Schmied V, Aslam H, Zapart S. Child and family outcomes of a long-term nurse home visitation programme: a randomised controlled trial. Arch Dis Child 2011; 96 533–40.
| Child and family outcomes of a long-term nurse home visitation programme: a randomised controlled trial.Crossref | GoogleScholarGoogle Scholar |
[23] NSW Ministry of Health. Service Framework to Improve the Health Care of People with Intellectual Disability. Sydney: NSW Ministry of Health; 2012. Available at: http://www.health.nsw.gov.au/disability/Publications/health-care-of-people-with-ID.pdf [verified 6 August 2018].
[24] Australian Institute of Health and Welfare (AIHW). Indigenous identification in hospital separations data – quality report. Health Services Series no. 35. Catalogue no. HSE 85. Canberra: AIHW; 2010. Available at: https://www.aihw.gov.au/reports/hospitals/indigenous-identification-hospital-separations/contents/table-of-contents [verified 31 July 2018].
[25] Ombudsman New South Wales. New South Wales child death review team annual report of child deaths registered in 2011. 2012. Available at: https://www.ombo.nsw.gov.au/news-and-publications/publications/annual-reports/nsw-child-death-review/nsw-child-death-review-team-annual-report-2011 [verified 31 July 2018].
[26] New South Wales Government Department of Health. NSW suicide prevention strategy 2010–2015 first progress report. 2012. Available at: http://www.health.nsw.gov.au/mentalhealth/programs/mh/Pages/suicide-prev-strategy.aspx [verified 31 July 2018].
[27] New South Wales Government Department of Health. The NSW Aboriginal perinatal health report. 2003. Available at: http://www.health.nsw.gov.au/kidsfamilies/MCFhealth/Pages/aboriginal-perinatal- health-report.aspx [verified 31 July 2018].
[28] Trinh LTT, Rubin G. Late entry to antenatal care in New South Wales, Australia. Reprod Health 2006; 3 8
| Late entry to antenatal care in New South Wales, Australia.Crossref | GoogleScholarGoogle Scholar |
[29] Australian Institute of Health and Welfare. Headline indicators for children’s health, development and wellbeing, 2011. Catalogue no. PHE 144. 2011. Available at: https://www.aihw.gov.au/reports/children-youth/headline-indicators-for-childrens-health-2011/contents/table-of-contents [verified 31 July 2018].
