Australian health service organisation assessment outcome data for the first 2 years of implementing the Comprehensive Care Standard
Margherita Murgo![https://orcid.org/0000-0002-3879-7268](/media/client/orcid_16x16.png)
A Partnering with Consumers, Australian Commission on Safety and Quality in Health Care, Level 5, 255 Elizabeth Street, Sydney, NSW 2000, Australia.
B Susan Wakil School of Nursing and Midwifery, Camperdown Campus, University of Sydney, Camperdown, NSW, Australia.
Australian Health Review 46(2) 210-216 https://doi.org/10.1071/AH21299
Submitted: 6 May 2021 Accepted: 11 December 2021 Published: 3 March 2022
© 2022 The Author(s) (or their employer(s)). Published by CSIRO Publishing on behalf of AHHA. This is an open access article distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND)
Abstract
Objective To review implementation of the Comprehensive Care Standard (CCS) by evaluating assessment outcome data of Australian health service organisations (HSOs) from January 2019. The CCS was introduced with the National Safety and Quality Health Service (NSQHS) Standards (second edition). It has 36 actions and over 40 resources to support implementation.
Methods Retrospective assessment outcome data submitted by accrediting agencies were examined to gauge progress of CCS implementation by considering met or other ratings of the CCS actions.
Results There were 495 assessments completed between January 2019 and December 2020. Most (71% n = 352) HSOs met the requirements of the CCS after initial assessment. Seventy-four (15%) of the HSOs did not meet all the requirements and a further 58 (12%) HSOs were provided with recommendations to meet the actions.
Conclusions There was indication of underperformance related to some actions in the CCS. The assessment data highlighted common issues for organisations including difficulties implementing governance processes, demonstrating effective care planning, implementing the end-of-life care actions and some minimising harm actions. Future evaluation of the implementation of the CCS may benefit from strengthening links between other safety and quality programs.
Keywords: accreditation, health data, health standards, healthcare, outcome measurement, patient safety, quality of care, risk management
Introduction
Patients in Australia receive health care that has been ranked highly for safety and quality when compared to other countries.1 There are around 11 million patient separations per year in Australian public and private hospitals.2 Most patients receive safe care; however, more than 180 000 hospital-acquired complications are recorded in admitted patient data each year. In 2016–17, more than 1000 serious clinical incidents were reported, indicating further initiatives could improve the safety and quality of Australian health care.3
The Australian Commission on Safety and Quality in Health Care (the Commission) is a national agency with functions specified in the National Health Reform Act 2011,4 including promotion and support for implementation of safety and quality matters in health care, and collection, interpretation and dissemination of safety and quality healthcare data. The Commission is responsible for the development of standards, including the National Safety and Quality Health Service (NSQHS) Standards that apply to a wide variety of health service organisations (HSOs) operating across a range of settings. Part of this responsibility includes coordinating the Australian Health Service Safety and Quality Accreditation (AHSSQA) Scheme.5
Standards are recognised as a method for improving the provision of health care.6 Since 2013, the NSQHS Standards have been used as a framework for improving the quality of health service provision in Australia, and minimising the risk of harm to patients.7,8 Evaluation of the first edition of the Standards demonstrated improvements in some patient outcomes.9 The increasing complexity and frailty of patients at risk of adverse events was a recognised safety and quality gap.9 Identification of these gaps contributed to the development and inclusion of the Comprehensive Care Standard (CCS) in the second edition.8
The actions in the CCS aligned with the best available evidence on areas of greater risk of harm to patients and were developed in collaboration with the Australian Government, states and territories, the private sector, clinical experts, patients and carers.8 The actions in the CCS strengthen the role of consumers and carers in making decisions and setting goals for their health care. This approach of providing care that meets the clinical and personal needs and goals of individual patients is an expectation of the Australian health system.
There are eight NSQHS Standards in the second edition, with 148 associated actions. Accreditation to the NSQHS Standards every 3 years is mandatory for all public and private hospitals, day procedure services and most public dental practices in Australia.10 To achieve accreditation, HSOs need to meet the NSQHS Standards. Accreditation may still be granted with some actions met with recommendations. Actions not met may be remediated by HSOs within specific timeframes to complete assessment.10
The number of HSOs requiring accreditation fluctuates. In December 2020, 1323 HSOs required accreditation by assessment to the NSQHS Standards. Commission advisories provide context about assessment to the actions in different circumstances. Assessment is undertaken by one of seven Commission-approved independent accrediting agencies. Agencies are responsible for ensuring assessors understand the requirements of rating hospital performance. Ratings are applied to each action in the Standards (Table 1). Completed assessment outcome data are submitted to the Commission and collated in a secure database. Assessment to the CCS commenced in January 2019. The CCS includes 36 actions listed under four criteria (Table 2), totalling 24% of the NSQHS Standards actions.8
![]() |
![]() |
Healthcare outcomes are not the same for all Australians, although health care is generally considered high-quality when measured by life expectancy and other clinical indicators.12 The CCS aims to ensure that patients receive safe, effective care that aligns with their needs and preferences. The actions in the CCS aim to promote continuous service improvement and mitigate risks in areas where patient harm continues.
Resources were released between 2016 and 2020 to support CCS implementation. Existing related resources were already available. The implementation resources included the foundational, cultural and organisational aspects important to implementation. The resources were derived from anticipated need, by reviewing common queries submitted to the Commission, and stakeholder consultation. Consumer resources were also developed for specific issues identified by consumer representatives.
This paper describes a retrospective examination of CCS assessment outcome data to identify areas of underperformance (actions not met and met with recommendations) and potential misinterpretation of the requirements (inappropriate application of ratings). This was part of a larger strategy to evaluate CCS implementation, which included qualitative components and evaluation of resource use. Post hoc evaluation prior to completion of the 3 year accreditation cycle was part of the Commission’s organisational work plan for comprehensive care.
Methods
A retrospective examination of assessment outcome data routinely submitted by accrediting agencies to the Commission was undertaken as a measure of progress of CCS implementation. The recorded status of actions in the CCS at the completion of initial assessment to the NSQHS Standards for HSOs between January 2019 and December 2020, was retrieved from the Commission database for analysis.
The number of CCS actions met, not met, met with recommendations and not applicable were calculated. Not applicable actions were reviewed to assess compliance with the relevant advisory.13 Non-compliant actions assigned not applicable were coded as inappropriate.
Ethics approval
Ethics approval was not sought as the intended purpose of the data analysis was for quality improvement. This was a retrospective examination of de-identified assessment outcome data routinely submitted to the Commission by accrediting agencies. There was no testing of novel policies or departure from standard care. The data related to HSOs and no individual patients or minority or vulnerable groups were identifiable.
Results
A total of 495 assessments relating to 468 unique HSOs were undertaken between January 2019 and December 2020. Although all Australian states and territories were represented in the assessment outcome data, 24 HSOs were registered for short notice assessment that may not have included assessment to the CCS. Most (71.1%, n = 352) assessed HSOs met the requirements or received not applicable status for the 36 actions of the CCS after completion of initial assessment (Fig. 1, Table 3). For the remaining assessments, CCS actions were not met (11.7%, n = 58) and met with recommendations (14.9%, n = 74) (Table 4).
![]() |
![]() |
![]() |
There have been 17 820 CCS actions assessed during the 495 assessments. CCS actions were not assessed (n = 866) in most of the HSOs that undertook short notice assessments (n = 24).
All 36 actions in the CCS were rated not met or met with recommendations by at least one HSO. The actions not met, met with recommendations and inappropriately assigned not applicable are outlined in Fig. 2. All actions are accounted for in at least one category. The not met rating (assigned 325 times) was issued for all but one action (Preventing falls and harm from falls 5.25), met with recommendations (assigned 331 times) was issued for all but five actions (collaboration and teamwork 5.05, documenting advance care plans 5.09, preventing and managing pressure injury 5.21 and 5.23, preventing falls and harm from falls 5.24) and not applicable status (n = 3028) was issued for all but 12 actions (5.01–5.08, 5.11–5.14) and was inappropriately assigned at least once for 23 actions.
![]() |
Appropriately assigned not applicable ratings most often related to minimising restrictive practices-seclusion (5.36 n = 265), followed by end-of-life care actions including identifying end-of-life 5.15 (n = 177), accessing specialist palliative care 5.16 (n = 191), accessing supervision and support 5.18 (n = 190), reviewing care provided 5.19 (n = 189), and shared decision -making 5.20 (n = 178). The not applicable rating included 208 occurrences (9%) that were not compliant with the advisory13 and therefore coded inappropriate. Inappropriately rated actions also often concerned end-of-life care. These included action 5.15 (n = 17), 5.16 (n = 25), 5.18 (n = 24), 5.19 (n = 24) and 5.20 (n = 14). Action 5.31, predicting, preventing and managing self-harm and suicide, was also represented 15 times.
Discussion
Examination of the assessment outcome data provided a preliminary review of CCS implementation, representing 35.4% of Australian HSOs requiring accreditation. Most HSOs (71%) met initial assessment requirements for the CCS more often than the Clinical Governance, Medication Safety and Preventing and Controlling Healthcare-Associated Infection Standards. An anticipated wash-in period, planned implementation, and actions already part of health service provision may have contributed to this outcome. Prior to implementation, the NSQHS Standards (2nd edn) were released for public consultation with extensive lead-time to provide opportunities for change.
Gap analysis for specific actions was a requirement of the staged implementation approach set out in advisories.14,15 Initial consultation highlighted the potential inability of HSOs to meet actions related to risk screening and assessment, multidisciplinary teamwork, and comprehensive care plans. Resources supporting these actions were developed and released between 2016 and 2018. Results demonstrated that three of the top five actions not met and met with recommendations were from the Minimising patient harm criterion. Action 5.5 requiring teamwork and collaboration was only allocated not met twice. The advisories14,15 allowed additional time for HSOs to implement actions identified as problematic, with scheduled milestones up to 2022 when full implementation was expected. The date was extended to 2023 as a result of the COVID-19 pandemic. Accreditation was suspended for much of 2020 and HSO accreditation status was maintained during the pandemic response. Hybrid assessments were also permitted to complete initial assessment for HSOs that were not accessible to accreditation teams. There was no capacity to separate the data in this evaluation.
Integrating clinical governance and applying quality improvement systems (5.01 and 5.02) were the most frequently not met actions. This aligns with the poor performance at initial assessment in relation to the Clinical Governance Standard, which was only met by 45% of HSOs. Clinical governance is an integrated component of corporate governance. It ensures that frontline clinicians, managers and board members are accountable to patients and the community for assuring the delivery of safe care.16 The relevant CCS actions are integral to ensuring organisational culture and governance supports comprehensive care delivery, and were described in the earliest tranche of implementation resources.17 Website analytics data from the Commission indicates that although the CCS is highly accessed, the implementation resources are not. Increased use of these resources could support improvement.
The CCS promotes continuous quality improvement in person-centred care. It does not prescribe condition-specific care activities, although the Minimising patient harm criterion includes requirements related to high-frequency adverse events. None of the CCS actions were met by every HSO, including actions relating to pressure injuries and falls. Both were previously addressed as separate Standards to which many organisations were assessed. Pressure injury and falls prevention were encompassed by the CCS with significantly truncated actions.
Falls18 and pressure injuries19 continue to be a major source of harm for patients in hospital. These risks were added to a national list of Hospital-Acquired Complications (HACs) made up of 16 agreed, high-priority complications, which clinicians, managers and others can address and improve patient care. Action 5.22 preventing and managing pressure injuries appears in the top five not met actions and action 5.25 related to falls prevention appears in the top five met with recommendations. The actions most frequently requiring remediation are likely to change as the accreditation cycle continues and more HSOs are assessed.
There are a number of HACs defined and reported from the administrative data that could be related to the CCS. HAC data could not be linked for this evaluation. In the 2019–20 financial year, the HACs rate for pressure injury was 43.8 per 100 000 separations, for falls 43.7 per 100 000 separations and delirium 357.1 per 100 000 separations. The CCS could be strengthened by linking with programs such as HACs funding penalties.20 The actions in the CCS are unlikely to fully reduce harm to patients from these adverse events and the funding penalty is intended to improve outcomes by encouraging strong risk management processes. Clinical care standards may also have a role in supporting CCS implementation.
Day procedure services were slightly more likely to meet the actions at initial assessment, although no statistical test was performed. This is probably due to the number of actions that are permitted a not applicable rating. The majority of not applicable ratings were allocated in day procedure services. There were 208 inappropriate not applicable ratings,13 mostly related to end-of-life care. The end-of-life care actions were also highly represented in the not met, met with recommendations, and not applicable ratings. The only end-of-life care action where HSOs were performing consistently was receiving and documenting advance care plans (5.17). The number of non-compliant not applicable end-of-life care actions increased in April 2020. This was the result of an update to the advisory14 where some actions that did not previously apply to day procedure services were reconsidered. The end-of-life care actions were included in the CCS to support implementation of the National Consensus Statement: essential elements for safe and high-quality end-of-life care.21 End-of-life care is a recognised area for improvement in Australia,22 which is demonstrated in the assessment outcome data. Although it is unlikely that day procedure services frequently provide care to patients in their last days, some procedures may be appropriate for people who are at the end of life. Day procedure services should have capacity to meet the specified actions when these patients are admitted. Increased oversight of the application of the not applicable rating may also decrease the variability of assessments between HSOs.
The accreditation scheme was reviewed in 2018 to improve reliability of accreditation processes. This necessitated changes to the way in which assessor teams conduct assessments. Assessors would have been developing assessment skills required by the changes; however, it is difficult to assess what effect this had on the assessment outcome data. Assessments are undertaken by different assessor teams and some variation in the outcomes was expected. Additionally, there was no capacity to determine variation between assessors and accrediting agencies in their application of ratings. The advisories and appeals process provide a balancing feature to the AHSSQA scheme. At the time of this evaluation, all HSOs had been awarded accreditation. Accreditation is important to ensuring quality and safety, but is not the panacea to preventing all patient harm. Repeated review of the assessment outcome data would provide a more complete understanding of progress towards CCS implementation and changes that may be required in future editions of the NSQHS Standards.
Conclusion
The CCS had 71% of actions met by the 35.4% of Australian HSOs assessed during the first 2 years of implementation of the NSQHS Standards (2nd edn) and performed well compared to other Standards. There was some indication of underperformance, particularly with regard to clinical governance systems that support delivery of comprehensive care. Measuring the implementation of the CCS is complicated and further investigation is needed to understand the extent of variation and attributable factors. Some actions in the CCS are linked to additional measures and incentives that also drive change, such as inclusion in the HACs list. Future evaluation of the implementation of the CCS may benefit from strengthening links between these synergistic quality and safety programs. To improve the implementation of comprehensive care within HSOs, effort should focus on clinical governance systems that support organisational safety culture, teamwork and clinicians to deliver comprehensive care.
Data availability
The data for this article are not available in a public repository. Data will be shared by request to the corresponding author pending permission from the Australian Commission on Safety and Quality in Health Care.
Conflicts of interest
The authors declare no conflicts of interest.
Declaration of funding
This research did not receive any specific funding.
Acknowledgements
The authors would like to acknowledge Cruzette Manalo from the National Standards Team, Australian Commission on Safety and Quality in Health Care (ACSQHC), the custodian and creator of the database of assessment outcome data, and the Communications Team, ACSQHC.
References
[1] Schneider EC, Sarnak DO, Squires D, Shah A, Doty MM. Mirror, mirror 2017: International comparison reflects flaws and opportunities for better U.S. health care. The Commonwealth Fund; 2017.[2] Australian Institute of Health Welfare. Admitted patient care 2017–18. Canberra: AIHW; 2019.
[3] Australian Commission on Safety and Quality in Health Care. The state of patient safety and quality in Australian hospitals. Sydney: ACSQHC; 2019.
[4] Office of Parliamentary Counsel. National Health Reform Act. Canberra: Australian Government; 2011.
[5] Australian Commission on Safety and Quality in Health Care. Review of the Australian Health Service Safety and Quality Accreditation Scheme: Improving the reliability of health service organisation accreditation processes. Sydney: ACSQHC; 2018.
[6] Braithwaite J, Vincent C, Nicklin W, Amalberti R. Coping with more people with more illness. Part 2: new generation of standards for enabling healthcare system transformation and sustainability. Int J Qual Health Care 2019; 31 159–63.
| Coping with more people with more illness. Part 2: new generation of standards for enabling healthcare system transformation and sustainability.Crossref | GoogleScholarGoogle Scholar | 30476117PubMed |
[7] Australian Commission on Safety and Quality in Health Care. National safety and quality health service standards Sydney. Sydney: ACSQHC; 2012.
[8] Australian Commission on Safety and Quality in Health Care. National safety and quality health service standards. 2nd edn. Sydney: ACSQHC; 2017.
[9] Australian Commission on Safety and Quality in Health Care. Creating safer, better health care – the impact of the National safety and quality health service standards. Sydney: ACSQHC; 2018.
[10] Australian Commission on Safety and Quality in Health Care. Fact sheet 1: standardised accreditation cycles. Sydney: ACSQHC; 2018.
[11] Australian Commission on Safety and Quality in Health Care. Fact sheet 4: rating scale for assessment. Sydney: ACSQHC; 2020.
[12] Australian Institute of Health and Welfare. Australia’s health 2020: In brief. Canberra: AIHW; 2020.
[13] Australian Commission on Safety and Quality in Health Care. AS18/01: advice on not applicable actions. Sydney: ACSQHC; 2020.
[14] Australian Commission on Safety and Quality in Health Care. AS18/15: comprehensive care standard: developing the comprehensive care plan. Sydney: ACSQHC; 2020.
[15] Australian Commission on Safety and Quality in Health Care. AS18/14: comprehensive care standard: screening and assessment for risk of harm. Sydney: ACSQHC; 2020.
[16] Australian Commission on Safety and Quality in Health Care. National model clinical governance framework. Sydney: ACSQHC; 2017.
[17] Australian Commission on Safety and Quality in Health Care. Implementing the comprehensive care standard: a conceptual model for supporting comprehensive care delivery. Sydney: ACSQHC; 2018.
[18] Morello RT, Barker AL, Watts JJ, Haines T, Zavarsek SS, Hill KD, et al. The extra resource burden of in-hospital falls: a cost of falls study. Med J Aust 2015; 203 367
| 26510807PubMed |
[19] Nguyen KH, Chaboyer W, Whitty JA. Pressure injury in Australian public hospitals: a cost-of-illness study. Aust Health Rev 2015; 39 329–36.
| Pressure injury in Australian public hospitals: a cost-of-illness study.Crossref | GoogleScholarGoogle Scholar | 25725696PubMed |
[20] Independent Hospital Pricing Authority. Safety and quality. Sydney: IHPA; 2019. Available at https://www.ihpa.gov.au/what-we-do/safety-and-quality [Accessed 12 August 2020]
[21] Australian Commission on Safety and Quality in Health Care. National consensus statement: essential elements for safe, high-quality end-of-life care. 2nd edn. Sydney: ACSQHC; 2015.
[22] Mitchell I, Lacey J, Anstey M, Corbett C, Douglas C, Drummond C. Understanding end-of-life care in Australian hospitals. Aust Health Rev 2021; 45 540–47.
| Understanding end-of-life care in Australian hospitals.Crossref | GoogleScholarGoogle Scholar |