Partnering with consumers in the public reporting of quality of care: review of the Victorian quality of care reports

Additional keywords: analysis framework, consumer participation.

Quality of care within healthcare reflects the notion that patients, also known as clients or consumers, are provided with a high level of care that is underpinned by evidence-based, efficient, timely, equitable and culturally appropriate treatment that is goal based and seeks to produce a desired outcome.^1–5 There are several mechanisms that assist and encourage health services to engage in continuous quality improvement activities and evaluation of their provision of care. Since the 1980s, public reporting has been one such mechanism for improving quality and safety in healthcare.⁴ Public reporting is also a tool through which health services communicate with and engage their consumers in order to improve the quality of care they provide.^6,7 International health departments and bodies responsible for monitoring, regulating and overseeing health services require annual public reporting on quality and safety.^7–10 Since 2001, Victorian public health services have been required by the Department of Health, now known as the Department of Health and Human Services, to produce an annual quality of care report as a means of communicating to the community quality and safety measures, consumer participation and continuity of care.^11,12 These reports aim to relay performance data to the community, motivate and engage healthcare staff to monitor and improve their quality of care provision, educate consumers and increase their level of health literacy and demonstrate a given health service’s accountability to the public about their provision of care.^11,12

The World Health Organization’s (WHO) Declaration of Alma Ata¹³ states that ‘the people have the right and duty to participate individually and collectively in the planning and implementation of their health care’. Since the publication of that statement, bodies responsible for quality and safety in healthcare and accreditation standards have incorporated this right as a key priority. The Australian Commission on Safety and Quality in Health Care (ACSQHC) defines patient-centred care as ‘healthcare that is respectful of, and responsive to, the preferences, needs and values of patients and consumers’.¹⁴

Partnering with consumers is associated with improved clinical outcomes, decreased readmission rates and rates of hospital-acquired infections, adherence to treatment recommendations and health literacy.¹⁴ Engaging consumers has been recognised as a means to increase the accessibility and appropriateness of healthcare to individuals.^15,16 As such, the ACSQHC incorporated a consumer focus including the EQuIP5 Standards⁵ in 2010.From 2012, the NSQHS Standard was further expanded with the development of the Partnering with Consumers health standard within accreditation and quality of care requirements.¹⁶ Public reporting is considered to be a core element of the Partnering with Consumers standard, outlined as ‘the community and consumers are provided with information that is meaningful and relevant on the organisation’s safety and quality performance’.¹⁶ This reflects the value to both consumers and healthcare services of quality of care reporting that is made publicly available.

With an increased focus on engagement with consumers, it was hypothesised that health services would have a greater incentive to use the quality of care reports as a mechanism for communicating and partnering with consumers for quality and safety activities. As such, the aim of the present study was to investigate whether there was a difference between the quality of care reports published by Victorian metropolitan public health services before and after the introduction of the Partnering with Consumers NSQHS standard.

In reviewing the literature before conducting the study, no peer-reviewed scientific articles were identified that discussed the relationship between public reporting of quality of care and consumer participation. Most of the relevant material was obtained from searching of the grey literature and so this review was aligned as a narrative review rather than a systematic review. Publications were found to support the notion that, internationally, quality of care public reporting is an annual requirement, as in Victoria, Australia. In the UK, the NHS requires health services to generate and submit an annual quality account that needs to include core quality indicators.⁸ The purposes of the quality account seemed to reflect similar principles to that of the Victorian quality of care report, including: (1) patient safety; (2) the effectiveness of treatments that patients receive; and (3) patient feedback about the care provided.¹⁷ In the US, Congress mandated that the Agency for Healthcare Research and Quality produce an annual report on health care quality in the US beginning in fiscal year 2003. This report aims to include a broad set of performance measures that will be used to monitor the country’s progress towards improved health care quality.⁹ This highlights the international recognition that public reporting of quality and safety of a given health service is beneficial, both for consumers and healthcare staff alike.^7,10

In Australia, South Australia’s Department of Health publishes an annual patient safety report that ‘demonstrates the continued systematic improvement across SA Health in a number of Safety and Quality programs’¹⁸ based on the NSQHS Standards. Although the South Australian Department of Health does not require each public hospital to produce an annual quality of care report, the annual patient safety report includes information relating to the overall hospital system’s achievement of each of the NSQHS standards, including consumer participation. Tasmania, Western Australia and Northern Territory departments of health do not require public quality of care reports. Queensland’s Department of Health does not mandate quality of care reporting, although four health services publish annual quality of care reports that are publicly available.¹⁹

Public reporting has been identified as a means to communicate and engage with consumers as well as a motivational tool for organisations and staff to gauge their performance and monitor their quality improvement activities and status.^10,12,20 Organisations may be more interested in their public reputation and image within the community at large when their performance is publicly presented, and this may motivate improvements in the quality of care provided.⁷ For consumers, an additional benefit of public reporting of organisational performance is the potential for educating the community and increasing consumers’ health literacy.¹² Health literacy, as defined by the WHO, is ‘the cognitive and social skills that determine the motivation and ability of individuals to gain access to, understand and use information in ways that promote and maintain good health’.²¹ Nielsen-Bohlman et al.²² added that health literacy is ‘the degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions’.

A comparative descriptive observational study design was implemented to compare the content of the annual quality of care reports published by Victorian metropolitan public health services before and after the introduction of the Partnering with Consumers NSQHS Standard.¹⁶ Each year, Victorian public health services are required to produce a quality of care report that includes quality and safety measures and data, evidence of consumer participation and continuity of care.

It was hypothesised that the 2014 reports would have a greater degree of content supporting consumer participation following the introduction of the Partnering with Consumers NSQHS standard in 2012 and that health services would have had approximately 12–18 months to embed these principles and actions. The Standard specifically requires the provision of consumer feedback and the incorporation of such feedback into publications prepared by the health service for distribution to patients.¹⁴ This would suggest that the publicly distributed reports on quality of care would reflect this consumer partnership. Victorian public metropolitan health services were eligible for inclusion in the study, but specialist public hospitals, private hospitals, non-metropolitan public health services and other health services not affiliated with Victorian metropolitan health services were excluded. The rationale for the inclusion criteria was that only Victorian publicly funded health services are required to produce an annual, publicly available quality of care report.¹² The nine health services were considered to be comparable services for the purposes of the study. Specialist public hospitals and health services were not included in the sample because these services may not be comparable given their specialisation. Similarly, non-metropolitan health services were excluded because they too may function and report differently. This could be considered selection bias, although it was hypothesised that the sample of nine health services would be a reasonable representation of Victorian metropolitan public health services.

The first time point of data collection was the quality of care reports of 2011. The second time point of data collection was the quality of care reports of 2014. The rationale for these time points was that the NSQHS Standard was introduced in Victoria in 2012.¹⁶ In order to obtain comparative data, sampling of the quality of care reports from before the introduction of this health standard and sometime after, is required. The rationale for choosing 2014 was to allow a reasonable amount of time for uptake and implementation in order to determine whether change has occurred within the organisation that is then demonstrated in their quality of care report. Annual quality of care reports from 2011 and 2014 were downloaded from each health service’s Internet website and then analysed. In some cases, the 2011 quality of care reports were listed as ‘archived reports or publications’. Ethics approval was not required for this study because the data collected were publicly available, published material.

A quantitative model of data measurement and analysis was chosen to approach the analysis of the data. A database search was conducted to find a framework tool that could be applied to analyse the quality of care reports but no suitable framework was found. A framework was devised to structure the analysis and comparison of the quality of care reports from 2011 and 2014 (see Table 1). The framework was based on the minimum requirements for quality of care reporting according to the Victorian Department of Health^11,12,23 and incorporates descriptors of the NSQHS Standard 2 Partnering with Consumers, as introduced in 2012. In addition, elements of the checklist of Currie et al.²⁴ for health services and practitioners to use when developing and assessing written consumer health information were included in the framework. The authors sought opinions of three experts on consumer engagement and quality of care reporting from the Victorian Department of Health and Human Services, who reviewed the framework and agreed that it was a reasonable approach.

Table 1.  Framework used for the analysis of quality of care reports

The research question was descriptive in nature although a quantitative model was applied to data analysis via quantitative scoring against predetermined criteria.

The content data from each of the quality of care reports were analysed using the framework. Each indicator was judged as a ‘yes’ or ‘no’ response. The framework sought to minimise any judgement bias via an objective, binary analysis table where scores were given based on exactly what was or was not included in each report. For comparative analysis, each ‘yes’ response was given 1 point, which was then summed to create total scores for each health service for each quality of care report and for each indicator overall. Indicators and scores were not weighted. Higher allocated scores suggested that the report had a greater focus on consumer engagement.

The total scores ranged from 28 to 35 for the nine 2011 quality of care reports and from 26 to 35 for the nine 2014 quality of care reports. Fig. 1 shows changes in total scores for each indicator comparatively between the 2011 and 2014 reports. As shown in Fig. 2, of the nine health services, only three demonstrated total scores in 2014 that suggested a greater focus on partnering with consumers compared with their 2011 quality of care report.

Fig. 1.  Overall comparison of total scores for each indicator in the 2011 and 2014 quality of care reports (calculated using the framework for this study). Red bars indicate maximum possible scores for the different indicators. Indicator 1, feedback on the quality of care provided; Indicator 2, feedback on the quality of care report; Indicator 3, consumer partnership in service planning; Indicator 4, consumer partnership in designing care; Indicator 5, consumer partnership in service measurement and evaluation; Indicator 6, continuity of care; Indicator 7, distribution of the quality of care report; Indicator 8, availability of the quality of care report; Indicator 9, presentation of the quality of care report; Indicator 10, data and figures used in the quality of care report; Indicator 11, report on four of six quality and safety measures; Indicator 12, consumer stories.

Fig. 2.  Overall comparison of total scores for each of the nine health services in their 2011 and 2014 quality of care reports. Red bars indicate maximum possible scores for the different indicators.

In both 2011 and 2014, all the health services included at least four of six of the NSQHS standards as per the Victorian Department of Health^11,23 minimum reporting requirements. In 2011, four health services included five of the NSQHS standards and one included six. In 2014, this increased, with five health services including five of the NSQHS standards and two including six. All health services’ quality of care reports fully met the ‘presentation’ criteria, including consumer quotes and stories, which indicated that they have consumers as members within their committees, advisory forums and panels. Of the 12 indicators, there were no changes in three: the presentation of the report and the inclusion of at least four of six of the NSQHS standards and consumer stories. There were variations in the remaining 11 indicators across the health services and time points. There was no clear pattern, with various quality of care reports demonstrating both improvements and reductions in these indicators, resulting in less variation among the individual indicators than among the health services.

In summary, three health services demonstrated a total score that reflected greater consumer engagement in 2014 quality of care reports compared with their 2011 quality of care report. One health service’s score remained the same for both years, and the total scores for the remaining five health services decreased by 16–17% in 2014 compared with 2011. Of these five health services whose total scores decreased, three decreased by 6 points, one by 5 points and one by 1 point.

Because partnering with consumers has been identified in the literature as a positive contributor to better health outcomes, it was hypothesised that after the introduction of the Partnering with Consumers NSQHS standard in 2012 there would be an increase in the level of consumer participation focus in the quality of care reports. The researchers expected that all or most of the health services would have achieved higher total scores for their 2014 quality of care report given the requirements for the report outlined by the Victorian Department of Health and the detailed NSQHS Partnering with Consumers standard, which was introduced in 2012.

The results of the present study were mixed but do not support the hypothesis. Marshall et al.²⁵ discuss the relationship between the effect of public reporting on quality and safety and the culture of an organisation. In addition, public reporting has the potential to engage consumers and affect the culture of consumer participation within healthcare organisations with the support of healthcare policies and NSQHS standards for quality and safety. Reporting to consumers fosters reflection of the quality of care provided and the importance on quality in order to deliver desired health outcomes.²⁵ In most cases, the scores decreased for the 2014 reports compared with the 2011 reports. This suggests that the consumer participation focus within the reports had declined in 2014 in contrast with the partnering with consumers content of the 2011 reports. Several possible explanations for these study results have been explored.

Given the timing of the introduction of the new NSQHS standards in 2012 and the accreditation surveys for the health services in 2013 and 2014, it is highly likely that the health services undertook a greater focus on meeting the accreditation requirements than focusing on the content to include in the quality of care report. When looking at the criteria within the Partnering with Consumers NSQHS standard, it is far more detailed than simply reporting to the public. Public reporting is just one potential action for meeting the standard’s criteria,^16,26 and this suggests that there may well have been a disassociation by the health services between the action of producing a quality of care report and their adherence to the standard otherwise.

In addition, in recognising the difficulty in full implementation of this Standard, the NSQHS only included four actions in the Partnering with Consumers standard as core, with the remainder identified as developmental. It is suggested that, over time, as these developmental actions are addressed, there may be more coherence between the requirements of the Partnering with Consumers standard and the reporting to consumers.

Similarly, the Doing it With Us Not For Us: Strategic Direction 2010–2013 policy for consumer, carer and community participation in the healthcare system²⁷ was arguably no longer directly relevant in 2014 and there was no further policy document for health services to reference at the time. This may explain the reduction in the level of consumer participation evident in the quality of care reports in 2014, because health services may have lacked the direction as to how to action consumer participation principles. It would be recommended that government policies are reviewed towards the end of the policy’s relevant timeframe in order to have either a new policy document ready to proceed from the earlier document or have a minimal gap. This would support health services in guiding their ongoing quality of care and consumer participation operations.

Furthermore, the Victorian Department of Health^11,23 mandated that all public health services were required to produce an annual quality of care report, although no formal assessment or feedback on the reports was performed. This could also explain why the health services may have deprioritised their attention to the quality of care reports, particularly compared with their attention to meeting accreditation standards. Because there are significant consequences should a health service not achieve accreditation, it is plausible that this took priority, because the only requirement for quality of care reports is that they are submitted annually. It is therefore recommended that future quality of care reports are formally assessed, with feedback provided in order to maximise the level of content regarding consumer participation that is required.

Public reporting via the annual quality of care reports is a valuable mechanism for driving continuous quality improvement and consumer engagement within health services and increasing health literacy within the community. The aim of the present study was to see whether there was a difference between the content in the quality of care reports before and after the introduction of the Partnering with Consumers NSQHS standard. The study demonstrated mixed findings and, in some cases, lower scores for the 2014 reports, which was after the introduction of the Partnering with Consumers NSQHS standard, compared with the 2011 reports. Several possible explanations for the results have been proposed. Overall, there is future scope to enhance the degree of consumer participation evident within the quality of care reports. Future research may focus on whether the production of the quality of care report affects consumers’ knowledge and confidence in the quality of care within a health service and whether the quality of care report meets the aim of increasing consumers’ health literacy.

The authors declare no competing interests.