Developing indicators and measures of high-quality for Australian general practice

Keywords: general practice, high-quality care, indicators, measures, patient‐centred medical home, primary care, primary health care, quality improvement.

In Australia, as internationally, there is a critical need for high-quality primary health care (PHC) to address rising health care costs (Dwyer and Duckett 2016). Additionally, there are inequitable health outcomes with particular impact on Aboriginal and Torres Strait Islander peoples; culturally and linguistically diverse populations; those at socioeconomic disadvantage; children; and those with disabilities, mental health, drug and alcohol problems (Royal Australian College of General Practitioners (RACGP) 2015a; Organisation for Economic Co-operation and Development (OECD) 2017).

Health systems with high-performing PHC deliver more equitable and cost-effective health outcomes (Basu et al. 2019). The Declaration of Astana asserted that ‘strengthening primary healthcare is the most inclusive, effective and efficient approach to enhance people’s physical and mental health, as well as social well-being, and that PHC is a cornerstone of a sustainable health system for universal health coverage’ (World Health Organization and the United Nations Children’s Fund 2018).

In Australia, PHC is delivered in general practices, community health centres, Aboriginal Community Controlled Health Services and allied health practices. High-quality care may not always be achieved, particularly as funding rewards throughput rather than quality, with fee-for-service payments a higher proportion of PHC funding, compared with payment for quality and population health outcomes, than in many other developed countries (OECD 2017).

In North America and New Zealand, PHC has been strengthened through implementation of Patient-Centred Medical Home (PCMH) models (Nutting et al. 2010; Cumming et al. 2018). There is growing evidence of efficacy of PCMHs, especially when aligned with integrated primary and secondary/tertiary care (Primary Care Collaborative 2017). There is support for PCMH models to improve the quality of PHC in Australia; however, funding approaches do not adequately support such models (The George Institute for Global Health, Consumers Health Forum of Australia, University of Queensland MRI Centre for Health System Reform and Integration 2018; Metusela et al. 2020a). The introduction of the Practice Incentives Program Quality Improvement payment (PIPQI) in 2019 provided a mechanism for payment of general practices for activities improving patient care, with enrolled practices committing to continuous quality improvement, and to providing data for local, regional and national health outcomes datasets (Australian Government Department of Health 2019).

Primary Health Networks (PHNs), regional organisations whose key objective is to increase the efficiency and effectiveness of PHC (Australian Government Department of Health 2021), assist general practices through collection and analysis of data, driving quality improvement and contributing to the national dataset. The Western Sydney PHN (WentWest) engages in a range of health care quality improvement initiatives in western Sydney, including supporting general practices transitioning to PCMH care models.

In view of the current focus on improving quality in Australian general practice, WentWest and the Western Sydney University research team identified the need for agreed measures of high quality in Australian general practice. Although a number of relevant frameworks map attributes of high-quality PHC, these are not specific to Australian general practice (Bodenheimer et al. 2014), do not identify a wide range of evidence-based measures of high-quality PHC (RACGP 2012, 2015b) or are focused on process measures rather than outcome indicators (Crossland et al. 2016; see Table 1 for definitions).

Table 1.  Table of definitions.

This research, undertaken with Wentwest and two other PHNs in western Sydney (Nepean Blue Mountains and South Western Sydney PHNs) aimed to develop a suite of evidence-based indicators and measures of high quality based on routinely collected data in Australian general practice. The longer-term aim is for use of these indicators and measures to drive quality improvement and potentially inform changes in the Australian PHC funding model to adequately support high-quality care.

We conducted an initial focused literature review to identify current quality frameworks related to Australian general practice and to PCMH models. A subsequent, wider literature review assisted us to revise and expand data routinely collected by WentWest. We identified an initial set of indicators and measures of high-quality likely to be relevant and feasible in Australian general practice, and reviewed these in three workshops with key stakeholder groups. The draft suite of indicators and measures was supplemented and refined in light of workshop feedback and further targeted literature review.

A core group of researchers (CM, NC, HVW, JR), three of whom are general practitioners, met quarterly with researchers from a range of PHC disciplines including a further four general practice academics, and an international consultant and primary care physician (KM). The team also met quarterly with an Advisory Group including senior staff and the chair of the WentWest Board, two of whom are general practitioners (refer to Supplementary File S1 for further details of the Advisory Group and research team). We received ethics approval from Western Sydney University Human Research Ethics Committee (H12003).

Our initial focused review of the literature was supplemented by a comprehensive literature search of PubMed and Medline databases to identify guidelines and documents related to high-quality PHC, using terms: ‘general practice’, ‘primary health care’, ‘health care quality access and evaluation/standard of care’; ‘quality improvement/quality indicators’, as well as the following text words: ‘attributes’, ‘indicators’, ‘elements’, ‘high-quality’, ‘general practice’, ‘primary care’ and ‘primary health care’. We also used Google searches for grey literature. The inclusion criteria included articles in English, published within the past 10 years that examined high-quality care in general practice in regard to either quality improvement, or models or approaches to quality care, or attributes, characteristics, elements or indicators of quality care.

The core research team reviewed all papers and documents identified for evidence-based indicators and measures of high-quality PHC relevant to Australian general practice. These were incorporated into the suite of indicators and measures.

In parallel with the literature review, the core research team reviewed indicators and measures WentWest reported to general practices engaged in quality improvement, assessing the evidence for each, and in consensus with the wider team, excluding those not considered relevant or important indicators of quality in Australian general practice. The expression of the indicators and measures was reviewed to ensure each was explicitly defined and measurable (Campbell et al. 2003; Bovend’Eerdt et al. 2009; Table 1). The WentWest list was then expanded to include other measures and indicators identified in the literature.

In consultation with the Advisory Group, we determined a sampling frame of key stakeholders to provide important and varied perspectives. Each PHN invited GPs and practice staff (including practice managers and practice nurses) from within their PHN footprint, relevant PHN staff, and consumer representatives, to attend separate workshops. All attendees signed a participant consent form, including for audio-recording of plenary discussions. The objective of the initial WentWest workshop was to develop a quality framework, and review an early draft of indicators and measures, based largely on those used by WentWest for general practice quality improvement. In the latter two workshops, the objective was to review the expanded suite of indicators and measures arranged according to the agreed framework to determine their feasibility and relevance in Australian general practice.

Participants in each of the workshops were divided into small groups, and each group was allocated a different set of indicators and measures to review. Hence, across both later workshops, each of the indicators and measures was reviewed at least twice by a small group of stakeholders. Each small group appointed a scribe who recorded the group’s decisions regarding relevance and feasibility of each indicator and measure, as well as open-ended comments informing the decision. Plenary discussion was held at the start and conclusion of each workshop. Written reports from the small groups and recorded plenary discussion constituted the data collected for analysis.

All workshop responses were collated and then descriptively categorised. Following the first workshop, development of the evolving quality framework was informed by several rounds of email review and feedback from a smaller number of participants in parallel with similar input from the wider research team and Advisory Group. Following the second and third workshops, a final draft of indicators and measures deemed relevant and feasible in an Australian general practice setting was compiled by the research team.

The research was granted approval by the Western Sydney University Human Research Ethics Committee (H12003). The research was undertaken with appropriate informed consent of participants.

Attributes of high-quality general practice described in the literature were considered in order to provide a framework within which to organise indicators and measures. We selected relevant reports from the RACGP (RACGP 2012, 2015a, 2015b, 2016) as the largest professional body responsible for general practice in Australia, and other key documents, including Australian Commission on Safety and Quality in Health Care (ACSQHC) recommendations for a national indicator set (ASCQHC 2012). We also reviewed the international literature related to models of high-quality primary care. Those attributes of high-quality primary care noted in four or more of the reviewed papers were gathered under a number of overarching descriptors (Metusela et al. 2020b). These descriptors were strongly aligned with the holistic, patient-centred, evidence- and data-informed, team-based approach to PHC described by Bodenheimer et al. (2014). This structure was reviewed in the first workshop, and subsequent correspondence with the descriptors above included under each of four general practice responsibilities reflective of the Quadruple Aim of improving the individual experience of care; improving population health; efficient use of resources; and improving the work life of clinicians and staff (Bodenheimer and Sinsky 2014; Table 2).

Table 2.  Attributes of high-quality care aligned with the Quadruple Aim.

To ensure indicators and measures beyond just process measures were considered, we incorporated a Donabedian model across the framework, seeking to identify structures, processes and outcomes for each attribute (Donabedian 2005; Table 1).

The initial WentWest list included 63 indicators. After reviewing the evidence-base for each indicator, 19 were excluded as not being relevant or important indicators of quality in Australian general practice. This was often because they were used to evaluate local programs (such as mental health services or diabetes case conferencing provided by WentWest). Some had been superseded (such as those related to ‘pap smears’). Others were general rather than specific in their focus (such as ‘patient count’ and ‘revenue not claimed’). All 44 remaining were refined, so that they were clearly defined and measurable (Campbell et al. 2003). Measures were assigned to appropriate indicators and positioned under the relevant attribute. For example, the WentWest measure ‘Allergy Coded/Not Coded’ was reworded ‘% active patients with allergy or nil known coded’ under an indicator labelled ‘Data quality and completeness of demographic and key health data’. This structural indicator addresses data-enabled practice quality improvements under the attribute ‘Professionally accountable’. The WentWest measure ‘FOBT Eligible Male’ was reworded ‘% active patients aged 50–74 years with FOBT recorded in the previous 24 months’ under a process indicator ‘Early detection of cancer’ addressing preventive health care, under the attribute ‘Accountability to our patients’; ‘Patients with Diabetes with poor glycaemic control (Hba1C > 7)’ was reworded ‘% active DM II patients with Hba1C ≤ 8%’ under the outcome indicator ‘Optimal diabetes outcomes’ addressing chronic care under the attribute ‘Accountability to our patients’.

Informed by our second, comprehensive literature review, we added 91 evidence-based measures to those used by WentWest. These were positioned according to the general practice attribute addressed, across all three elements of the Donabedian model, and in particular helped populate indicators and measures for Attribute 3: ‘Accountable to the community’ and Attribute 4: ‘Accountable to society’, which WentWest indicators and measures did not address (Table 2).

Accreditation according to RACGP or the Australian College of Rural and Remote Medicine standards was an assumed baseline measure, so indicators and measures in accreditation standards were not included in our model, which sought to define high quality. As the PIPQI is likely to form the basis of general practice quality improvement data collection into the future, all 10 measures of the PIPQI were included and were given precedence where other similar measures were noted.

We categorised some measures as ‘blue skies’, meaning these were difficult to measure in Australian general practice at the time of our research; however, were important and potentially measurable in the future. This included measures requiring integrated healthcare data, such as the outcome measure ‘use of linked data to measure potentially preventable hospital admissions’.

The 37 workshop participants included general practitioners, practice nurses, practice managers, consumers and PHN staff (Table 3).

Table 3.  Workshops by participant type and number.

The initial 2-h WentWest workshop in August 2019 focused on reviewing the proposed general practice attributes, and an early draft of indicators and measures based largely on those used by WentWest. Participants recommended aligning the attributes we had identified with the Quadruple Aim (Bodenheimer and Sinsky 2014). In adapting this to a general practice context, a framework of ‘accountabilities’ addressing each of the four aims was recommended. In subsequent email discussion with six workshop participants, the attributes identified were aligned with the Quadruple Aim, and expressed as accountabilities to patients, communities, healthcare providers and society more widely (Table 2).

Two subsequent workshops convened by South Western Sydney and Nepean Blue Mountains PHNs provided opportunities to review the expanded suite of indicators and measures. Out of the proposed 135 evidence-based indicators and measures, 125 were agreed to be relevant and 80 feasible, at both workshops. Following these workshops, eight measures were deleted, four were reassigned as ‘blue skies’ measures and four additional measures were added to the suite (Table 4).

Table 4.  Measures removed, added or changed to ‘blue skies’ by the research team.

Descriptive analysis of audio recordings of workshops two and three identified challenges in implementation; and in interpretation of data.

Implementation challenges included GP burden, lack of time and resources, and the importance of correct coding. The need to reduce the burden on practitioners and practice staff while improving practice quality was raised. It was noted that ‘time, staff, funding’ would be required ‘for follow-up implementation’. It was also noted that not all GPs use computers in their practice, particularly in south west Sydney where ‘at least 25% of our solo practitioners don’t use computers’ (GPW2). There was discussion about challenges in consistency of coding, especially where GPs had ‘already formed their habits’. It was suggested that training in coding should begin in postgraduate general practice training.

There were several challenges noted in interpreting data, applicability to specific subpopulations and how to define a high-quality result. The question of ‘how do we know we are measuring quality?’ (GPW2) was raised. It was recommended that future iterations should include specific subpopulations, and that for equity, use of the measures should take account of the practice patient population. It was suggested that improvement at the practice level over time, rather than an absolute outcome be the focus, or alternately, goal ranges could indicate quality rather than a required score: ‘having a range of what is the achievable goal rather than a set mark’ (GPW3).

Following analysis of the data, including ongoing review of the literature and the outcomes of the workshop consultations, a revised suite of 135 indicators and measures, arranged across the agreed framework, was drafted by the core researchers (CM, NC, HVW, JR). This was reviewed, minimally revised, and approved through consensus by the wider research team and Advisory Group. Table 5 provides an overview of the framework. Refer to the Supplementary File S2 and the project report for details of the full suite of indicators and measures (Metusela et al. 2020b).

Table 5.  Indicator headings under the four attributes.

This paper describes the development of a suite of evidence-based indicators and measures of high-quality PHC in Australian general practice. The suite includes structure, process and outcome measures grouped under four attributes, aligned with the Quadruple Aim and is intended to be incorporated into routinely collected general practice quality improvement data. Conforming with international evidence for high-quality PHC, the indicators and measures promote patient-centred, team-based care delivery (Bodenheimer et al. 2014), and outcome measures align with current Australian health priorities (RACGP 2020).

We describe indicators and measures that can largely be drawn from data extracted from general practices; however, accurate and consistent coding is imperative for the data to be reliable (Crossland et al. 2016). This highlights the importance of support and training for general practices (The George Institute 2018). Additionally, a study exploring the facility for current GP software and data extraction tools to collect this data is recommended. Research is also required to understand how the indicators and measures would apply in other settings, such as beyond western Sydney, and in other PHC settings, such as Aboriginal Community Controlled Health Services and in prison health settings.

The controversy surrounding implementation of the initial 10 PIPQI measures highlights the importance of considering the challenges of implementing such measures of quality, and those related to interpretation of the data derived from our 135 indicators and measures, as noted by participants in our research. Addressing these challenges will require further consultation (Woodley 2019).

Importantly, consideration of targets for the indicators and measures will need to take into account the practice setting. Practices in disadvantaged areas may require greater investment to achieve the same health outcomes, compared with practices in more advantaged settings. To avoid negative effects of benchmarking, assessment of quality using these measures should be modified according to practice population. Assessment could also focus on improvement over time rather than achievement of targets (Oregon Health Authority 2018).

Our research focused on urban general practices in one region in Australia, although western Sydney is large and socioeconomically diverse. Future research will engage PHNs in different areas of Australia, including in rural settings, to review the content and construct validity of our model. Our research was also largely limited to GP and practice staff perspectives. Patient-determined indicators and measures will be considered in the next phase of our research.

Although improvement in quality of Australian general practice is widely supported, challenges include an absence of financial reward for providing high-quality PHC and substantial cost to practices seeking to transform to high-quality models of care. Conversely, Australian general practice is unlikely to receive a large injection of government funding until it can demonstrate its capacity to deliver care that is consistently high quality, and achieve outcomes similar to those demonstrated in other settings (Primary Care Collaborative 2017). This research is an early step in defining structures, processes and outcomes that would enable such high-quality health care to be delivered in the Australian setting, and in the longer term provide justification for improved remuneration of Australian general practice.

Our proposed model makes a contribution towards measuring and improving the quality of Australian general practice. Our research is timely, particularly as new funding models and approaches to general practice in Australia are under consideration (Australian Government Department of Health 2017, 2022). With further development, consensus and testing, our model has the potential to inform health systems innovation nationally and in other countries seeking to strengthen PHC.

The data used to generate the results in this paper are available by contacting the research team – email Professor Jenny Reath: j.reath@westernsydney.edu.au.

The authors declare no conflicts of interest.

The research was funded by the Ainsworth Foundation Medical Research Innovation Fund. The funder had no involvement in the preparation of the data or manuscript or the decision to submit for publication.

Supplementary material is available online.