Contextual antecedents of quality improvement: a comparative case study in rural, urban and Kaupapa Māori general practice

Keywords: case study, context, distributed leadership, general practice, implementation, learning climate, networks, quality improvement, reflection and sensemaking, relationships, teamwork.

People are living longer with multiple chronic health conditions while health-care costs are rising. Health systems increasingly look to primary care to achieve quality, patient-centred integrated care while improving the sustainability of the health system.^1–4 Definitions of health-care quality vary depending on context and perspective.^5,6 Our definition is a New Zealand (NZ) adaption of the quadruple aim^7–9 specifying value rather than cost, and using the NZ quality dimensions, which use equity and access within the aim of improving individual quality, safety and experience of care^5,10,11 (see Table 1 below for the full definition).

Table 1.  Definition of health-care quality, the quadruple aim.

Health-care interest in quality improvement (QI) has grown, as achieving the aims outlined proves elusive.¹² Results from QI efforts have varied^13,14 and interest in understanding the ‘black box’ of quality improvement has grown.^15,16 We use the Batalden¹⁷ definition of QI ‘as the combined and unceasing efforts of everyone—health-care professionals, patients and their families, researchers, payers, planners and educators—to make the changes that will lead to better patient outcomes (health), better system performance (care) and better professional development (learning)’.

Understanding the factors that influence successful QI in primary care is critical to enable high-performing primary health-care systems.^2,3 Context contributes to the variation in improvement outcomes seen in many initiatives.^18–20 However, information on the impact of contextual variation in primary care is limited.^4,21,22 Definitions of context vary,^19,20 and there is confusion about the extent, and how, context is interrelated with other elements of improvement efforts, the improvement intervention and implementation process.^13,19,20

The persistence of unsafe and variable health care and the unreliable impact of quality improvement interventions (QII) in health care has generated significant research seeking to increase QI success.^13,23 This research has seen the development of the Consolidated Framework for Implementation Research (CFIR)²⁴ and the Model for Understanding Success in Quality (MUSIQ).¹⁸ Culture, leadership and teamwork are included in these frameworks as contextual factors.^18,24,25

Fig. 1 synthesises the current research into understanding the factors and interrelationships influencing QI. In this model, ‘process’ is the implementation process, including improvement methodology and how this is facilitated.^19,20,22 The quality improvement intervention (QII) refers to the improvement topic, the planned changes and their characteristics such as adaptability.^19,24 Context factors are not part of the QI process or intervention, but influence or surround the implementation effort.^19,20,24

Fig. 1.  Primary care quality improvement.

The bi-directional arrows in Fig. 1 represent the interrelationships between context, intervention, and process.^19,26,27 The QII may be a better fit in some contexts and with certain methods than others.²⁶ A successful QII needs to be simple to implement, show a relative advantage over the current state, and generate support from the front-line.^14,16,28 Success is enhanced if the initiative is perceived as necessary and the planned changes as appropriate, acceptable, and adaptable to the local context.^14,16,28

Most studies into factors influencing QI success have focussed on secondary care organisations and large-scale collaborative efforts.^2,29 Primary care services, such as general practice, present different challenges to change as they vary considerably, are smaller and more numerous than secondary care services.^4,28,30 Addressing this gap prompts the following questions:

1. What are the contextual elements influencing primary care improvement initiatives?

2. How do contextual factors (inner and outer setting), quality improvement interventions (topic and planned changes) and the implementation process, influence each other and the improvement outcomes in primary care?

A qualitative multi-case study approach was used in conjunction with the CFIR²⁴ to guide data collection and analysis. The qualitative case study approach was chosen to provide an in-depth view of primary care QI, with the researcher as the primary instrument of data collection.³¹ The CFIR was included because of its fit with quality improvement theory and generic conceptual applicability.^32,33

The five major domains of CFIR are illustrated in Fig. 2; these are well aligned with the constructs in the primary care improvement model shown in Fig. 1. The study results are reported using the five CFIR domains, with the addition of project outcome.

Fig. 2.  Consolidated Framework for Implementation Research (CFIR) Domains.

Sites were selected by purposeful theoretical sampling to include a mix of general practice settings and improvement topics typical of the New Zealand general practice environment; specifically, a rural, urban and kaupapa Māori practice. Site characteristics and their improvement topics are outlined in Table 2. Suitable sites were identified via Primary Health Organisation (PHO) referrals of practices who had completed a successful QII to provide a view of ‘what worked’.

Table 2.  Quality improvement intervention topic and site characteristics.

Qualitative data were collected from the teams through in-person, semi-structured interviews between October 2018 and January 2019. The practice provided written consent for participation in the research, identified the QII and interviewees. A semi-structured interview guide was developed based on our model for QI and the ‘Interview Guide Tool’ available on the CFIR website: www.cfirguide.org. All interviews were recorded, and the data were transcribed and sent back to the participants for verification. NVivo qualitative analysis software (QSR International) was used to draw out the themes both intuitively and regarding the CFIR constructs. Themes were revisited and revised in an iterative process throughout the writing and reflecting process.

Specific care was taken with the data from the kaupapa Māori provider, as our worldview is that of the pākehā, (non-Māori) New Zealander. Even though we strive to be respectful of a Māori worldview, we do not have the lived experience and embedded cultural understanding of being Māori. Feedback from the case study organisation on interpretation of the information reported was vital to ensure correct representation of Māori concepts and perspectives. The draft report was provided to the senior leadership team at the practice for feedback, minor errors were addressed, and a follow-up meeting was held with the community board to discuss the results and conclusions.

This study was deemed low-risk and not requiring a full ethical review (Massey University Ethics Notification No. 4000018920).

The case characteristics, staff interviewees, the intervention and the reason for undertaking the study are outlined in Table 2. Cases 2 and 3 are based in urban settings, and Case 1 in a remote rural setting.

Representative quotes illustrating the common themes and the project outcomes for each Case are provided in Supplementary Material S1. The quotes are grouped under themes that are a mix of CFIR categories and intuitive themes identified during the study.

The Cases reported successful QIIs in achieving the desired patient outcomes of improved access and experience, and in benefitting the practice and staff. Methods of evaluating success varied. Case 1 monitored a patient list and undertook an internal evaluation based on patient and staff feedback and performance against this patient list. Feedback from patients and staff were the key evaluation sources for the other two Cases. Case 2 also collected time series data related to the aim of their project (see Supplementary Material S2). Case 3 manually collected specific quantitative data at times to answer specific queries.

Teams chose improvement topics based on their context; an identified patient need that aligned with their values. The QII topics shown in Table 2 have similarities as they are all about access, one of the quality dimensions. Case 1 aimed to improve physical health services for mental health consumers, Case 2 implemented Doctor telephone triage and Case 3 a template-based receptionist prioritisation process. Staff from each case commented on their focus on what was best for the patient. There was tension for change and a perceived relative advantage for patients, staff and the practice. The choice of QII, capacity, capability, and values of the inner setting drove team implementation. In Cases 2 and 3, the teams had researched their initiatives. In all cases, the ability to adapt the QII to fit their context was key.

No formal improvement methodology or patient co-design was used in these QIIs, although elements of other change methods were used based on the experience of team leaders. Limitations were noted in the ability to collect quantitative data, which often required manual collection. Case 2 de-identified data are shown in Supplementary Material S2. Informal problem-solving approaches of testing ideas, reviewing, and making rapid changes from feedback were key. Qualitative patient and staff feedback, team discussions, reflection and sensemaking were used to plan and review actions in an iterative process.

All the Cases have a network of relationships with other health organisations, providers and community organisations locally that includes their PHOs and DHBs. In Case 1, the practice is owned by the PHO, whereas in Cases 2 and 3, members of the practice team have governance roles with their PHO. These close relationships also meant that the practices were aware of new initiatives and reports that would have an impact on them such as the Health Care Home model and the national Equally Well movement for patients with mental health diagnoses. This provided evidence in support of the QII and the need for change.

The ‘patients’ needs and resources’ is an ‘outer setting’ construct in the CFIR model,²⁴ but here the boundaries overlapped with the inner setting. This was noted in Case 1, and particularly in Case 2 where the needs and wellbeing of the community is deeply embedded in the culture of the organisation.

All of the team members in each Case identified a strong team culture with respect, loyalty and high regard for each other and their relationships. In Case 1, this was strengthened by the remote rural location, and in Case 3, it was based on relationships and respect built over time. Case 2 culture founded in Māori kaupapa values, particularly whanaungatanga (defined by the practice as relationships, kinship, and connections), was commented on by all participants.

All Cases describe a continuous improvement culture with many improvement efforts. The quotes show a learning climate that supports a culture of innovation and continuous improvement. A psychologically safe learning climate where staff feel valued and listened to was present to varying degrees.

All teams had very capable members and leaders, with several comments about a ‘can-do’ attitude. Key staff with strong internal and external relationships levered these to learn, share, and engage support. Staff shared leadership roles for the project in varying ways and at varying times in the projects in a combination of roles including implementation leaders, relationship builders and carrying out necessary tasks.

The findings are synthesised into a revised model depicting the factors influencing primary care QI (Fig. 3). The model shows overlapping domains due to their interrelatedness. These relationships are described in Supplementary Material S3. The cross-case comparison in Table 3, highlights the relative strength of the identified themes.

Fig. 3.  Revised model of quality improvement in primary care.

Table 3.  Comparison of factor strength between the three cases.

The QIIs were strongly connected to the context in all three cases. The teams perceived a fluid, complex relationship between inner and outer settings rather than a strong boundary. The QII was chosen to address a pressing patient need with clear benefits for patients and staff, creating motivation and a shared purpose.

The evaluation of QII impact in health care is problematic due to the combination of multi-factorial interventions, complex dynamic settings and limitations in data.^34–36 As noted by one participant, ‘The data is limited in general practice, it is strong around things that are paid for and variable around the rest.’ These teams are embedded within the systems they are improving, and staff and patient experience narratives are a reasonable assessment of the value of their QIIs for their use and this study, despite the value of valid time-series data to support learning from improvement.^34–36

The QII was related to outer settings through community and networks with local and national organisations providing support and expertise. Organisational leaders and staff were confident in the relative advantage of the QII and their ability to adapt the QII to their setting. The strong community and network relationships of the practices meant they were able to both draw on and provide resources across these networks. Different priorities of the factors identified in Fig. 3 for each case and relative to each other are shown in Table 3. The identified factors are present in all cases, but some factors were more critical across the cases. The weighting given to these factors was based on the frequency and passion associated with the participants' comments.

Context is sometimes referred to as the ‘why’ of a QII as it provides the reason the initiative is commenced.¹⁹ The strong sense of responsibility for and connection to their patients and community provided motivation to pursue and sustain the improvement. This was fundamental in the rural Case and in Case 2 where it was embedded in the practice by the deep cultural connection of manaakitanga and whanaungatanga that provided motivation for the improvement and how it was carried out. These values are not able to be completely described through a western world view; they are inseparable from who they are as an organisation. A sense of community connection and responsibility is not something that has been identified in secondary care improvement studies, but has been found in high-performing Indigenous primary care services³⁷ and may be a factor found in other high-performing primary care providers.

Despite the lack of formal improvement methodology in any of these cases, there were common practices followed by all three teams that will be useful for primary care teams with or without formal QI expertise. The QIIs were based on research into the topic that showed relative advantage in the QII. Implementation was supported by regular team meetings and an engaged team that exhibited agency. An iterative learning process was followed, with feedback consisting of staff and patient feedback and data where available. Regular communication and this iterative process of learning were key elements of the process followed by the teams.

Specific individual champions were not identified, instead, there were several champions and a cross-over with other CFIR constructs of ‘opinion leaders’ and ‘implementation leaders’.³³ We found different types of leadership behaviours shared among the team. Every person played a role, depending on what was needed at the time, demonstrating distributed leadership that relied on relationships and influence rather than a specific champion. This finding contrasts with secondary care studies,³⁸ but is similar to the findings of QII research in smaller primary care organisations^39,40 and that of Ilott et al.³³

The learning climate varied among the teams in subtle ways, with Case 2 being particularly time poor and Case 1 involving several members working together for the first time. However, they all felt psychologically safe to try new things and reflect and evaluate as equal partners. All team members individually and collectively exhibited ‘agency’ in their ‘can-do’ attitudes to improvement. Agency is the ability to make choices and to act with purpose when faced with uncertainty or difficulties.^41,42 Although the initiator of the QII may be the first person to display agency, every team member demonstrated agency and contributed to the successful outcome.

The teams learnt their way towards successful outcomes through collaboration, iterations of testing ideas, natural sensemaking narratives and conversations, learning and perseverance. It was these factors, the relationships, communication, learning climate and the team culture that brought about improvement. These human-centred and relationship-based processes are vital to the holistic approaches required in complex health-care systems.⁴³

Sites visits and interviews took place between October 2018 and January 2019 and some of the QIIs were carried earlier. All the teams still work on their QIIs and felt confident that the details were still fresh in their minds. However, time may have altered their recollection of events. The reliance on mainly subjective assessments of QII success is a limitation in this study, which was also reported in the development of the MUSIQ assessment tool.²³ All cases were contacted through PHOs, potentially skewing the sample. This comparison study only considers three cases, but primary care services are many and varied. Further research such as a larger survey is required to assess the wider applicability of these findings.

The findings support the importance of distributed leadership, teamwork and an iterative approach involving reflection, sensemaking and adaptation^30,41,44 when conducting QI in primary care. It was the strength of practice relationships, collaborative learning, commitment to change, perseverance and distributed leadership that helped teams to navigate their way to success. The underpinning values of the organisations, their connections and commitment to their community were core motivating factors. This community commitment has not been reported in the secondary care literature and may be a unique factor found in high-performing primary care practices.

The teams evaluated progress to test and refine their ideas iteratively to achieve the desired outcome without the use of formal improvement methods. This does not necessarily mean that formal improvement methods would not have been useful, but here, they were not essential for success. The successful outcomes were driven by the interrelationships between the QII, the process and context as the process of change, and the QII all arose from and were connected to context. This suggests that when there are strong community and network connections for knowledge and support, within a setting of a learning climate, distributed leadership, teamwork, and time is found for reflection and sensemaking, a lack of formal improvement methods can be overcome. It is also possible that the fundamental principles that underpin improvement methods are the enabling capabilities rather than specific tools and techniques.

Deidentified interview transcripts are available on request from the lead author.

The authors declare that they have no conflicts of interest.

This research did not receive any specific funding.

Supplementary material is available online.