Structured yet simple approaches to primary care data quality improvements can indeed strike gold

Keywords: data quality, general practice, surveillance, morbidity, primary care data, clinical information management.

The availability of data is revolutionising health care, with effective data management having the capacity to inform high-quality clinical decision making and care delivery (Madsen 2015). In primary health care (PHC), the need to transform routine data collection and reporting into data-driven policies and health care planning locally, regionally and nationally is vital in order to address the rising rates of chronic diseases and the ageing population (Levene 2016). In Australia, regionally located Australian Government-funded Primary Health Networks (PHNs) have been created to enhance the efficiency and effectiveness of PHC services, particularly for people who are at risk of poor health outcomes. PHNs also work with general practices to promote coordination of care to ensure that individuals receive care that is tailored to their specific needs (Australian Government Department of Health 2018).

A key aspect of the PHN objectives includes supporting general practices and other healthcare providers to use locally collected data for population surveillance, planning and improving care (Australian Government Department of Health 2018). This requires both assurances that all primary care interaction and clinical data are available for extraction and that these data are accurate. However, general practice clinical management systems in Australia are not currently required to comply with any coding or best practice standards (Davies 2018). Deficiencies in the quality and challenges with ensuring the accuracy of data collected in PHC settings have been widely described (Ghosh et al. 2013). This variability jeopardises the capacity of organisations like PHNs to assist primary care practitioners in undertaking data-driven care service delivery and hampers the use of these data to understand local population needs (Russell and Dawda 2015) and for broader population planning (Bailie et al. 2016). Studies have identified, for example, the collection of accurate demographic data may affect the identification of regional hot spots and future planning for conditions such as obesity (Ghosh et al. 2016) and other chronic conditions (Ghosh et al. 2013; Smurthwaite and Bagheri 2017).

The Sentinel Practices Data Sourcing (SPDS) project was established in 2012 to address the accuracy and comprehensiveness of PHC data within South Eastern New South Wales (SENSW), Australia. This has resulted in the use of a structured process for data cleansing and improvement of data accuracy by participating general practices across the region (Ghosh et al. 2014). Findings from these data are used for both patient and general practice level clinical improvements, as well as for regional population health planning. The SPDS project is a continuous and evolving body of work around the improvement of data quality and data utilisation in the region. It has enabled analysis, identification of regional pockets of key chronic conditions and the management of local population health issues, such as the impact of socioeconomic disadvantage on levels of obesity (Ghosh et al. 2016). This paper describes a mixed-methods appraisal of the SPDS project to investigate the improvement in data quality and satisfaction with the program.

A mixed-methods evaluation was undertaken to assess preliminary impact, experience and outcomes of the project over a 12 month period (January 2018–January 2019). Quantitative data provided insights into the effect of the data quality initiative, and qualitative data revealed the experiences of the practice team in the implementation of the program.

Although the SPDS project’s chronic disease surveillance methodology has been reported elsewhere (Ghosh et al. 2013), its continuous and progressive quality improvement methodology can be summarised as a coordinated approach that works on principles of capacity building and staff empowerment. This is facilitated through individual and workshop training, education and advocacy provided by a dedicated project support team and PHN staff, along with the provision of a user manual and other relevant resources. The project team works with general practices using a stepwise approach to continuous and progressive quality improvement on a consistent set of measurable yet practical metrics adapted and inspired from several internationally proven quality improvement and system redesign methodologies (Geonnotti et al. 2015; Project Management Group 2017; Australian and New Zealand College of Anaesthetists (ANZCA) 2018; McCalman et al. 2018; Fig. 1). The project centres on supporting participating general practices to meaningfully use their practice data in a planned way, eventuating in incrementally escalating clinical data auditing.

Fig. 1  The Sentinel Practices Data Sourcing project’s continuous and progressive quality improvement methodology.

The project team provides ongoing training and support to participating practices to enable implementation of steps to achieve data quality improvement on a set of agreed indicators. Through exercises of peer-based performance benchmarking on key indicators, practices are given three-tiered targets (Fig. 2) for every quarter to continuously assess their performance and strive to constantly improve. This facilitates general practices to then engage in their own regular data-driven performance and health outcomes monitoring.

Fig. 2  Three tiered Sentinel Practices Data Sourcing targets used in quarterly performance benchmarking. PHN, Primary Health Network.

Ninety-nine practices within a single PHN in regional NSW were recruited to participate in the SPDS program during the study period. To be eligible, practices had to be computerised with compatible electronic medical records software and willing to engage in the quality improvement program. Practice staff, including GPs, nurses and practice managers, who were involved in the implementation of the program were sought to participate in the semi-structured interviews. Interview participants were recruited sequentially from those who were contacted and volunteered to participate until data saturation was achieved.

Quantitative evaluation consisted of establishing and monitoring quality improvement indicators. The SPDS project’s quality improvement indicators are a subset of a larger list of indicators that have been adapted from several sources, which include the Australian primary care accreditation and compliance guidelines (Royal Australian College of General Practice 2019) and evidence-based clinical practice guidelines and primary care indicator sets, specifically the UK quality and outcomes framework indicators (Lester and Campbell 2010; National Institute for Health and Care Excellence 2018). A subset of 10 indicators was chosen through consultation with clinicians at the commencement of the project. These were replaced with new indicators as improvements were achieved. This continuous update process was selected to maintain a manageable number of indicators and to avoid perceptions of quality improvement initiatives being too onerous or resource intensive. The 10 indicators used at the inception of the SPDS project were used for the evaluation reporting in this paper (Table 1). Data on project indicators from all participating practices were collected each quarter by extracting aggregated data from practice software.

Table 1.  Indicator definitions
‘Active’ patients used in this project are patients show have visited their general practice at least once in the past 2 years from the date of data extract. This definition differs from the Royal Australian College of General Practitioners (RACGP 2010) defined measure because this project aims to target a wider cohort of primary care consumers so that vulnerable patients at risk of poor health outcomes can be followed-up more comprehensively and more inclusively. BMI, body mass index; BP, blood pressure; CHD, coronary heart disease; COPD, chronic obstructive pulmonary disease; EHR, electronic health record; eGFR, estimated glomerular filtration rate

A semi-structured interview schedule was developed by the researchers, based on their experience with the program and a review of the literature. Given the geographical spread of participants, most interviews were conducted via telephone by one of two research assistants. The interviewers used prompts and probing questions to explore participants’ experiences in more detail. Each interview was audio recorded and transcribed verbatim by a professional transcription company. Interviewers recorded field notes about their perceptions and observations following each interview.

Aggregated quantitative data were exported from the practice software, converted and analysed using Microsoft Excel. Data from all participating practices consistently for all five extraction quarters (n = 99) were analysed for a baseline versus end-of-period comparison. Basic proportional changes in percentage scores for all indicators were calculated at the PHN catchment level. For all indicators, PHN-level aggregated sum totals of patient counts were calculated for baseline extract (January 2018) and follow-up extract (January 2019). Using a z-score value of 1.96, 95% confidence intervals (CIs) were calculated from the margin of error, which was calculated from the standard deviation of the point estimates from each indicator in both time points (baseline and follow-up). Because the analysis was based on aggregated data, differences between baseline and follow-up data were considered statistically significant (P < 0.05) where there was no overlap of CIs.

Transcripts of the interviews were imported into NVivo Version 10. The transcripts were then analysed separately by two researchers (EH and SM) using the thematic analysis approach reported by Braun and Clarke (2006). Results were discussed among the project team, and agreement on themes was reached.

The study was performed with the approval of the Human Research Ethics Committee (Health and Medical) of the University of Wollongong (HE13/433). Written informed consent was obtained from individual participants for all qualitative data. Quantitative practice-level data sharing was supported by detailed service-level agreements along with interagency data security agreements. Names have been removed in order to protect the identity of participants.

Aggregated scores demonstrated improvements in all 10 indicators (Fig. 3). Most statistically significant improvements were observed in patient identification indicators, such as in the identification and accurate recording of patient ethnicity (whether Indigenous (i.e. Aboriginal and/or Torres Strait Islander) or non-Indigenous), and in preventive and health screening indicators, such as recording of smoking status, height, weight and body mass index for adult patients (Fig. 4). Chronic condition monitoring variables also showed modest improvements. Although the quantum of these improvements was variable across indicators, they demonstrated positive trends suggestive of growing momentum, and are estimated to be significant with time and continuation of the project.

Fig. 3  Quality improvement report card for the SENSW PHN catchment. BMI, body mass index; BP, blood pressure; CHD, coronary heart disease; COPD, chronic obstructive pulmonary disease; eGFR, estimated glomerular filtration rate.

Fig. 4  Quality improvement statistical results (point estimates and confidence intervals for both time periods) for the SENSW PHN catchment. BMI, body mass index; BP, blood pressure; CHD, coronary heart disease; COPD, chronic obstructive pulmonary disease; eGFR, estimated glomerular filtration rate.

Overall, 48 interviews were undertaken. Half the interviews were conducted with practice managers (PM) and administrative staff (R; n = 24; 50%), whilst one-third were with nurses (GPN; n = 16) and eight (17.7%) were conducted with GPs. Most participants were female (n = 40; 83.3%) and aged between 35 and 64 years (n = 34; 70.8%). As can be seen from Table 2, participants were drawn from across 12 PHC ‘clusters’, or medical neighbourhood areas created by the PHN.

Table 2.  Participant profile

Participants described that the project positively affected data management, although the degree of impact was related to the level of practice engagement and commitment to the project. Several participants described having ‘a better understanding’ (Participant 14, GPN; Participant 26, GPN) and that ‘it’s made the staff more aware’ (Participant 15, R) of the importance of data cleansing and management since attending the workshops and receiving support from PHN staff. It was recognised that clinical software tools ‘are quite useful when the data is inputted correctly’ (Participant 8, GPN). Others identified how they had ‘a better understanding of the software’ (Participant 14, GPN) and felt more confident that they were able to use clinical software to its full capacity. One participant encapsulated this feedback, stating ‘It’s just been an eye opener for me really’ (Participant 12, PM).

The gains in knowledge, skills and confidence meant that participants ‘use the software more now than previously’ (Participant 13, GPN):

Yeah, so we’re using it far more widely, rather than just the areas that we’ve had a chance to get into and identify as being pertinent to us. So, as we go along, we’re actually using far more fields within Pen CAT [the SPDS project’s data auditing tool], realising that it has a far wider capability than we used to use it for [Participant 34, PM].

Participating in the SPDS project caused many participants to reflect on the quality of their clinical data and the training needs of staff new to the practice:

Overall it’s highlighted to us how poor we were at our data entry. The [practice manager is] all over it now that when we get new staff, that she points out what’s acceptable and what’s not acceptable [Participant 14, GPN].

It’s been beneficial…because it identifies patients that need something updated on their file, like in the demographic area. I’m talking about the admin side of it. I don’t know if there’d be any non-benefits for it or – the only difficult thing is, I suppose, is finding the time to investigate a bit further and use it probably to its full potential [Participant 24, PM].

Participation was also described as key to improving practice around clinical data management:

I think we’re able to capture better data now because not only have we cleansed that up but I think we’re entering better data now as well. So it’s been good for us to learn from it and hopefully we’re producing better data as an outcome [Participant 14, GPN].

We thought we were so on top of it but when I actually looked at the data,…people weren’t recording in the right spot and I was going ‘Oh my God’. We’ve been doing this for years and I was shocked that people didn’t know, even though we’d had meetings saying does everyone know where to put this information and it turned out they really didn’t. That had shocked me, like I was very shocked because I thought we were all on the same page [Participant 21, GPN].

In contrast to practices engaged and committed to the program, others did not follow up on skills learned:

I went and did the training, but that’s about where it’s begun and ended [Participant 15, R].

Unfortunately because [one] of the doctors left, I haven’t been able to use it since she had left. So no, it hasn’t really helped in any way really with us. When we did have to download the data and everything like that, it was so much simpler than manually going through and everything. That was very easy. But unfortunately now, we don’t use it at the moment. [Participant 2, R].

Personal experiences and perceived opinions on the implementation of the SPDS project were very positive and encouraging:

So over a long time since we’ve been doing this program in…[the PHN] I would have to say yes they’ve all been beneficial to the practice. Because it’s teaching – it’s also teaching the doctors more of how to use best practice as well when they’re seeing patients [Participant 29, PM].

Participants spoke about their experiences with the four key aspects of the program, namely the training workshop, ongoing training, project team support and the instruction manual.

The workshops were described as very useful, because they allowed people to network, share experiences and develop an equal understanding of the nature of the project and data management:

I loved the workshop. Also it was the fact that everybody came together, because of the other practices involved and it kind of encouraged you to just…have a go at it and be involved in it. The training was excellent [Participant 12, PM].

It’s good in that when you have the workshops you meet other people and they can tell you about their experiences and how they do things in their practice. Sometimes things don’t work in your practice or aren’t working well and you come across somebody else who says X, Y, Z in their practice that they do this, and it works a bit better. Then you can try and incorporate that into your practice [Participant 8, GPN].

Although the workshops were positively received, ongoing support was identified as being important to maintain the momentum achieved:

I came back really invigorated and enthusiastic from it, but that was a while ago now, and so I sort of lost the enthusiasm perhaps [Participant 15, R].

Many participants emphasised the importance of having someone who:

…could come to a practice, and show the set-up of how to use the tools. I know…they’ve got a lot of people that they connect in [remotely] and they show you how to do it over the telephone. But sometimes it’s better in person, so you can ask questions, and you might not know something until you go in and start using it [Participant 8, GPN].

Participants overwhelmingly reported feeling very well supported by project staff. This high-level support was described as a key factor in the success of the project:

The support was great, both from [the PHN] and the project’s technical support team. I think they all did a fabulous job on that [Participant 3, PM].

Very helpful. Any time I email [contact person] and say ‘Can we go through this?’, he’s like ‘Yeah, let’s make an appointment’, and he makes an appointment that week for me. It’s very helpful [Participant 14, GPN].

She’s constantly in touch, also she’s come out with [practice support officer]…he actually came in and showed me exactly how to follow the manual and that sort of thing… her support has been good [Participant 33, PM].

All I can do is highly commend the [PHN] team on this project and teaching and guidance [Participant 18, GPN/PM].

The manual developed by the project team was also seen as a valuable tool and ongoing resource that was regularly used:

I’m always pulling out the manual, always going back to the manual. It’s like a little [first-aid] there. It’s useful. Especially if you’re not doing it every day you need to refer all the time [Participant 12, PM].

The manual that’s been my biggest help, I can just always go back and look at it [Participant 12, PM].

The book is excellent, the book is a fantastic resource. I mean it’s just simple, you just do what it says and it works. So there’s no impediment there [Participant 16, GP].

A few participants identified areas where they saw that the program could be further developed or improved:

Is it possible if you get feedback from different practices about the diagnoses that are not coded in the system, whether you could feedback to the actual software and say, Medical Director, or Best Practice to say you haven’t got these coded, you need to put these things in there [Participant 6, GP].

Maybe a refresher every so often for the other staff who haven’t trained [Participant 6, GP].

Another aspect identified for improvement was the need for the project team to oversee implementation and provide ongoing support. Participant 9 (PM) felt that it would be useful to have the team visiting:

…maybe once a month, but then after a few months, you could possibly drop back. I think really just keeping it on our agenda and running little sessions – just making sure we’re still going okay and just keeping it sort of current with us, because it can sort of slip off…the back burner [Participant 23, GPN].

Participants also provided feedback about the perceived value of expanding the program to other practices:

I guess I just hope that maybe the project or something like it will be offered again for other people who didn’t uptake the first time. Maybe feedback will come out how good it was [Participant 3, PM].

Having programs like this is extremely helpful and will be essential in the future to find out what our patient database is [Participant 32, PM].

The importance of having access to accurate data to undertake population planning and improve the provision of PHC has been widely recognised by governments and health planners (Thorlby et al. 2011). In Australia, recent changes to PHC policies and funding (Australian Government Department of Health 2018) have resulted in exploration of local processes such as the SPDS to improve data quality (Crossland et al. 2014; Davies 2018). Although PHNs and general practices are required to develop their use of the available technologies to overcome data quality inadequacies, there has been little guidance to tackle the data quality issue, and no nationally accepted standards for coding (Davies 2018). The SPDS is a continuous and evolving body of work around the improvement of data quality and data utilisation in general practices across the participating PHN (Ghosh et al. 2016). This report of the outcomes and end user experiences demonstrates that it is a simple structured intervention that has achieved positive outcomes in key indicators and has been very well received by participants. It has led to quantifiable improvements in indicators directly related to improving patient care over a relatively short time frame and, as reported elsewhere (Ghosh et al. 2016), it has demonstrated the capacity to be used in population surveillance across the region.

A major factor in achieving quality health data has been identified as managing the ‘human element’ (Munyisia et al. 2017). This includes educating people in the systems and training them to accurately record data, being cognisant of varying levels of computer literacy and software familiarity (Avery et al. 2007). Efforts to improve data cleansing processes in PHC require initial and ongoing support to practices and individuals, and access to resources (Crossland et al. 2014, 2016).

The difficulties in engaging with practices to focus on data improvement and the need to have champions to drive change have previously been reported (Schattner et al. 2011). A major contribution to this project’s success was the establishment of a team of highly skilled professionals who developed resources and were the facilitators working with practice staff to provide education and individual support both face to face and electronically. Of particular value was the data cleansing manual, which provided immediate access to guide practice staff in problem solving issues as they arose. Participating in the project also led to increased understanding of the contribution of data entry within the practice to population-level surveillance and how improvements in practice data can enhance the quality of care delivered in general practice. Although face-to-face support was noted to be particularly helpful, it is anticipated that the SPDS model could be replicated in similar regions to improve the quality of data recorded, and used across a range of urban, rural and remote locations through the provision of well-designed resources and regular communication via videoconferencing and other communication technologies, such as those described by Anikeeva and Bywood (2013). However, the findings may have limited generalisability in some regions of Australia or in other countries where access to relevant data and professional support is limited.

Despite the level of support provided by the project team, barriers such as practice management issues, staff engagement, time factors, workload and technical issues negatively affected the implementation of the project in some locations. These barriers are not dissimilar to those reported elsewhere around the implementation of various interventions in general practice (Halcomb et al. 2015; Stephen et al. 2018). The development of additional strategies to address these challenges has the potential to increase uptake by practices, and further improve outcomes.

As a result of this evaluation, the PHN has identified its own opportunities for further improvement to quality monitoring and reporting to practices. Development and support of formal peer-to-peer benchmarking at practice level will also be explored to assist in the long-term sustainability of this work.

This paper reports the experiences of one PHN implementing a data improvement program across a selection of engaged general practices. Although the generalisability of the data is enhanced by the mix of practices participating, with variable uptake and implementation, local cultural practices and relationships may have affected the way in which the intervention was delivered. In addition, although many attempts were made to contact a range of practice staff to participate in the interviews, there was less uptake of this by nurses and GPs than administrative staff and practice managers. This may mean that the experiences of GPs and nurses are less visible in the qualitative data.

In order to plan effectively for future healthcare needs of local and wider populations, quality PHC data must be readily available and accessible. PHC staff need to understand the importance of accurate data recording, data cleansing and data utilisation to patient care and health planning. Successful local initiatives, such as the SPDS, provide evidence of the importance of such approaches. Initial evaluation of this program has demonstrated the benefits of having expert teams available to provide appropriate levels of support and ongoing feedback to ensure that quality data are available and used to improve patient outcomes. Such local approaches also have wider significance because they provide opportunities for local, regional and national benchmarking, as well as guiding policy development at different levels of government.

The authors declare no conflicts of interest.