Process evaluation of the Safer Prescribing and Care for the Elderly (SPACE) cluster randomised controlled trial in New Zealand general practice

Keywords: aged care, prescription medicines, primary health care, quality and safety, randomised trials.

Efforts are ongoing to identify interventions that support safer medication prescribing in general practice to minimise adverse drug events (ADEs) and related hospitalisations.^1,2 The Safer Prescribing and Care for the Elderly (SPACE) cluster randomised controlled trial was conducted in New Zealand general practice from 2018 to 2020 to investigate the effect of the SPACE intervention on high-risk prescribing of non-steroidal anti-inflammatory and/or antiplatelet medicines and related hospitalisations.³ High-risk prescribing places patients at increased risk of ADEs. SPACE is a complex intervention involving: (i) an automated search of practice records to identify and generate for each general practitioner (GP) a list of patients with high-risk prescribing for the prescribing topic; outreach visit from pharmacist to deliver: (ii) group education to GPs (30–60 min) and (iii) one-on-one feedback with each GP (15–30 min); (iv) tick-box for GPs to indicate an intended action for each patient in their list (Letter; No letter but possible change and/or discussion with patient when next in; No action); and (v) letter from a GP to selected patients inviting them to discuss their medicines when they are next in (Supplementary Table S1).³ All GPs who attended the one-on-one session with the study pharmacist received a NZ$100 gift voucher. The trial found that SPACE decreased high-risk prescribing for gastrointestinal ADEs at 6 months, primarily by increasing proton-pump inhibitor protection, but did not decrease high-risk prescribing for ardiovascular or renal ADEs. This partial effect was not sustained at 12 months and there was no difference in ADE-related hospitalisations. The detailed methods and results have been published previously.^3–5

Process evaluations are recommended to understand how and why an intervention has or has not worked, and to help understand variation in responses and discrepancies between expected and observed effect.^6,7 Process evaluations can help to understand whether an intervention was implemented and delivered in the way that was intended and reasons for variation, and can outline contextual influences on effectiveness to inform future intervention development and implementation.^6,7 Reported here is the SPACE process evaluation describing recruitment of practices and GPs, intervention delivery and uptake, and GP responses to the intervention.

The process evaluation was planned prospectively using a mixed methods approach including quantitative and qualitative data. The evaluation is reported using the framework developed by Grant et al.⁶ The SPACE cluster randomised controlled trial was registered with the Australasian Clinical Trials Register (ACTRN12618000034235, January 2018), and approved by the University of Auckland Human Participants Ethics Committee (Ref. UAHPEC 020092).

The SPACE trial was conducted in 39 general practices with 21 867 participants identified as at increased risk of experiencing gastrointestinal, renal or cardiac ADEs from non-steroidal anti-inflammatory or anti-platelet medications at baseline, of whom 1479 (6.8%) had received high-risk prescribing.³ Of the 39 practices, 20 were allocated to the intervention group, including 100 GPs and 613 participants with high-risk prescribing at baseline. All practices had electronic medical records and used practice management systems compatible with the trial system for a remote electronic search of the practice database.

Quantitative data were collected from a number of sources including: (i) publicly available data on the number of GPs working at practices; (ii) log of practice recruitment; (iii) practice demographic data on forms completed by recruited practices; and (iv) log of intervention delivery maintained by study pharmacist that included a number of GPs attending the group education, number of GPs attending the one-on-one sessions to review their list of patients with high-risk prescribing, and GP tick-box selection for each patient. Practice variables collected and included in the regression analyses were: practice list size, number of GPs, proportion of patients aged ≥65 years, proportion of patients of Māori or Pasifika ethnicity, whether the practice was a ‘high needs’ practice, practice accreditation status, whether the practice taught medical students, or whether the practice allowed online electronic ordering of repeat prescriptions, had a system of medications reconciliation or tagged a diagnosis to prescriptions. A ‘high needs’ practice was defined as having at least 50% of enrolled patients from the lowest socio-economic quintile or Māori or Pasifika ethnicity. ‘High needs’ practices are funded differently, requiring the practice to charge lower patient fees.⁸ Qualitative data were collected by the trial pharmacist in an electronic reflective journal summarising GP comments and responses, and pharmacists’ perceptions of interactions and descriptions of intervention delivery.

Quantitative analyses used descriptive statistics to compare characteristics of recruited practices with those of eligible practices that declined, to ascertain generalisability of results. Univariate and multivariate logistic regression models,including variables listed above, were used to examine associations between practices and variation in intervention uptake and GP responses. To assess whether practice characteristics were associated with level of engagement and responses, a sensitivity analysis was conducted, excluding intervention practices that did not engage at all. General inductive analysis was used to categorise qualitative data.

A search of GP databases identified 220 general practices in the study region. Of these, less than half were eligible for the trial (101/220) due to earlier participation in the SPACE pilot, involvement in a contemporaneous non-trial safer prescribing initiative, or using practice management software not compatible with the study clinical outcomes data extraction process. Thirty-nine practices (39%) agreed to participate. Larger practices with more GPs tended to be more likely to participate (mean (s.d.) number of GPs 5.1 (3.4) vs 3.8 (3.1), P = 0.08). The most common reason for practices declining was ‘too busy’.

Twenty practices were randomised to receive the intervention, including 100 GPs and 613 patients identified as having high-risk prescribing at baseline (Table 1). Of the 100 GPs, only 64 (64%) attended the one-on-one session with the study pharmacist, reviewing only 416 of the 613 (68%) patients (Table 1). In two large practices, no GP engaged with the intervention, and in a third practice, only one GP engaged, resulting in 197 (32%) patients receiving no review. Larger practices with more patients and/or more GPs were less likely to engage than practices with fewer patients or fewer GPs (P = 0.009 and P = 0.03 respectively) (Table 2). Other practice factors did not significantly influence engagement, including ethnic make-up of the practices’ patients, practice high-needs status, involvement in teaching, use of electronic repeat prescriptions or tagging diagnosis to prescriptions when adjusted for practice (P = 0.13, 0.60, 0.47, 0.63 and 0.95 respectively) (Table 2).

Table 1.  SPACE intervention delivery and GP tick-box selection: practice size, number of GPs receiving SPACE intervention, number of participants with high-risk prescribing (HRP) at review, and GP tick-box selections indicating intended action.^3,11

Table 2.  Associations of practice characteristics with intervention uptake and response to intervention.

The study pharmacist’s journal data confirm the difficulty securing a meeting with GPs, especially in larger practices, and sometimes GPs did not attend arranged meetings citing more pressing demands and being ‘too busy’ (Supplementary Table S2). In one practice, the trial pharmacist made multiple attempts over a 6-month period. The trial pharmacist was not known to practices, which presented a barrier to engagement.

The 64 GPs who did engage indicated in the tick-box their intended action for 416 participants, including ‘Letter’ 97 (23%), ‘No letter but possible change and/or discussion with patient when next in’ 151 (36%), and ‘No action’ 168 (40%) (Table 1). The most common reason for GPs selecting ‘No action’ was because the high-risk prescribing had already ceased; for example, short-course non-steroidal anti-inflammatory, although sometimes the GP did not agree that the prescribing was high-risk or believed it was patient preference. The GPs from practices that were college (Royal New Zealand College of General Practitioners) accredited were more likely to send letters, whereas GPs from practices with a greater proportion of older patients or Māori/Pasifika patients were less likely to send letters (Table 2), although the latter association was not significant when practices that did not engage were excluded (Table 3).

Table 3.  Associations of practice characteristics with intervention uptake and response to intervention, excluding practices that did not engage at all.

The qualitative data reveal that the SPACE intervention was positively viewed by GPs who did engage, and that GP tick-box selections were based on knowledge of their patients. GPs sometimes opted not to send a letter for fear of confusing or upsetting the patient; for example, if a patient had dementia or English was their second language (Table 1).

The SPACE intervention prompted medication review and shared decision-making for some patients, but any effect on high-risk prescribing was short-lived. This process evaluation used mixed methods to help understand implementation of the SPACE intervention in general practice and the limited effect of the intervention, adding to the knowledge gained from the pilot study and qualitative interviews.^3–5

Recruitment into the trial was challenging. Time pressures and poor remuneration remain major barriers to GPs participating in research, but it was also bad timing that a non-trial safer prescribing initiative targeting the same prescribing topic was introduced at the same time. Recruitment of patients through an automated search of practice records was successful, and facilitated by not needing patient consent for use of anonymised patient data.

It was challenging for the trial pharmacist to engage GPs in some of the enrolled larger practices where there was no pre-existing relationship and the practice manager had provided consent to participation rather than individual GPs. For such an intervention to work, the majority of GPs need to support the intervention. Time pressure and competing demands were the main reasons for GPs not engaging with the intervention. We found that GP engagement with the pharmacist-delivered intervention was higher in the pilot study where the study pharmacist had a pre-existing relationship with practices and GPs.³ For pharmacist-supported safer prescribing intervention benefit to be sustained, ongoing relationships between pharmacists and practices may help.⁹

The opportunity for GPs to select an action for each patient is an important step in the behaviour change process. However, GP uptake of the patient letter was less than intended (23%). Some GPs were concerned that the letter might confuse or upset patients, despite earlier work showing that patients responded well to such a letter from their GP.¹⁰ The GPs elected ‘No letter but possible change and/or discussion with patient when next in’ for another 36% patients. As a formal alert or reminder was not part of the intervention, GPs often left themselves a written reminder to follow up. The SPACE intervention may be improved by including a GP-initiated alert in the tick-box to serve as a reminder for GPs. The most common reason for GPs selecting ‘no action’ was that the high-risk prescribing had already stopped. Refining the search to exclude short-course medications would be an improvement, similarly running the search in real-time to avoid the information being ‘out of date’ at the time of review by GP and pharmacist.

The trial was not powered to address whether inequities could be addressed by this intervention. However, the finding that in practices with higher proportion of Māori/Pasifika patients, GPs were less likely to select ‘Letter’, was discouraging. Conversely, the findings that GPs in accredited practices were more likely to send out letters, and GPs in teaching practices were more likely to engage with the intervention is encouraging (Table 2).

A strength of this study is the analysis of the tick-box data providing insight into GPs’ responses to the feedback. A limitation is that our data did not allow follow up and so we do not know whether any change in prescribing followed on from the ‘Letter’ and ‘No letter’ options. The qualitative data provide useful insights, despite the pharmacist’s journal being potentially biased. Qualitative interviews with the GPs would have been preferable. Earlier qualitative work found that GPs thought the patient list and educational sessions were useful, but added to time pressures.¹⁰ It also found that although GPs were concerned about sending out a letter to patients, patients were pleased to receive a letter and to be invited in to see their GP for medication review. Another limitation is the lack of data from GPs who did not engage with the intervention.

Findings are consistent with previous trials in general practice that have encountered difficulties in recruitment and/or poor uptake and engagement with interventions.^11–15 An intervention that included a performance-based payment to each GP per patient was more effective.¹⁶ Remuneration for practice participation in research is important, but it is generally accepted that quality improvement initiatives form part of standard practice.

The SPACE intervention was originally based on aspects of the Australian Veterans’ Medicines Advice and Therapeutics Education Services (MATES) programme.¹⁷ Although MATES involved quarterly targeted patient-specific prescriber feedback to GPs, SPACE was administered only once over a 12-month period. A key aim of MATES was the closer cooperation of GP and pharmacist in patient care and to prompt more pharmacist home medication reviews (HMR), as well as assess the effect of HMRs on adverse events. MATES sent programme materials to 249 454 veterans, 34 527 GPs and around 8000 pharmacies and accredited pharmacists. Pharmacist HMR rates went from 0.6/1000 veterans in 2004 to around 2.2/1000 in 2010 after 21 MATES interventions.¹⁷ Substantial reductions in hospitalisations from heart failure and haemorrhage in those at risk were seen in those who received HMR. It may be that repeated cycles of the SPACE intervention and more involvement of the pharmacist in funded practice- or home-based medication reviews at a patient level may have resulted in more substantial improvements in prescribing safety and patient outcomes. Furthermore, the letter to patients coming directly from the SPACE program rather than from the GP would have increased uptake of this aspect of the intervention.

This process evaluation found that the limited effect of the SPACE intervention could be due to factors including time pressures, lack of existing relationship between trial pharmacist and GPs, the patient list being out-of-date or inaccurate by the time of outreach feedback, and GP fear that a letter might upset a patient. More integration of pharmacists into general practices could support engagement and communication. We found engagement higher in the pilot study where the pharmacist was well-known to the GPs and a regular visitor to practices.^3,18 The practice database search could be run at the time of the pharmacist visit, ensuring the patient list is up-to-date and relevant.

GPs often elected ‘No letter but possible change and/or discussion with patient when next in’. Including a formalised prompt or alert option for this selection might support GPs to follow through on this. Future studies may link GP tick-box selection with prescribing at the patient level, to determine whether selections resulted in change in prescribing.

The data are available on reasonable request to the corresponding author.

The authors declare no conflicts of interest.

This work was supported by the Auckland Medical Research Foundation, New Zealand (1117005).

Supplementary material is available online.