Register      Login
Journal of Primary Health Care Journal of Primary Health Care Society
Journal of The Royal New Zealand College of General Practitioners
RESEARCH ARTICLE (Open Access)

Process evaluation of the Safer Prescribing and Care for the Elderly (SPACE) cluster randomised controlled trial in New Zealand general practice

Katharine Ann Wallis https://orcid.org/0000-0002-2580-9362 1 2 * , Carolyn Raina Elley 1 , Joanna Frances Hikaka https://orcid.org/0000-0001-6792-6607 1 , Simon A. Moyes https://orcid.org/0000-0001-9580-1545 1
+ Author Affiliations
- Author Affiliations

1 Department of General Practice and Primary Health Care, University of Auckland, New Zealand.

2 General Practice Clinical Unit, Medical School, University of Queensland, UQ Health Sciences Building, Brisbane, Qld 4029, Australia.

* Correspondence to: k.wallis@uq.edu.au

Handling Editor: Felicity Goodyear-Smith

Journal of Primary Health Care 14(3) 244-253 https://doi.org/10.1071/HC22052
Published: 4 August 2022

© 2022 The Author(s) (or their employer(s)). Published by CSIRO Publishing on behalf of The Royal New Zealand College of General Practitioners. This is an open access article distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND)

Abstract

Introduction: The Safer Prescribing and Care for the Elderly (SPACE) cluster randomised controlled trial in 39 general practices found that a search of the practice database to identify and generate for each general practitioner (GP) a list of patients with high-risk prescribing, pharmacist-delivered one-on-one feedback to GPs, and electronic tick-box for GPs to select action for each patient (Patient letter; No letter but possible medication review when patient next in; No action), prompted safer prescribing at 6 months but not at 1 year.

Aim: This process evaluation explores research participation, intervention uptake and effect on GPs.

Methods: Mixed methods were used including quantitative data (log of practice recruitment, demographic data, intervention delivery and GP responses including tick-box selections) and qualitative data (trial pharmacist reflective journal). Data were analysed using descriptive statistics and general inductive analysis, respectively.

Results: Recruitment of general practices was challenging, with only 39% of eligible practices agreeing to participate. Those who declined were often ‘too busy’. Engagement was also challenging, especially in larger practices, with the trial pharmacist managing to meet with only 64% of GPs in the intervention group. The GPs who did engage were positive about the intervention, but elected to send letters to only 23% of patients with high-risk prescribing, either because the high-risk prescribing had already stopped, the GP did not agree the prescribing was ‘high-risk’ or the GP was concerned a letter would upset the patient.

Conclusions: Effectiveness of the SPACE cluster randomised controlled trial could be improved by changes including ensuring searches are current and relevant, repeating cycles of search and feedback, and integrating pharmacists into general practices.

Keywords: aged care, prescription medicines, primary health care, quality and safety, randomised trials.

WHAT GAP THIS FILLS
What is already known: The SPACE intervention comprising general practice electronic database searches to identify patients with high-risk prescribing, pharmacist-delivered education and feedback to GPs, and letters from a GP to invite patients to a medication review, had only a partial short-term effect on prescribing safety.
What this study adds: This process evaluation found that barriers to GP uptake of the SPACE intervention included time pressures, absence of existing relationship with pharmacists, and reluctance to send automated letter to patients. To improve uptake and effectiveness of the SPACE intervention, integrating pharmacists into practices, repeating the search, remunerating GPs for participation and repeated cycles may be necessary.



Introduction

Efforts are ongoing to identify interventions that support safer medication prescribing in general practice to minimise adverse drug events (ADEs) and related hospitalisations.1,2 The Safer Prescribing and Care for the Elderly (SPACE) cluster randomised controlled trial was conducted in New Zealand general practice from 2018 to 2020 to investigate the effect of the SPACE intervention on high-risk prescribing of non-steroidal anti-inflammatory and/or antiplatelet medicines and related hospitalisations.3 High-risk prescribing places patients at increased risk of ADEs. SPACE is a complex intervention involving: (i) an automated search of practice records to identify and generate for each general practitioner (GP) a list of patients with high-risk prescribing for the prescribing topic; outreach visit from pharmacist to deliver: (ii) group education to GPs (30–60 min) and (iii) one-on-one feedback with each GP (15–30 min); (iv) tick-box for GPs to indicate an intended action for each patient in their list (Letter; No letter but possible change and/or discussion with patient when next in; No action); and (v) letter from a GP to selected patients inviting them to discuss their medicines when they are next in (Supplementary Table S1).3 All GPs who attended the one-on-one session with the study pharmacist received a NZ$100 gift voucher. The trial found that SPACE decreased high-risk prescribing for gastrointestinal ADEs at 6 months, primarily by increasing proton-pump inhibitor protection, but did not decrease high-risk prescribing for ardiovascular or renal ADEs. This partial effect was not sustained at 12 months and there was no difference in ADE-related hospitalisations. The detailed methods and results have been published previously.35

Process evaluations are recommended to understand how and why an intervention has or has not worked, and to help understand variation in responses and discrepancies between expected and observed effect.6,7 Process evaluations can help to understand whether an intervention was implemented and delivered in the way that was intended and reasons for variation, and can outline contextual influences on effectiveness to inform future intervention development and implementation.6,7 Reported here is the SPACE process evaluation describing recruitment of practices and GPs, intervention delivery and uptake, and GP responses to the intervention.


Methods

The process evaluation was planned prospectively using a mixed methods approach including quantitative and qualitative data. The evaluation is reported using the framework developed by Grant et al.6 The SPACE cluster randomised controlled trial was registered with the Australasian Clinical Trials Register (ACTRN12618000034235, January 2018), and approved by the University of Auckland Human Participants Ethics Committee (Ref. UAHPEC 020092).

Study population

The SPACE trial was conducted in 39 general practices with 21 867 participants identified as at increased risk of experiencing gastrointestinal, renal or cardiac ADEs from non-steroidal anti-inflammatory or anti-platelet medications at baseline, of whom 1479 (6.8%) had received high-risk prescribing.3 Of the 39 practices, 20 were allocated to the intervention group, including 100 GPs and 613 participants with high-risk prescribing at baseline. All practices had electronic medical records and used practice management systems compatible with the trial system for a remote electronic search of the practice database.

Data collection

Quantitative data were collected from a number of sources including: (i) publicly available data on the number of GPs working at practices; (ii) log of practice recruitment; (iii) practice demographic data on forms completed by recruited practices; and (iv) log of intervention delivery maintained by study pharmacist that included a number of GPs attending the group education, number of GPs attending the one-on-one sessions to review their list of patients with high-risk prescribing, and GP tick-box selection for each patient. Practice variables collected and included in the regression analyses were: practice list size, number of GPs, proportion of patients aged ≥65 years, proportion of patients of Māori or Pasifika ethnicity, whether the practice was a ‘high needs’ practice, practice accreditation status, whether the practice taught medical students, or whether the practice allowed online electronic ordering of repeat prescriptions, had a system of medications reconciliation or tagged a diagnosis to prescriptions. A ‘high needs’ practice was defined as having at least 50% of enrolled patients from the lowest socio-economic quintile or Māori or Pasifika ethnicity. ‘High needs’ practices are funded differently, requiring the practice to charge lower patient fees.8 Qualitative data were collected by the trial pharmacist in an electronic reflective journal summarising GP comments and responses, and pharmacists’ perceptions of interactions and descriptions of intervention delivery.

Analyses

Quantitative analyses used descriptive statistics to compare characteristics of recruited practices with those of eligible practices that declined, to ascertain generalisability of results. Univariate and multivariate logistic regression models,including variables listed above, were used to examine associations between practices and variation in intervention uptake and GP responses. To assess whether practice characteristics were associated with level of engagement and responses, a sensitivity analysis was conducted, excluding intervention practices that did not engage at all. General inductive analysis was used to categorise qualitative data.


Results

Recruitment of practices

A search of GP databases identified 220 general practices in the study region. Of these, less than half were eligible for the trial (101/220) due to earlier participation in the SPACE pilot, involvement in a contemporaneous non-trial safer prescribing initiative, or using practice management software not compatible with the study clinical outcomes data extraction process. Thirty-nine practices (39%) agreed to participate. Larger practices with more GPs tended to be more likely to participate (mean (s.d.) number of GPs 5.1 (3.4) vs 3.8 (3.1), P = 0.08). The most common reason for practices declining was ‘too busy’.

Intervention delivery

Twenty practices were randomised to receive the intervention, including 100 GPs and 613 patients identified as having high-risk prescribing at baseline (Table 1). Of the 100 GPs, only 64 (64%) attended the one-on-one session with the study pharmacist, reviewing only 416 of the 613 (68%) patients (Table 1). In two large practices, no GP engaged with the intervention, and in a third practice, only one GP engaged, resulting in 197 (32%) patients receiving no review. Larger practices with more patients and/or more GPs were less likely to engage than practices with fewer patients or fewer GPs (P = 0.009 and P = 0.03 respectively) (Table 2). Other practice factors did not significantly influence engagement, including ethnic make-up of the practices’ patients, practice high-needs status, involvement in teaching, use of electronic repeat prescriptions or tagging diagnosis to prescriptions when adjusted for practice (P = 0.13, 0.60, 0.47, 0.63 and 0.95 respectively) (Table 2).


Table 1.  SPACE intervention delivery and GP tick-box selection: practice size, number of GPs receiving SPACE intervention, number of participants with high-risk prescribing (HRP) at review, and GP tick-box selections indicating intended action.3,11
Click to zoom


Table 2.  Associations of practice characteristics with intervention uptake and response to intervention.
Click to zoom

The study pharmacist’s journal data confirm the difficulty securing a meeting with GPs, especially in larger practices, and sometimes GPs did not attend arranged meetings citing more pressing demands and being ‘too busy’ (Supplementary Table S2). In one practice, the trial pharmacist made multiple attempts over a 6-month period. The trial pharmacist was not known to practices, which presented a barrier to engagement.

Response to intervention

The 64 GPs who did engage indicated in the tick-box their intended action for 416 participants, including ‘Letter’ 97 (23%), ‘No letter but possible change and/or discussion with patient when next in’ 151 (36%), and ‘No action’ 168 (40%) (Table 1). The most common reason for GPs selecting ‘No action’ was because the high-risk prescribing had already ceased; for example, short-course non-steroidal anti-inflammatory, although sometimes the GP did not agree that the prescribing was high-risk or believed it was patient preference. The GPs from practices that were college (Royal New Zealand College of General Practitioners) accredited were more likely to send letters, whereas GPs from practices with a greater proportion of older patients or Māori/Pasifika patients were less likely to send letters (Table 2), although the latter association was not significant when practices that did not engage were excluded (Table 3).


Table 3.  Associations of practice characteristics with intervention uptake and response to intervention, excluding practices that did not engage at all.
Click to zoom

The qualitative data reveal that the SPACE intervention was positively viewed by GPs who did engage, and that GP tick-box selections were based on knowledge of their patients. GPs sometimes opted not to send a letter for fear of confusing or upsetting the patient; for example, if a patient had dementia or English was their second language (Table 1).


Discussion

The SPACE intervention prompted medication review and shared decision-making for some patients, but any effect on high-risk prescribing was short-lived. This process evaluation used mixed methods to help understand implementation of the SPACE intervention in general practice and the limited effect of the intervention, adding to the knowledge gained from the pilot study and qualitative interviews.35

Recruitment into the trial was challenging. Time pressures and poor remuneration remain major barriers to GPs participating in research, but it was also bad timing that a non-trial safer prescribing initiative targeting the same prescribing topic was introduced at the same time. Recruitment of patients through an automated search of practice records was successful, and facilitated by not needing patient consent for use of anonymised patient data.

It was challenging for the trial pharmacist to engage GPs in some of the enrolled larger practices where there was no pre-existing relationship and the practice manager had provided consent to participation rather than individual GPs. For such an intervention to work, the majority of GPs need to support the intervention. Time pressure and competing demands were the main reasons for GPs not engaging with the intervention. We found that GP engagement with the pharmacist-delivered intervention was higher in the pilot study where the study pharmacist had a pre-existing relationship with practices and GPs.3 For pharmacist-supported safer prescribing intervention benefit to be sustained, ongoing relationships between pharmacists and practices may help.9

The opportunity for GPs to select an action for each patient is an important step in the behaviour change process. However, GP uptake of the patient letter was less than intended (23%). Some GPs were concerned that the letter might confuse or upset patients, despite earlier work showing that patients responded well to such a letter from their GP.10 The GPs elected ‘No letter but possible change and/or discussion with patient when next in’ for another 36% patients. As a formal alert or reminder was not part of the intervention, GPs often left themselves a written reminder to follow up. The SPACE intervention may be improved by including a GP-initiated alert in the tick-box to serve as a reminder for GPs. The most common reason for GPs selecting ‘no action’ was that the high-risk prescribing had already stopped. Refining the search to exclude short-course medications would be an improvement, similarly running the search in real-time to avoid the information being ‘out of date’ at the time of review by GP and pharmacist.

The trial was not powered to address whether inequities could be addressed by this intervention. However, the finding that in practices with higher proportion of Māori/Pasifika patients, GPs were less likely to select ‘Letter’, was discouraging. Conversely, the findings that GPs in accredited practices were more likely to send out letters, and GPs in teaching practices were more likely to engage with the intervention is encouraging (Table 2).

Strengths and limitations

A strength of this study is the analysis of the tick-box data providing insight into GPs’ responses to the feedback. A limitation is that our data did not allow follow up and so we do not know whether any change in prescribing followed on from the ‘Letter’ and ‘No letter’ options. The qualitative data provide useful insights, despite the pharmacist’s journal being potentially biased. Qualitative interviews with the GPs would have been preferable. Earlier qualitative work found that GPs thought the patient list and educational sessions were useful, but added to time pressures.10 It also found that although GPs were concerned about sending out a letter to patients, patients were pleased to receive a letter and to be invited in to see their GP for medication review. Another limitation is the lack of data from GPs who did not engage with the intervention.

International comparisons

Findings are consistent with previous trials in general practice that have encountered difficulties in recruitment and/or poor uptake and engagement with interventions.1115 An intervention that included a performance-based payment to each GP per patient was more effective.16 Remuneration for practice participation in research is important, but it is generally accepted that quality improvement initiatives form part of standard practice.

The SPACE intervention was originally based on aspects of the Australian Veterans’ Medicines Advice and Therapeutics Education Services (MATES) programme.17 Although MATES involved quarterly targeted patient-specific prescriber feedback to GPs, SPACE was administered only once over a 12-month period. A key aim of MATES was the closer cooperation of GP and pharmacist in patient care and to prompt more pharmacist home medication reviews (HMR), as well as assess the effect of HMRs on adverse events. MATES sent programme materials to 249 454 veterans, 34 527 GPs and around 8000 pharmacies and accredited pharmacists. Pharmacist HMR rates went from 0.6/1000 veterans in 2004 to around 2.2/1000 in 2010 after 21 MATES interventions.17 Substantial reductions in hospitalisations from heart failure and haemorrhage in those at risk were seen in those who received HMR. It may be that repeated cycles of the SPACE intervention and more involvement of the pharmacist in funded practice- or home-based medication reviews at a patient level may have resulted in more substantial improvements in prescribing safety and patient outcomes. Furthermore, the letter to patients coming directly from the SPACE program rather than from the GP would have increased uptake of this aspect of the intervention.


Conclusions

This process evaluation found that the limited effect of the SPACE intervention could be due to factors including time pressures, lack of existing relationship between trial pharmacist and GPs, the patient list being out-of-date or inaccurate by the time of outreach feedback, and GP fear that a letter might upset a patient. More integration of pharmacists into general practices could support engagement and communication. We found engagement higher in the pilot study where the pharmacist was well-known to the GPs and a regular visitor to practices.3,18 The practice database search could be run at the time of the pharmacist visit, ensuring the patient list is up-to-date and relevant.

GPs often elected ‘No letter but possible change and/or discussion with patient when next in’. Including a formalised prompt or alert option for this selection might support GPs to follow through on this. Future studies may link GP tick-box selection with prescribing at the patient level, to determine whether selections resulted in change in prescribing.


Data availability

The data are available on reasonable request to the corresponding author.


Conflicts of interest

The authors declare no conflicts of interest.


Declaration of funding

This work was supported by the Auckland Medical Research Foundation, New Zealand (1117005).


Supplementary material

Supplementary material is available online.



Acknowledgements

We would like to acknowledge all participating general practices, general practitioners and staff.


References

[1]  Tecklenborg S, Byrne C, Cahir C, et al. Interventions to reduce adverse drug event-related outcomes in older adults: a systematic review and meta-analysis. Drugs Aging 2020; 37 91–8.
Interventions to reduce adverse drug event-related outcomes in older adults: a systematic review and meta-analysis.Crossref | GoogleScholarGoogle Scholar |

[2]  Anderson LJ, Schnipper JL, Nuckols TK, et al. A systematic overview of systematic reviews evaluating interventions addressing polypharmacy. Am J Health Syst Pharm 2019; 76 1777–87.
A systematic overview of systematic reviews evaluating interventions addressing polypharmacy.Crossref | GoogleScholarGoogle Scholar |

[3]  Wallis KA, Elley CR, Moyes S, et al. Safer Prescribing and Care for the Elderly (SPACE): a cluster randomised controlled trial in general practice. BJGP Open 2021; 6 BJGPO.2021.0129
Safer Prescribing and Care for the Elderly (SPACE): a cluster randomised controlled trial in general practice.Crossref | GoogleScholarGoogle Scholar |

[4]  Wallis KA, Elley CR, Moyes S, et al. Safer Prescribing and Care for the Elderly (SPACE): a pilot study in general practice. BJGP Open 2018; 2 bjgpopen18X101594
Safer Prescribing and Care for the Elderly (SPACE): a pilot study in general practice.Crossref | GoogleScholarGoogle Scholar |

[5]  Wallis KA, Elley CR, Lee A, et al. Safer Prescribing and Care for the Elderly (SPACE): protocol of a cluster randomized controlled trial in primary care. JMIR Res Protoc 2018; 7 e109
Safer Prescribing and Care for the Elderly (SPACE): protocol of a cluster randomized controlled trial in primary care.Crossref | GoogleScholarGoogle Scholar |

[6]  Grant A, Treweek S, Dreischulte T, et al. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials 2013; 14 15
Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting.Crossref | GoogleScholarGoogle Scholar |

[7]  UK Medical Research Council. Developing and Evaluating Complex Interventions: New Guidance. London: UK Medical Research Council; 2008.

[8]  New Zealand Ministry of Health - Manatu Hauora. Very Low Cost Access scheme. Wellington: New Zealand Government; 2006. Available at https://www.health.govt.nz/our-work/primary-health-care/primary-health-care-subsidies-and-services/very-low-cost-access-scheme [Accessed 21 September 21]

[9]  Haua R, Harrison J, Aspden T. Pharmacist integration into general practice in New Zealand. J Prim Health Care 2019; 11 159–69.
Pharmacist integration into general practice in New Zealand.Crossref | GoogleScholarGoogle Scholar |

[10]  Wallis K, Tuckey R. Safer Prescribing and Care for the Elderly (SPACE): feasibility of audit and feedback plus practice mail-out to patients with high-risk prescribing. J Prim Health Care 2017; 9 145–52.
Safer Prescribing and Care for the Elderly (SPACE): feasibility of audit and feedback plus practice mail-out to patients with high-risk prescribing.Crossref | GoogleScholarGoogle Scholar |

[11]  Clyne B, Cooper JA, Hughes CM, et al. A process evaluation of a cluster randomised trial to reduce potentially inappropriate prescribing in older people in primary care (OPTI-SCRIPT study). Trials 2016; 17 386
A process evaluation of a cluster randomised trial to reduce potentially inappropriate prescribing in older people in primary care (OPTI-SCRIPT study).Crossref | GoogleScholarGoogle Scholar |

[12]  Dreischulte T, Grant A, Hapca A, et al. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: quantitative examination of variation between practices in recruitment, implementation and effectiveness. BMJ Open 2018; 8 e017133
Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: quantitative examination of variation between practices in recruitment, implementation and effectiveness.Crossref | GoogleScholarGoogle Scholar |

[13]  McDermott L, Yardley L, Little P, et al. Process evaluation of a point-of-care cluster randomised trial using a computer-delivered intervention to reduce antibiotic prescribing in primary care. BMC Health Serv Res 2014; 14 594
Process evaluation of a point-of-care cluster randomised trial using a computer-delivered intervention to reduce antibiotic prescribing in primary care.Crossref | GoogleScholarGoogle Scholar |

[14]  Dyas JV, Togher F, Siriwardena AN. Intervention fidelity in primary care complex intervention trials: qualitative study using telephone interviews of patients and practitioners. Qual Prim Care 2014; 22 25–34.

[15]  Page MJ, French SD, McKenzie JE, et al. Recruitment difficulties in a primary care cluster randomised trial: investigating factors contributing to general practitioners’ recruitment of patients. BMC Med Res Methodol 2011; 11 35
Recruitment difficulties in a primary care cluster randomised trial: investigating factors contributing to general practitioners’ recruitment of patients.Crossref | GoogleScholarGoogle Scholar |

[16]  Dreischulte T, Donnan P, Grant A, et al. Safer prescribing — a trial of education, informatics, and financial incentives. N Engl J Med 2016; 374 1053–64.
Safer prescribing — a trial of education, informatics, and financial incentives.Crossref | GoogleScholarGoogle Scholar |

[17]  Bell JS, Kalisch LM, Ramsay EN, et al. Prescriber feedback to improve quality use of medicines among older people: the Veterans’ MATES Program. J Pharm Pract Res 2011; 41 316–9.
Prescriber feedback to improve quality use of medicines among older people: the Veterans’ MATES Program.Crossref | GoogleScholarGoogle Scholar |

[18]  Wallis KA, Elley CR, Moyes S, et al. Safer Prescribing and Care for the Elderly (SPACE): a pilot study in general practice. BJGP Open 2018; 2 bjgpopen18X101594
Safer Prescribing and Care for the Elderly (SPACE): a pilot study in general practice.Crossref | GoogleScholarGoogle Scholar |