Supporting primary care practitioners to promote dementia risk reduction in Australian general practice: outcomes of a cross-sectional, non-randomised implementation pilot study

Study design

This was a cross-sectional, non-randomised implementation study in five Australian general practices. The Umbrella intervention and implementation strategies were grounded in theory and evidence with high face validity and minimal risk. The study was conducted within existing practices and workflows, allowing concomitant care and interventions. The evaluation focused on implementation outcomes, not clinical treatment outcomes (Proctor et al. 2011), and therefore was not prospectively registered.

Primary and secondary outcomes

The primary outcomes (rightmost in Fig. 1) comprised adoption, fidelity, and penetration of the Umbrella intervention (Damschroder et al. 2022). The three preceding sets of outcomes were secondary outcomes. Each outcome was linked to research questions, as shown in Fig. 1. We examined the perspective of practice managers (PMs), PCPs, and patients regarding outcomes of the Umbrella intervention, while outcomes of the implementation strategies were focused exclusively on the perspectives of PMs and PCPs.

Participants

The Umbrella intervention

The intervention was named for its central premise: DRR is like putting up an umbrella of healthy behaviours to protect your brain. The intervention was based on guidelines for promoting DRR in primary care, barriers and facilitators to promoting DRR in Australian primary care, existing evidence for supporting preventive care, and stakeholder input (Godbee 2024).

The Umbrella intervention involved a waiting room survey, patient information cards, and suggested PCP actions. The waiting room survey asked about behaviours associated with reduced risk of dementia (e.g. smoking, nutrition, alcohol consumption, physical activity). Patients aged 40–64 years completed the survey, and selected a priority area and rated it in terms of importance and confidence. The PCP then reviewed and discussed the patient’s responses, discussed DRR using relevant patient information cards, provided brief motivational interviewing, made a plan with the patient to address personal risk factors, signposted the patient to additional support, and/or arranged follow-up. The behaviours could be performed across multiple appointments as clinically appropriate.

Implementation strategies

Implementation strategies were targeted at barriers to promoting DRR in general practice in order to support PCPs to use the Umbrella intervention. We blended educational strategies and strategies to develop stakeholder inter-relationships; both types of strategies are important and feasible methods to support practice change (Waltz et al. 2015).

The implementation approach incorporated two consecutive 4-week cycles. A single ‘champion’ PCP used the Umbrella intervention in the first cycle (supported by an implementation facilitator); all participating PCPs joined in the second cycle. Each cycle began with an educational outreach visit from the implementation facilitator (KG) in which educational materials were distributed and explained, followed by a consensus discussion involving the implementation facilitator, PM, and PCPs about whether and how to proceed. Throughout implementation, participating PCPs across practices shared their experiences and suggestions in an online peer discussion. The implementation facilitator moderated the discussion and provided information and resources as requested. PMs, PCPs, and reception staff could access educational materials on a project website. The implementation approach was adaptable based on PCP feedback.

Procedures

The study took place between January and March 2020. Participating sites agreed to use the Umbrella intervention and implementation strategies as described. All study materials are listed in Table 1 and provided in the Supplementary material.

Data on outcomes were gathered through on-site visits, analysis of the content of online peer group discussions, website usage metrics, and telephone and video interviews.

The implementation facilitator visited sites fortnightly to collect outcome data, replenish materials, provide technical and coaching support, and populate a field journal with observations pertinent to study outcomes. Total visits to the project website by PMs, PCPs, and receptionists were counted. To help assess appropriateness of the intervention and implementation strategies, PMs described their practices using the Practice Assessment Tool and staff completed a survey about barriers to promoting DRR that were identified in previous research (Godbee et al. 2019, 2020, 2021; Zheng et al. 2021). Consenting patients were interviewed by telephone in the week following their PCP appointment, and PMs were interviewed by video at the end of the study. These semi-structured interviews sought participants’ perspectives on the acceptability, appropriateness, and feasibility of the Umbrella intervention. The patient interview guide underwent prior review and refinement by a panel of healthcare consumers involved in developing the intervention; the PM interview guide was not reviewed prior to the current pilot study.

Analysis

Actual outcomes (adoption, fidelity, penetration) were quantified using proportions. Descriptive statistics were generated using Microsoft Excel (ver. 2302, Microsoft Corporation, https://office.microsoft.com/excel). Fidelity and penetration proportions were categorised as low (below 50%), moderate (50–80%), or high (above 80%) following precedents in the literature (e.g. Toomey et al. 2017). For adoption, a lower threshold was employed because the Umbrella intervention and implementation strategies were untested innovations. According to Diffusion of Innovation Theory (Rogers 2003), innovations are adopted in the first instance by 2.5% of individuals in a system. These so-called ‘innovators’ are typically venturesome pioneers who are interested in new ideas, willing to take risks, and able to cope with a high degree of uncertainty about an innovation at the time it is adopted (Rogers 2003). Therefore, we expected up to 2.5% of eligible practices to adopt the intervention and implementation strategies in the pilot study. Adoption was classified as low (below 50% of 2.5%), moderate (50–80% of 2.5%), or high (above 80% of 2.5%).

Antecedent assessments (acceptability, appropriateness, feasibility) drew on qualitative data analysed using a directed content analysis method (Hsieh and Shannon 2005). We developed a codebook from the Implementation Outcomes Framework (Proctor et al. 2011) and the research questions (Fig. 1). Field notes, discussion board posts, and transcripts of all group meetings and interviews were imported into Dedoose computer-assisted qualitative data analysis software (ver. 9.0.107, SocioCultural Research Consultants, https://www.dedoose.com/).

Data were coded by the first author according to the outcome, implementation component (intervention or implementation strategies), and stakeholder perspective (general practice staff or patients). Potential codes derived from the data were discussed with coauthors for interpretation and refinement. Examples were added to the codebook to enable coauthor review and ensure consistent application of codes over time. Reflexivity was maintained throughout the coding process by continually reflecting on the researchers’ assumptions, biases, and potential influences on analysis. The analysis was displayed in a matrix, with outcomes from each perspective plotted against implementation components. Coauthors examined the matrix and categorised outcomes as low, moderate, high, or mixed, based on consensus discussions and consideration of the relevant literature.

Ethics approval

The study was approved by the University of Melbourne Human Ethics Advisory Group (Ethics ID 1955339). General practice staff provided written informed consent. Opt-out consent was employed for patients completing waiting room surveys. Interviewed patients provided audio-recorded verbal consent.

Recruitment

Staff from 17 eligible practices expressed interest in participating in the study. PMs from eight practices met with the implementation facilitator and five sites consented to take part. Reasons that sites declined to take part included being too busy and lack of interest. Three of the five participating practices were small (fewer than 2500 active patients). Together, participating sites comprised 66 staff (four PMs, 35 PCPs, and 27 reception staff), and approximately 4563 patients aged between 40 and 64 years. Characteristics of participating sites, staff, and patients are described in the Supplementary material.

Actual outcomes

Sixteen PCPs (16/35, 48%) attended the initial outreach visit and consensus discussion. One PCP declined further participation and three additional PCPs agreed to deliver the intervention without attending the initial outreach visit. In total, 18 PCPs (18/35, 51%) delivered the intervention to 159 patients (159/4563, 3%) across five sites.

The calculation of actual outcomes and the rationale for the ratings are summarised in Table 2. For the Umbrella intervention, there was moderate adoption with moderate to high fidelity, but penetration was low. For the implementation strategies, adoption and penetration were moderate, but fidelity was mixed. Additional evidence supporting the ratings is provided in the Supplementary material.

Antecedent assessments

Ratings and rationales for the antecedent assessments are summarised in Table 3. The Umbrella intervention was moderately to highly acceptable, highly appropriate, and moderately feasible. Of the patients interviewed, 14/16 patients (88%) thought the Umbrella intervention should be offered to all patients aged 40–64 in general practice. Of the two dissenting patients, one thought ‘in principle, it would be fantastic’ but it needed funding; the other did not recall dementia being discussed during their recent appointment.

Antecedent assessments for the implementation strategies were mixed. Most implementation strategies were acceptable, apart from the online peer discussion. Initially, PCPs were excited at the prospect of sharing and learning from other general practices in an online discussion they could participate in flexibly. However, from a feasibility perspective, many PCPs either forgot to contribute, ran out of time, or had limited computer skills. PCPs suggested sharing weekly (instead of twice a week) and using a messaging app on their smartphones (e.g. WhatsApp) instead of a web-based password-protected portal. In terms of appropriateness, the implementation strategies were suitable for PCPs but not reception staff. Few reception staff read the information provided; they learned about the intervention verbally from their PMs or colleagues or from the implementation facilitator during site visits. On the request of PMs and GP champions, a one-page instruction sheet for reception staff was disseminated (see the Supplementary material). Field notes indicated at least some staff found the instructions useful and the supporting website was visited 101 times. Nevertheless, at the end of the study, at least one PM still thought the reception staff needed more guidance. ‘He didn’t really know what to say, even though I told him 20 million times, so he tended not to do it’ (PM, Practice A). Data from surveys and interviews that support these findings are provided in the Supplementary material.

Possible influences on outcomes

The pilot study was not powered to evaluate differences in implementation outcomes between patients, staff, and study sites. Nevertheless, evaluation data suggested that characteristics of patients, staff, and the general practice setting might have influenced outcomes. These characteristics are summarised in Table 4. For some characteristics (e.g. being new to the practice; having a family history of dementia), participants were divided on whether the characteristic was an important influence on outcomes.

Possible associations between outcomes

We expected that antecedent assessments would influence actual outcomes. The study design did not allow for definitive assessment of associations between outcomes; however, qualitative data were consistent with our expectations. For example, it was not always feasible for reception staff to distribute waiting room surveys, contributing to low penetration of the Umbrella intervention among patients. Similarly, joining the online peer discussion was not feasible for some PCPs which reduced fidelity to this implementation strategy.

Also as expected, the actual outcomes of the implementation strategies seemed to influence antecedent assessments for the Umbrella intervention. For example, educational meetings and consensus discussions were conducted as intended, helping staff who attended to conclude that the Umbrella intervention was broadly acceptable, appropriate, and feasible. Nearly all PCPs who engaged with the implementation strategies (16/17, 94%) agreed to use the Umbrella intervention.

Interestingly, we also found evidence suggesting that actual incomes could influence antecedent assessments. For example, the low penetration of the intervention among patients meant it was feasible for PCPs to use the Umbrella intervention alongside their existing workload. ‘It would be overwhelming if it was more than 5 each a day’ (PM, Practice E). Similarly, poor fidelity to the online peer discussion made this implementation strategy less acceptable as time went on.

Summary

The Umbrella intervention was a new approach to supporting Australian PCPs to promote DRR. It was complemented by educational and relational strategies intended to mitigate barriers to promoting DRR. Piloting of the intervention and implementation strategies in five Australian general practices produced mixed findings. Participants thought the Umbrella intervention was an acceptable, appropriate, and feasible means of supporting practice change. Approximately half of the eligible PCPs within participating practices promoted DRR using the Umbrella intervention at least once, with moderate fidelity. Similarly, approximately half of eligible PCPs engaged with at least one implementation strategy. PCPs found most implementation strategies acceptable, appropriate, and feasible, although they wanted a simpler platform for online peer discussion and strategies to better prepare reception staff for the change in practice.

Implications

Of the 12 outcomes assessed in the pilot study, penetration of the Umbrella intervention was the only reliably low outcome. Penetration was low despite positive antecedent assessments, which seem necessary but not sufficient for intervention uptake (Sekhon et al. 2017). A more robust online peer discussion may have yielded suggestions and momentum to improve penetration. Low penetration of the intervention may have been partly attributable to low attendance at the initial outreach visit, which in turn may have been because lunchtime in-person meetings were not feasible for many PCPs and reception staff. However, even allowing for low penetration among PCPs, the Umbrella intervention reached very few patients. For comparison, a similar waiting room survey (eCHAT) was completed by 211 patients during a 2- to 3-week period in two general practices in New Zealand (Goodyear-Smith et al. 2013). The Umbrella intervention waiting room survey reached fewer patients over a longer period in more practices. There were two critical differences between the studies: a dedicated research assistant invited patients to complete the eCHAT, and it was completed on an iPad. In our study, patients and staff liked the waiting room survey being paper-based, but an electronic survey that bypassed reception staff might reach more patients. It is also possible that eligible patients were missed in our study because they were acutely ill, preoccupied with other health concerns, or did not attend the general practice in the 8-week study period (e.g. postponing non-essential health care due to COVID-19 concerns). Future research should examine these hypotheses about the low penetration of the Umbrella intervention and consider strategies to improve it.

Our study produced similar findings to a pilot evaluation of the dementia awareness component in the National Health Service (NHS) Health Check when it was introduced for adults younger than 65 years (Solutions Strategy Research Facilitation Ltd and Cornish and Grey Ltd 2017). In both evaluations, patients and PCPs liked the idea of promoting DRR, thought it complemented messages about risk reduction for other non-communicable diseases, and found written information helpful. As in the NHS Health Check evaluation, the intervention was seen as manageable, straightforward, and reasonably memorable; it was recalled by 79% of patients in the NHS Health Check evaluation and 69% of patients in the current study. In both cases, PCPs sometimes judged whether it was appropriate to promote DRR based on individual patient characteristics (NHS Health Check 2017). Interestingly, PCPs in the NHS Health Check evaluation lacked confidence in raising the topic of dementia and wanted further training, prompts, and resources. In the current study, PCPs reported that many of the implementation strategies (e.g. information cards, site visits, taking an incremental approach) helped them deliver the intervention.

Strengths and limitations

The pilot study considered 12 implementation outcomes for a new intervention and implementation strategies. The in-depth focus on implementation outcomes was a departure from the efficacy perspective, in which randomised controlled trials are the gold standard but can be problematic in DRR research because they need a large sample and a long follow-up. While not contributing data on the efficacy of promoting DRR, the pilot study generated new knowledge about implementation in the primary care setting that could reduce research waste and decision-maker uncertainty.

It is likely that staff in some eligible practices did not read the recruitment notice in the South East Melbourne PHN newsletter or attend network meetings marketing the pilot study and were therefore unaware of the Umbrella intervention. An alternative recruitment strategy may have led to higher adoption.

The mixed-methods approach to data collection triangulated outcome data from multiple sources without undue burden on participants. Confidential, individual interviews with practice champions and PMs complemented consensus-focused group meetings. However, the same researcher facilitated implementation and collected outcome data, and telephone interviews may have deterred participants not comfortable communicating in English.

This study predated the COVID-19 pandemic, which led to the expansion and increased uptake of telehealth services in general practice (MBS Review Advisory Committee 2024). It is unclear whether the Umbrella intervention (which hinges on patients completing a paper-based survey and receiving physical information cards) is appropriate in this changed context and, if not, how it should be modified.

Future directions

The Umbrella intervention and implementation strategies should be refined based on findings from the pilot study and further stakeholder input. These innovations show promise but there is still work needed to understand and improve adoption and penetration. Additional funding may be needed to support marketing of the innovations, reimbursement of practice staff, and integration of the innovations with existing practice software.

While the pilot study may have successfully reached innovative practices, there remains a question of how to engage the remaining 98% of practices. It is possible that staff in these practices (if they were aware of the study) viewed the Umbrella intervention and implementation strategies as less acceptable, appropriate, and feasible. Patients in these practices could potentially be from more diverse backgrounds and settings and therefore the intervention and implementation strategies may require adaptation. Stakeholders from these practices should be involved in adapting and refining the intervention and implementation strategies. Future research could also evaluate additional outcomes such as cost, sustainment, and patient health outcomes. Showing that the Umbrella intervention benefits patients and is cost-effective could build support for wider use.