A mixed-methods evaluation of an intervention for enhancing alcohol screening in adults aged 50+ attending primary health care

Study design

This was a mixed-method, longitudinal, multi-site study detailing the process of planning, implementing, and evaluating a PHC-based practice intervention, a method useful for targeting a ‘real world’ issue with existing health service delivery.²⁷ This project was funded by the Te Whatu Ora Health New Zealand and a published report for this organisation provides significantly greater detail of our study design, intervention development, data collection, and analysis (see Newcombe et al.²⁸). In brief, our cross-disciplinary research team was drawn from experts in local PHC practice and alcohol screening (JM; who was also our ‘clinical champion’), screening and assessment of older adult alcohol use issues (AM), pharmacology (JS), case study methodology (GW), and alcohol and drug epidemiology (AT, DN, JR). We collectively designed and implemented this intervention across 11 PHC practices (ie general practice organisations) in the Whanganui region of New Zealand, an area with significant comorbidities²⁹ and alcohol consumption³⁰ relevant to the current intervention. Furthermore, we engaged in significant consultation with Māori health leaders and iwi (community groups) to discuss and understand appropriate approaches for the design of this intervention, how our team should engage with and acknowledge Māori, as well as identifying appropriate training approaches for health professionals that build on and facilitate such approaches. Ethics approval for undertaking this project was obtained from the New Zealand Health and Disability Ethics Committee (HDEC) reference 19/STH/65/AM04. This project included three components: (1) algorithm development and integration, (2) supporting discussions about alcohol with patients, and (3) outcome evaluation.

Algorithm development and integration

The algorithm was based on the Comorbid Alcohol Risk Evaluation Tool (CARET), an expert-derived, evidence-based screening tool for identifying alcohol-related risk in adults aged 50+ presenting to PHC settings.^31,32 The CARET integrates standard alcohol use questions (ie drinking frequency, quantity, and binge episodes) with questions determining the presence of alcohol-harm risk factors for aging adults, namely chronic health conditions (eg high blood pressure, depression), health problems (eg heartburn, falls), and alcohol-interactive medications (eg pain killers, sedatives). We have previously shown moderate agreement between the CARET and Alcohol Use Disorders Identification Test for Consumption (AUDIT-C) in the classification of hazardous drinking in older adults.³³ Algorithm development required multiple steps, outlined in brief here. First, this project reflected an intervention within ‘real world’ PHC practices utilising tools and data available to these health professionals. Of the alcohol-harm risk factors listed in the CARET, our team determined that only chronic health conditions and prescribed medications would be accurately recorded in the PMS, as the remaining factors (eg heartburn, falls) were neither routinely screened for nor recorded and would not, therefore, be evidenced within the PMS. Second, we undertook a rapid and targeted literature review to confirm that the key health conditions and medications contra-indicated for alcohol use in the CARET still reflected current medical evidence. An overview of this rapid review is available from the authors on request. Third, we confirmed that the key medications identified in the CARET were assessed to fall within four broad medication groups identified in the PMS used across the 11 PHC practices (ie musculoskeletal, cardiovascular, analgesia, and central nervous system). Fourth, led by our clinical champion (JM) and the CARET author (AM), and abiding by the precautionary principle,³⁴ the medication groups and health conditions were individually assessed by the team to determine the nominal threshold of alcohol use frequency and typical quantity at or beyond which consumption might interact/exacerbate and be hazardous to health (see Table 1).

Where alcohol use is already recorded in the PMS, the algorithm would classify the patient as being in the ‘hazardous’ drinking range if drinking thresholds for any of the medication or comorbid health problems were met or exceeded (see Fig. 1). This would appear as a prompt (red flag) on the PMS, alerting the PHC professional to rescreen the patient for current alcohol use to confirm that the patient was still at risk. Where alcohol use was not recorded in the PMS, the algorithm would identify that the patient had chronic conditions and/or alcohol-interactive medications likely to increase the risk of alcohol use and prompt the health professional to screen the patient using an indicator on the PMS (see Fig. 2). The clinical champion worked with the PMS software engineer to embed this algorithm into the PMS. Mock cases differing in alcohol use patterns, levels of comorbid issues, and medication use were created by the team and entered into the PMS to test both the screening process, the set thresholds distinguishing ‘hazardous’ drinking from non-hazardous drinking, and the appearance of appropriate prompts. The embedded algorithm was then pilot tested by the clinical champion with health professionals at one PHC practice to assess health professional use and understanding of the process in situ. The algorithm (incorporating the CARET) went live simultaneously across all PHC practices.

Fig. 1.

Identifying potential hazardous drinking for older adults when alcohol use data are present.

Fig. 2.

Identifying potential alcohol risk for older adults with no alcohol use data.

Supporting discussions about alcohol with patients

In parallel to algorithm integration and intervention roll-out, the team supported health professionals’ confidence in discussing alcohol with patients by providing training on multiple areas related to screening and engagement. First, through face-to-face and ongoing online workshops with the clinical champion, health professionals were all provided training on the algorithm-supported screening process, what prompts to expect in the PMS, and how to manage them. Second, a broad set of training opportunities were designed for health professionals that supported engagement practices in communicating with patients about alcohol use, and the management of identified alcohol use concerns. This training was designed to provide continuing medical education (CME) credits (for general practitioners) and continuing professional development (CPD) credits (for other PHC professionals) and was supported via existing CME and CPD knowledge networks. Understanding the time and resource limitations facing health professionals, these training opportunities included e-learning components, such as an online short-course offered through the University of Auckland Goodfellow Unit, as well as webinars and podcasts about clinical management of alcohol use issues in PHC settings. The training also included face-to-face options, such as training workshops on Māori approaches to motivational interviewing including techniques such as open-ended questions, affirmations, reflections and summaries (OARS³⁵), evening educational seminars on the current evidence concerning alcohol use trends and risk for adults aged 50+, and clinical peer review of cases at the Royal New Zealand College of General Practitioners (RNZCGP) Whanganui sub-faculty. All training opportunities were optional for health professionals to attend (ie there was no compulsion).

Outcome evaluation

Three methods were used to evaluate the outcomes of this project. First, the primary intent of this project was to enhance the overall rate of alcohol screening undertaken in patients aged 50+ attending any of the 11 PHC practices. Our team assessed this change using quantitative analysis of the extant screening indicator variables in the PMS used by these PHC practices. Working with the PMS software engineer and data management officers from the Whanganui District Health Board, we developed a dataset of collective monthly screening rates across the 11 PHC practices for all patients aged 50+ (n = 24,411). We then compared collective monthly screening rates post-intervention (February 2020-November 2020) against a baseline rate reflecting the average monthly screening rate in a 3-month pre-intervention period (November 2019–January 2020). The use of PMS data reflects the intent of this project to be a pragmatic intervention in a ‘real world’ PHC setting which, if successful, could be replicated in other areas of the country with no requirement for additional and potentially burdensome data collection methods.

The second evaluation was a quantitative online survey exploring the level of health professional engagement with the training provided, awareness of change with the alcohol screening process, and perceived changes in their alcohol screening practice. Invitations for participation, including the online Qualtrics survey link, were sent to the 41 health professionals working across the 11 government-funded, community-based PHC practices included in this project, and 21 completed this survey (14 general practitioners, three practice nurses, two nurse practitioners, and two other health professionals).

The third evaluation method was semi-structured interviews of seven health professionals (five general practitioners, and two nurses) who completed the online survey and had indicated on that survey that they consented to be interviewed. These were undertaken face-to-face, via telephone or video call (depending on participant preference) by a qualitative interview specialist (GW) who was independent of the algorithm development and health professional training components of this intervention. Interviewees were asked about the utility of the intervention to support screening, discussions about alcohol with patients, attendance and utility of the training opportunities, and barriers experienced throughout this intervention. Interview responses were recorded, transcribed, and analysed using thematic analysis to identify key themes.³⁶

Screening rates

The number of alcohol-related screens increased at the time of initial training, indicating an immediate impact of the intervention over baseline. However, these screening numbers decreased immediately with the onset of the first national COVID-19 lockdown (April–May 2020) and effectively reverted to baseline in the months following lockdown (see Fig. 3). The reduction in screening rates associated with the national lockdown reflected direct advice given to all PHC providers indicating that the principle focus of their work during the national lockdown was on the management of chronic and/or acute illnesses, with normal screening processes significantly reduced in priority.

Fig. 3.

Number of patients screened each month (December 2019 to November 2020) compared to the number of patients screened at baseline (ie November 2019). Note: periods of COVID-19 restrictions are indicated by shaded bars.

Health professional engagement and practice change

Approximately 50% (n = 21) of the 41 health professionals included in this intervention completed an online evaluation survey. Table 2 provides an overview of this data, including their engagement with training and perceived effect training and intervention had on their practice. Only half of the respondents were able to engage with the training in this intervention, and this engagement favoured the face-to-face information and evidence workshop rather than skills training (eg the motivational interviewing workshop) or online CME-related e-learning opportunities. However, almost two-thirds of respondents were aware of the intervention and the ‘red flags’ indicating hazardous drinking in the PMS, and 43% felt this helped change their screening practice. Over 40% indicated that this process changed their alcohol screening practice. One-third of the respondents felt the training raised their awareness of the increased risk of alcohol for adults aged 50+ and that they were more comfortable in screening them for alcohol use, but only 14% indicated that training had improved confidence in discussing alcohol use with adults aged 50+.

Health professional feedback on screening intervention utility

The first theme from the interviews was that all interviewees felt the PMS prompts (ie red flags) were useful in identifying patient alcohol risk, facilitating discussions about alcohol use, and legitimising such discussions for both parties. Example comments to this effect were:

I pay attention when I see the red comes up … I am more likely to discuss the effect of alcohol on their general conditions. (Participant 2)

It’s made it much easier to bring it up with people because you can say “Hey it’s flashing up red … um … can I quickly ask you about alcohol?”. (Participant 3)

I might say “the computer is recommending this based on expert advice”. (Participant 7)

A further theme was that the PMS prompts helped remind interviewees to consider patient alcohol use within the wider context of morbidity and medication use, for example:

It made me more aware of the potential, yep, the interaction with the alcohol. (Participant 6)

Concerning training, a key theme from interviewees was that the training they had engaged with was useful for their practice, particularly the face-to-face skills training that enabled interactive feedback with others, for example:

We did some role plays and so on which gave people the opportunity to try out their skills and get some feedback on their communication. (Participant 3)

Peer meetings also have been great for discussion. (Participant 4)

However, a common theme for most interviewees was the existence of barriers to engagement with training opportunities, largely concerning their busy work roles, which may explain why 48% of survey respondents had failed to engage with any of the available training:

It’s just in times when I can’t drive down. (Participant 1)

There is a continual struggle with work life balance. (Participant 2)

I should say that I consider myself one of the burnt-out doctors. (Participant 4)

Strengths and weaknesses

This intervention had both strengths and weaknesses. The first strength was the partnership with a local ‘clinical champion’. They were able to lead the design and development of this intervention on behalf of the local PHC health professionals and could emphasise for them the clinical importance and patient benefits of the work being undertaken in a manner that facilitated their engagement. A second strength of this intervention was the focus on a low-burden approach to enhancing alcohol screening. By utilising extant patient medical records, we were able to demonstrate how normally time-burdensome screening processes could not only be automated but done so in a manner providing risk-thresholds specific to the health profile of the patient themselves. However, a key weakness of this current intervention was the inability to counter the lack of time available for health professionals to engage with training opportunities. While a range of face-to-face and online training opportunities were provided in anticipation of such issues, the reality was that almost half were unable to attend any such training. It is possible that a more efficient approach that may have supported health professionals’ attendance would have been to provide fewer training options but more repetition of these offerings.

Implications for health policy, practice, and future research

A PMS-based alcohol screening algorithm was delivered across PHC practice for the purposes of this project, but it remains live even now at project conclusion. To this extent, despite mixed findings, this algorithm will still support alcohol screening by prompting health professionals in these practices to discuss alcohol use with older clients whose records highlight risk factors for their drinking. This also supports the replication of this algorithm within the PMS of other PHC practices around the country as a simple and pragmatic foundation for supporting alcohol screening enhancement.

Three points relevant to the current intervention now require investigation. First, would it be possible to replicate this pilot in a randomised controlled trial approach within which we could assess ‘dose’ of the intervention across time and compare to a control set of PHC practices? Feedback suggested that this intervention supported knowledge and practice enhancement for some health professionals, but the limitations of this pilot design (eg use of time-limited baseline rather than PHC practice control sites) may have provided more evidence of utility. Second, while automated screening processes may enhance the detection of alcohol-related risk in those aged 50+, does this translate into improved management of these patients? It is unclear from our data the degree to which such detection was associated with adequate or enhanced management of alcohol-related issues. Third, do we have the appropriate resources and health system pathways required for the management of adults aged 50+ who are now identified as hazardous drinkers? International evidence highlights the need for the management of such drinkers but that current avenues for support are not designed for this cohort.⁴⁰ The answers to these questions have direct implications for the sustainability of alcohol screening and management practice for older adults. Our project provides an algorithm to support alcohol screening enhancement in PHC practices but highlights that the algorithm alone is insufficient to overcome barriers in alcohol screening and management. These barriers likely relate to the sustainability of and support for health professional training, and the need for a system of options appropriate for the management of older drinkers.