Applying implementation science theories to support practice change in the assessment of cognition by occupational therapists
Jacqueline Wheatcroft
A E * , Rebecca J. Nicks A B , Laura Jolliffe C D , Danielle Sansonetti A , Carolyn Unsworth A F *
A
B
C
D
E
F
Handling Editor: Grahame Simpson
Abstract
Understanding cognitive impairments is essential for effective rehabilitation and discharge planning for adults with neurological conditions. The aim of this study was to identify barriers to completing standardised cognitive assessments and evaluate the implementation of an intervention to support practice change.
A mixed-methods approach was applied to translate cognitive assessment recommendations into clinical practice using the Theoretical Domains Framework (TDF) and the Capability, Opportunity, and Motivation Behaviour model (COM-B) theories. Occupational therapists at one metropolitan health service in Australia were invited to participate. Pre- and post-implementation file audits and surveys were conducted, along with focus groups that collected qualitative data analysed using the TDF and COM-B.
Survey 1 (n = 40) and focus group data (n = 24) identified barriers in the TDF domains of knowledge (selection of assessments), environment and resources (equipment and time constraints), and social influences (pressure from other disciplines). To address barriers to implementing a cognitive assessment framework, scripts, cue cards, video-recorded training, and posters were developed as guided by the Behaviour Change Wheel (BCW). Survey 2 showed increased capability to physically administer cognitive assessments (53–74%) and improved clinician understanding of relevant clinical practice guideline (CPG) recommendations (22–50%). File audit data indicated a 30% increase in the number of standardised assessments completed.
The application of two implementation theories led to the development of an intervention that increased occupational therapists’ confidence and their adherence to CPG recommendations. This study serves as a potential model for using the TDF and COM-B to create implementation interventions in various clinical practice areas.
Keywords: behaviour change techniques, brain injury rehabilitation, cognitive assessment, implementation science, intervention development, knowledge translation, rehabilitation, stroke rehabilitation.
Introduction
Cognitive deficits, common following stroke and brain injury, often hinder an individual’s engagement in daily activities (Lannin et al. 2014; Kapoor et al. 2017; Stolwyk et al. 2021). In this context, occupational therapists play a crucial role in assessing and treating individuals with cognitive impairments (Unsworth 2017). The early assessment of cognition is a pivotal step in effective rehabilitation and discharge planning (Ozdemir et al. 2001) and is recommended in clinical practice guidelines (CPGs) (Bayley et al. 2014; Stroke Foundation 2019). However, many occupational therapists lack awareness of these CPG recommendations (Nott et al. 2020), and the challenges they face in selecting valid and reliable assessment tools contribute to the low implementation rates of guideline recommendations for using standardised cognitive assessments (Burns and Neville 2016; Stigen et al. 2018; Manee et al. 2020).
Internationally, occupational therapists consistently report challenges in using standardised cognitive assessments (Koh et al. 2009; Burns and Neville 2016; Stigen et al. 2018). One of the issues that has been identified is occupational therapists’ preference for functional observation over the use of assessments with published psychometric properties. Additionally, occupational therapists tend to rely on familiar and readily available assessments as well as those used by their colleagues (Koh et al. 2009; Burns and Neville 2016; Stigen et al. 2018). Compounding this issue, current guidelines do not provide prescriptive recommendations regarding specific cognitive assessment tools for patients with neurological conditions, thereby increasing variability in practice (McMahon et al. 2022). Given these challenges, occupational therapists may adapt assessments or use non-standardised methods without established psychometric properties (Koh et al. 2009). Functional observation in a controlled hospital setting is often insufficient to detect subtleties of impairments in cognition (Giles et al. 2020). Impairments may go unrecognised if standardised assessments are not completed, limiting rehabilitation goal attainment (Giles et al. 2020). Failing to assess cognitive impairments accurately can thus significantly affect the quality of neurological rehabilitation. Additionally, it can reduce occupational therapists’ ability to support discharge planning (Ablewhite et al. 2019). One method of enhancing guideline adherence at a local level is through knowledge translation (Romney et al. 2022), an iterative process where evidence is applied to improve health and healthcare systems (Straus et al. 2009).
Knowledge translation supports implementing evidence-based practices in the busy clinical setting (Michie et al. 2014). Whilst navigating practice change can be challenging, the use of theoretical frameworks such as the knowledge-to-action (KTA) framework (Graham et al. 2006) and the Theoretical Domains Framework (TDF) (Michie et al. 2005) facilitates this process. These frameworks detail the steps to tailor knowledge to the local context and then apply knowledge within the clinical setting. Several occupational therapy studies have successfully demonstrated the effective use of theoretical frameworks to guide clinical knowledge translation, such as the Capability, Opportunity, Motivation, and Behaviour model (COM-B) alongside the TDF (Eames et al. 2018; Jolliffe et al. 2019). Applying a stepped method, bound in theory, allows researchers to identify barriers and subsequently map these to behaviour change interventions (Michie et al. 2014), thus improving the likelihood of change in practice.
Accordingly, our study has three aims:
To identify the barriers and enablers to occupational therapists’ provision of recommended standardised cognitive assessments.
To describe the development of an intervention to increase occupational therapists’ use of standardised cognitive assessments for adult neurological patients.
To evaluate the implementation of this intervention in a single health service.
Methods
Participants
A convenience sample was selected from a large metropolitan-based occupational therapy team within a single health service that employed approximately 90 occupational therapists over three campuses. Occupational therapists work in various clinical specialties across the spectrum of care (i.e. emergency, acute services, rehabilitation, and community care). The department has a history of completing knowledge translation research and employs occupational therapist researchers with qualifications in implementation science. Approximately a quarter of the team holds or is working towards higher degree qualifications. Occupational therapists were excluded from the study if they were part of the research team, students, assistants, or working in psychiatry. Ethical approval was approved for this study (Alfred Health 180/17).
Design
A stepped method for developing the intervention on the basis of theory, evidence and practical issues was first applied. Then, a mixed-methods implementation design guided the three key stages outlined in the Behaviour Change Wheel (BCW) framework (Michie et al. 2014) (Fig. 1). The BCW consists of three stages: Stage 1, understand the behaviour; Stage 2, identify intervention options; and Stage 3, identify content and implementation options. Each of the stages of the BCW is further divided into steps. An additional stage (Stage 4) was added to evaluate the impact of behaviour change techniques (BCTs) in supporting best practices in cognitive assessments (Fig. 1). The study methods and results will be presented under these four stages, focusing on steps four through eight – identifying what needs to change and intervention strategies.
The behaviour change intervention design process (Michie et al. 2014) mapped to the knowledge-to-action (KTA) framework (Graham et al. 2006).
The clinical knowledge for implementation in this study consisted of a framework providing recommendations to select appropriate cognitive assessments on the basis of clinical needs, goals and diagnoses. To develop this, we completed the knowledge creation component of the KTA framework (knowledge inquiry and knowledge synthesis). We synthesised published research (Lannin and Scarcia 2004; Cusick et al. 2014; Hobson et al. 2016; Edlin et al. 2018; Romero-Ayuso et al. 2021), reviewed CPGs (Bayley et al. 2014; Jolliffe et al. 2018; Stroke Foundation 2019), and discussed practical considerations (such as training requirements and associated costs due to the large cohort of occupational therapists). Recommendations were synthesised into an organisational cognitive assessment framework (knowledge tools and products), which included 15 cognitive screening and assessment tools with sufficient psychometric research that have been shown to guide occupational therapy rehabilitation and education across the health service. The authorship team selected a range of cognitive screening and assessment tools to meet the specific needs of the service in terms of qualifications, patient population, and informational need (i.e. practice context). Whilst this cognitive assessment framework provided a clear, user-led approach to support occupational therapists’ procedural reasoning with regards to selecting a cognitive assessment, in itself, it does not consider what factors might affect occupational therapists’ ability to administer the recommended cognitive assessments nor the resources that would optimise its implementation.
To address the evidence–practice gap, we applied implementation science theories to improve occupational therapists’ routine use of the framework. Specifically, we sought to explore the barriers faced and enablers experienced by occupational therapists when conducting cognitive assessments. Next, we identified intervention functions that mapped to the barriers to develop our implementation package. Last, guided by the action cycle component of the KTA framework, we determined whether our implementation package enabled occupational therapists to select and administer assessments consistent with the framework (Graham et al. 2006).
Stage 1 (BCW): understand the behaviours
The first three steps of the BCW involve defining the problem in behavioural terms and selecting and specifying target behaviour, which is administering a cognitive assessment to guide occupational therapy education and/or rehabilitation. The current study commenced from the last step in Stage 1, investigating research question (i) the barriers, i.e. what needs to change. To understand the target behaviours and develop an awareness of what needed to change within the service, we completed a review of documented assessments (objective) and sought to understand occupational therapist beliefs about cognitive assessment (subjective).
This study used three data collection methods: file audits, clinician surveys, and clinician focus groups.
The first audit (Audit 1) of medical records of 150 inpatients with neurological conditions requiring occupational therapy cognitive assessment was completed by one author (J. W.) 8 weeks before the baseline survey. The audit collected de-identified patient demographic and clinical data alongside the type of cognitive assessment completed and whether that assessment conformed to the cognitive assessment framework. The same author repeated the file audit (Audit 2, n = 150) 2 months after delivering the implantation package.
All occupational therapists in physical settings received an email invitation with a survey link (Survey 1) and participant information document. Consent was implied through participation. Survey questions were developed by one author (N. L.) based on work by Huijg et al. (2014) and Skoien et al. (2016) to understand occupational therapists’ knowledge of CPGs and their views on the barriers and enablers to the cognitive assessment framework recommendations. Participants were asked to rate each survey question on a 5-point Likert scale, from strongly disagree to strongly agree. Two open-ended questions were asked:
Please list any cognitive assessment recommendations or clinical practice guidelines that you are aware of.
Please provide any further comments you may have regarding the use of standardised cognitive assessments.
The repeat survey (Survey 2) was sent out 8 weeks following the delivery of the intervention package.
The focus groups were conducted by one of two researchers (J. W. and R. N.) employed within the department in research and quality roles, with a senior researcher (N. L.) present during the first group as a mentor. Occupational therapists were recruited to join the focus group via email invitation from the research team, which included an attached information letter and consent form. The focus group discussion guides were developed following the analysis of the survey results and on the basis of work by Huijg et al. (2014) and Skoien et al. (2016). Focus group dialogue was audio recorded and transcribed verbatim. Individual participants’ voices were not coded separately; therefore, quotes are identified by focus group number only.
Survey data were analysed using descriptive statistics with weighted averages of occupational therapists’ level of agreement with TDF domains calculated within Microsoft Excel. Responses to open-ended survey questions were deductively analysed and coded by two authors (J. W. and N. L.) to the TDF domains. The survey and focus group responses were triangulated to gather a range of perspectives across the modes of data collection (Atkins et al. 2017). All focus group data were deductively coded to the TDF. Two authors (J. W. and L. J.) double-coded 20% of the transcripts to develop consistency in code selection. Discrepancies in coding were resolved through discussion and consultation with the senior researcher experienced in translational research (N. L.). Subsequently, a single researcher (J. W.) coded the remaining three focus groups. Pre- and post-implementation package file audit data (Audit 1 and 2) were compared using descriptive statistics, including calculating weighted averages and percentage changes.
Stage 2 (BCW): identifying intervention options
Sources of behaviour or barriers identified through the pre-implementation survey and focus groups were first summarised against the COM-B and then mapped to intervention functions on the BCW (Table 1). Intervention functions and BCTs were thus drawn from work by Michie and colleagues (Michie et al. 2014).
| Survey 1 (%) | Survey 2 (%) | ||
|---|---|---|---|
| Location of participants: A | (n = 38 B) | (n = 33) | |
| Acute hospitals | 13 (31) | 10 (24) | |
| Inpatient rehabilitation | 26 (62) | 24 (66) | |
| Community rehabilitation | 3 (7) | 3 (9) | |
| Using cognitive assessments: | (n = 39 B) | (n = 33) | |
| Yes | 39 (100) | 33 (100) | |
| Aware of a framework of cognitive assessments: | (n = 39) | (n = 33) | |
| Yes | 38 B (97) | 33 (100) | |
| Frequency of completion of cognitive assessments: | (n = 40) | (n = 32 B) | |
| More than daily | 3 (8) | 3 (9) | |
| Daily | 5 (13) | 5 (16) | |
| 1–2 times a week | 6 (15) | 5 (16) | |
| 3–4 times a week | 9 (23) | 5 (16) | |
| Fortnightly | 10 (25) | 6 (19) | |
| Once a month | 3 (8) | 5 (16) | |
| Less than monthly | 4 (10) | 3 (9) |
Stage 3 (BCW): identify content and implementation options
The specific content of the implementation package was selected on the basis of the APEASE criteria (affordability, practicability, effectiveness/cost-effectiveness, acceptability, side-effects/safety, equity) (Michie et al. 2014). Mapping the barriers involved discussion within the research team and consultation with managers regarding financial considerations aligning with the APEASE criteria. To address research question (ii), intervention content resources were developed and implemented within the occupational therapy department through three staff in-service presentations, communicating and sharing electronic resources, and providing physical resources to all occupational therapists. The intervention content resources are detailed in Supplementary Table S1; the intervention functions of enablement, education, training, environmental restructuring, and modelling are all featured in the package.
Stage 4 (BCW): evaluation of the effectiveness of the behaviour change techniques
The survey (Survey 2) was repeated 8 weeks after the intervention to determine whether the implementation package effectively increased occupational therapists’ perceived knowledge, confidence, and compliance with administering standardised cognitive assessments. The weighted average results were compared from the pre- and post-intervention data sets (Survey 1 and Survey 2), and the frequencies of open-ended questions were compared (i.e. the number of guidelines described). To further evaluate the impact of the implementation, patient medical records were audited (Audit 2) 8 weeks after the intervention resources were released.
Results
Survey
Out of 90 occupational therapists employed in the department, 40 occupational therapists responded to the pre-intervention Survey 1 (49%) and 33 (41%) to the post-intervention Survey 2 (Table 1). Occupational therapists reported consistent frequencies of the administration of cognitive assessments at both time points – with 21% (Survey 1) and 25% (Survey 2) reporting they conduct assessments ‘daily’ or ‘more than daily’. Over half (58% in Survey 1 and 56% in Survey 2) of occupational therapists reported completing cognitive assessments ‘weekly’ (1–2 times or 3–4 times weekly). All 100% of the occupational therapists who participated in Survey 1 and 2 indicated their awareness of the cognitive assessment framework.
In Survey 1, most occupational therapists agreed they had been allowed to develop their physical skills specific to administering cognitive assessments (85%) and agreed they knew how to administer assessments following recommendations (93%). However, some (18%) responded neutrally or disagreed to have received training in administering assessments. Overall, occupational therapists indicated their acceptance of the cognitive assessment framework, which helped them select appropriate assessments:
It is really great that OT [occupational therapy] is addressing issues around cognitive assessment across Alfred Health so there is consistency regardless of where we work. (S1:P8)
Most participating occupational therapists in Survey 1 (76%) reported they had the necessary resources to administer cognitive assessments:
The introduction of a written form to complete post a CAM [Cognitive Assessment of Minnesota] has been extremely helpful. (S1:P1)
On the other hand, they indicated there was further opportunity to ensure all staff have access to the required resources and facilities. In relation to the TDF domain of environmental context and resources, occupational therapists identified barriers, including environmental constraints such as:
It is very difficult to complete the Cognitive Assessment of Minnesota [CAM] in the acute setting due to noise and not enough space and privacy. (S1:P11)
Some occupational therapists (17%) in Survey 1 disagreed with the statement: ‘Planning the rehabilitation of cognition, based on the results of their assessments was very easy’ (belief about capabilities). A further 21% of respondents neither agreed nor disagreed with this statement. Occupational therapists reported that the experience of completing assessments enhances their abilities; one reported:
I feel confident with specific assessments that I have used and am familiar with; however, a few of the assessments I have not administered and therefore do not feel confident with. (S1:P14)
Focus groups
Occupational therapists (n = 24) participated in five focus groups (FG1–5) comprising two to six participants. The focus group duration was approximately 60 min long.
Occupational therapists recognise that guidelines support their practice (TDF knowledge domain). However, clinical guidelines only exist for certain patients, such as those with stroke (Stroke Foundation 2019). Participants recognised this barrier:
I think in stroke, we are quite fortunate to have quite clear guidelines that say what the gold standard is and that we should be doing these formal assessments, and I think other patient cohorts it’s less clear what the expectation is and perhaps how rigorous we should be testing their cognition… (FG4)
Co-existing medical conditions impacted clinicians’ ability to decide what and when to complete cognitive assessments coded to memory, attention, and decision processes (TDF).
Knowing when to do an assessment is often something that I find hard and in terms of the barrier because, particularly with aged care, in terms of if they are unwell, or … if they’ve got a UTI [urinary tract infection], they’ve got a delirium, those sorts of things can often then be quite hard to know. (FG4)
During the focus groups, participants were asked about their understanding of the reasons behind selecting assessments. While some participants were able to provide valid reasons, such as the psychometric properties of the assessments, many others were unclear. This highlighted the importance of incorporating evidence from systematic reviews and CPGs in the implementation package to enable occupational therapists to comprehend the rationale behind the cognitive assessment framework.
Environmental context and resources is a domain that affects occupational therapists’ ability to administer cognitive assessments. Occupational therapists faced environmental obstacles while working in four-bed bays, but newer parts of the hospital offered more suitable assessment areas.
I feel like I’m in paradise here in comparison to places I’ve worked because we’ve got that individual space; either the patient’s room or we’ve got other individual treatment rooms. (FG2)
Occupational therapists found the cognitive assessment framework to be a helpful tool, allowing new and rotating occupational therapists to learn about the selection and administration of cognitive assessments quickly.
Having a very clear starting point of where we should be looking in terms of cognitive assessments really helps, I think, junior staff as well find their feet a little bit around what the expectations are and what kind of assessments they can go forward with. (FG4)
Difficulties relating to team roles in administering cognitive assessments were identified as a potential barrier and coded under social influences: ‘… maybe more junior staff has maybe felt pressured by other disciplines to not use our recommended intervention tools or assessment tools. That can be a challenge’ (FG2).
Occupational therapists’ social role was seen as an enabler, with many identifying the administration of cognitive assessment as central to the profession. ‘I think it’s a core part of my role as an occupational therapist. Cognition can be aligned to multiple disciplines, but I think we play a key part’ (FG2).
Occupational therapists’ beliefs about their capabilities demonstrated that they recognise areas for self-improvement, particularly in assessing executive functioning.
I think it’s I’m less confident with the higher-level executive dysfunctions, especially with some of the really high-functioning pre-morbidly high-functioning people that are going home, they live by themselves. (FG3)
Stage 2: mapping the barriers to intervention strategies
The barriers identified from the survey and focus groups were mapped to the COM-B and TDF domains. Subsequently, these were mapped to the intervention functions and strategies (Fig. 2).
Stage 3: identifying content and implementation options
The content of the intervention strategies was developed from the focus group and survey data (Supplementary Table S1). The intervention package consisted of various strategies, including an in-service presentation that delineated the evidence supporting the selection of cognitive assessments. A video recording of the presentation was made available for new staff and staff who could not attend the in-service presentation. Further, occupational therapists were provided with resources, including posters and cue cards of the cognitive assessment framework, training videos demonstrating the administration of each cognitive assessment, quick reference guides, scripts for introducing assessments and providing feedback based on the results of the assessments, and guides for documenting assessment results in electronic medical records. Resources were provided to enable consistency of practice across the team. New staff were orientated to these resources by senior staff members to promote sustainability.
Stage 4: evaluation of the effectiveness of the behaviour change techniques
Analysis of the weighted average responses (Table 2) indicated a positive shift in 12 of the 14 survey domains (closer to a score of ‘1’ = ‘agree’). At the time of Survey 1, only 22% of occupational therapists reported a relevant CPG. In Survey 2, 50% reported a CPG, indicating increased knowledge of clinical practice guidelines underpinning assessment selection. Additionally, clinicians reported a greater understanding of the rationale behind the selection of assessments.
Since the cognitive framework in-service, I have observed increasing confidence in junior staff’s ability to articulate the reason for completing formalised cognitive assessments such as CAM and Kettle Test, and a greater understanding of why these assessments are specifically recommended for use at Alfred Health, i.e. psychometric properties. (S2:P17)
| COM-B | TDF domains | Survey question examples | Survey 1 (Baseline) weighted average ± (s.d.) | Survey 2 (Post-intervention) weighted average ± (s.d.) | Difference between groups | |
|---|---|---|---|---|---|---|
| Number of questions (n) | n = 40 | n = 33 | ||||
| Capability: Individual’s physical and psychological capacity to engage in the behaviour | Physical skills (2) | I have had the opportunity to develop my skills in administering cognitive assessments. | 1.8 (1.6) | 1.4 (1.5) | −0.4 | |
| Knowledge (3) | I know how to administer cognitive assessments following the recommendations. | 1.7 (1.6) | 1.5 (1.7) | −0.2 | ||
| Memory, attention, and decision processes (3) | I find it easy to remember what cognitive assessments are recommended. | 1.6 (1.5) | 1.3 (1.3) | −0.3 | ||
| Behavioural regulation (1) | I have a clear plan for administering the recommended cognitive assessments. | 1.7 (1.5) | 1.3 (1.4) | −0.4 | ||
| Opportunity: Factors outside the individual that make the performance of behaviour possible or prompt it | Social influences (1) | I can count on support from the management of my organisation when things get tough and I need to meet the recommendations. | 1.8 (1.7) | 1.2 (1.3) | −0.6 | |
| Environmental context and resources (3) | In my organisation, all necessary resources are available to administer cognitive assessments that adhere to best-practice guidelines. | 1.9 (1.9) | 1.6 (1.9) | −0.3 | ||
| Motivation: All brain processes that energise and direct behaviour | Beliefs about capabilities (7) | I am confident that I can deliver cognitive assessments according to the recommendations. | 2.3 (2.3) | 1.9 (2.1) | −0.4 | |
| Optimism (1) | When I administer cognitive assessments, I feel optimistic. | 2.5 (2.2) | 2.2 (2.4) | −0.3 | ||
| Belief about consequences (1) | If I deliver a recommended cognitive assessment, I will be able to tailor a person’s cognitive rehabilitation more accurately. | 2.0 (1.8) | 1.6 (2.4) | −0.4 | ||
| Intentions (1) | I always plan to administer cognitive assessments following the recommendations. | 1.7 (1.5) | 1.3 (2.4) | −0.4 | ||
| Goals (1) | Other assessments or interventions are often more urgent than administering cognitive assessments.A | 3.0 (2.8) | 2.6 (2.9) | −0.4 | ||
| Social/professional role and identity (1) | Administering standardised cognitive assessments, which the managers list, is part of my work as an occupational therapist. | 1.1 (0.4) | 1.2 (1.3) | 0.1 | ||
| Reinforcement (1) | When I administer cognitive assessments, I get recognition from my peers. | 3.1 (2.9) | 2.6 (2.9) | −0.5 | ||
| Emotion (1) | When I administer cognitive assessments, I feel nervous. | 2.1 (1.8) | 2.3 (2.6) | 0.2 |
The results of two file audits (Supplementary Table S2), one conducted prior to the intervention (Audit 1, n = 150) and one after (Audit 2, n = 150), found that all the assessments administered by occupational therapists were consistent with the cognitive assessment framework. Moreover, the second file audit (Audit 2) showed a 30% increase in patients who received a standardised cognitive assessment by an occupational therapist. Overall, there was a 29% increase in the total number of assessments completed. The number of files that contained more than one assessment remained essentially unchanged (Audit 1, n = 12 (30%) and Audit 2, n = 13 (25%)).
Discussion
This study emphasises the critical role of hospital-based occupational therapists in administering cognitive assessments. Results indicate clinical barriers to administrating cognitive categorised under the TDF domains of knowledge, social influences, and environmental context and resources. Occupational therapists reported enablers to the administration of cognitive assessments across the domains of reinforcement, social role and identity of their professional role, and environmental context and resources, suggesting the importance of harnessing these enablers for improving future use. On the basis of our findings, the resources developed in the implementation package included tools that built on these enablers and/or addressed the barriers, such as cue cards, new equipment, educational videos, and scripts for supporting clinical discussion of findings. Overall, the implementation of these targeted resources supported the frequency of administration and supported occupational therapists’ perceived ability to administer standardised cognitive assessments.
Targeting identified barriers raised by occupational therapists improved adherence to the department’s cognitive assessment framework. Consistent with previous studies, some occupational therapists lacked confidence in cognitive assessment and translating it into rehabilitation plans (Burns and Neville 2016). Another highlighted barrier was accessing a quiet space to conduct a cognitive assessment in a busy hospital. In contrast, single rooms enabled the administration of assessments in the more recently built facilities. Similarly, other studies using the TDF and comprising a sample of occupational therapists within a hospital have also identified barriers within the environment, which may indicate the confines of the setting (Eames et al. 2018; Jolliffe et al. 2019).
It is acknowledged that the standardised cognitive assessment tools recommended in the cognitive framework may be susceptible to floor and ceiling effects. Occupational therapists struggle to select appropriate assessment when patients cannot engage with a standardised or functional cognitive assessment or when high functioning patients need to return to paid employment, and an assessment of executive skills may be insufficient. Nonetheless, this study supports the idea that providing a framework improved clinicians’ knowledge of and physical capability to administer recommended cognitive assessments, resulting in an increased number of assessments completed. In this study, occupational therapists used national guidelines, including the Living Clinical Guidelines for Stroke Management (Stroke Foundation 2019), and the framework to select assessments. Occupational therapists recognised guidelines as enablers to support their practice; however, they highlighted the lack of guidelines for all patient groups. This finding is consistent with McMahon and colleagues’ review that reported a lack of guidelines regarding which cognitive assessments should be completed among specific patient groups (McMahon et al. 2022). Further work within CPGs detailing the timing and tools to use for specific populations may enhance procedural reasoning in clinicians, particularly those with less experience (McMahon et al. 2022).
Challenges to the occupational therapy role exist, such as other professional disciplines querying the selection of cognitive assessment and the overlap of responsibilities between professions. For instance, occupational therapists felt their capability was undermined when other professionals queried their selection of an assessment. Therefore, we recommend occupational therapists, particularly novices, who may benefit from training to understand the importance of assessing and rehabilitating cognitive impairments and the unique differences in assessment approaches between professionals. Strategies such as modelling through scripts were part of our implementation package; however, developing clear scope of practice between roles and within a workplace, explicitly concerning cognitive assessments, may be required.
The three domains of key enablement found in our study were: environmental context and resources, reinforcement, and social role and identity. Consistent with the literature, access to the recommended assessments and related resources is a primary enabler for administration (Manee et al. 2020). Occupational therapists strongly believe that conducting cognitive assessment is part of their role and aligns with the profession’s priorities (Giles et al. 2020; Swiatek and Jewell 2020). While worldwide implementation rates of standardised cognitive assessment are low, participants in the study completed these assessments routinely and followed the local framework. Further, occupational therapists reported that if they deliver cognitive assessments more frequently, it becomes easier. Hence, it may be implied that regular practice of cognitive assessment administration helps to develop skills and confidence in one’s capabilities.
Limitations
There are some limitations to this study. First, as with many clinical implementation studies, this work was slow and time-consuming. Hence, a high staff turnover contributed to the fluctuation of participating occupational therapists across the time points in this study. To mitigate the impact of staff turnover and to build sustainability, new staff were orientated to the intervention strategies by the first author (J. W.). Second, the study was conducted at a single organisation, which may limit the generalisability of identified barriers. The cognitive assessment framework was developed specifically to meet the needs of the local population. Therefore, it may be advantageous for other organisations to replicate and tailor the outlined process to their settings. Identified intervention strategies effectively increased the already high completion rate of cognitive assessments. Audit studies have shown that sites with lower adherence at baseline are likely to experience more effects from an intervention (Ivers et al. 2012). Hence, despite this study being completed at one site, it may be possible to translate the intervention strategies to other sites.
The hospital environment plays a key role in either facilitating or hindering cognitive assessments, as accessing the quiet space needed to conduct an assessment can be difficult in a busy ward. Additionally, the hospital environment presents barriers as evaluation of cognitive impairments often requires both standardised assessment and functional observation of performance in a naturalistic setting to be completed in parallel. Arranging cognitive assessment resources and quick reference guides to be readily available promoted ease of use whilst practice and frequency of administration supported skill and confidence. It is recognised that frameworks support procedural reasoning when selecting a cognitive assessment. Hence, pathways and frameworks need to be developed to meet local populations, but they should not remain static; instead, they should be reviewed iteratively and updated in line with new CPGs or changing service demands.
Conclusion
In conclusion, the hospital environment is critical in facilitating or hindering occupational therapists’ administration of cognitive assessments. The availability of cognitive assessment resources, such as quick reference guides, promotes ease of administration, while consistent practice and frequency of administration increase skill and confidence. Factors such as medical status and high-level cognitive impairments make it more difficult for occupational therapists to decide what and when to complete an assessment. However, frameworks can support procedural reasoning when selecting a cognitive assessment.
Our study highlights the significance of developing frameworks that cater to the needs of the local populations to support adherence to CPG recommendations and best practices in cognitive assessment. These frameworks should be reviewed iteratively and updated in line with new CPGs or changing service demands. By doing so, the ease of administration of cognitive assessments can be enhanced, and the quality of care can be improved.
Data availability
The data that support this study will be shared upon reasonable request to the corresponding authors.
Conflicts of interest
Senior author Natasha A. Lannin is a Guest Editor of Brain Impairment. To mitigate this potential conflict of interest they had no editor-level access to this manuscript during peer review. The authors declare no other conflicts of interest.
Declaration of funding
This research received no specific grants from funding agencies. The following researchers were supported by fellowship funding: N. A. L. [Heart Foundation of Australia Future Leader Fellowship #106762].
Author contributions
J. W., R. J. N., N. A. L.: Conceptualisation. J. W., R. J. N., L. J., D. S., C. U., N. A. L.: Investigation. J. W., L. J., N. A. L.: Methodology. J. W., R. J. N., L. J., N. A. L.: Analysis. J. W., N. A. L.: Writing – original draft. All: Writing – reviewing and editing.
Ethics standard
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
References
Ablewhite J, Geraghty J, das Nair R, Lincoln N, Drummond A (2019) Cognitive Management Pathways in Stroke Services (COMPASS): a qualitative investigation of key issues in relation to community stroke teams undertaking cognitive assessments. British Journal of Occupational Therapy 82, 404-411.
| Crossref | Google Scholar |
Atkins L, Francis J, Islam R, O’Connor D, Patey A, Ivers N, Foy R, Duncan EM, Colquhoun H, Grimshaw JM, Lawton R, Michie S (2017) A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implementation Science 12(1),.
| Crossref | Google Scholar |
Bayley MT, Tate R, Douglas JM, Turkstra LS, Ponsford J, Stergiou-Kita M, Kua A, Bragge P (2014) INCOG guidelines for cognitive rehabilitation following traumatic brain injury: methods and overview. Journal of Head Trauma Rehabililitation 29, 290-306.
| Crossref | Google Scholar | PubMed |
Burns SC, Neville M (2016) Cognitive Assessment Trends in Home Health Care for Adults With Mild Stroke. American Journal of Occupational Therapy 70, 7002290020p1-7002290020p8.
| Crossref | Google Scholar | PubMed |
Cusick A, Lannin NA, Hanssen R, Allaous J (2014) Validating the Western Neuro Sensory Stimulation Profile for patients with severe traumatic brain injury who are slow‐to‐recover. Australian Occupational Therapy Journal 61, 276-283.
| Crossref | Google Scholar | PubMed |
Eames S, Bennett S, Whitehead M, Fleming J, Low SO, Mickan S, Caldwell E (2018) A pre-post evaluation of a knowledge translation capacity-building intervention. Australian Occupational Therapy Journal 65, 479-493.
| Crossref | Google Scholar | PubMed |
Edlin K, Thompson E, Spillane C, Farquhar M, Schmidt J, Lannin N (2018) Assessments of cognition in older people after traumatic brain injury: an appraisal and review. New Zealand Journal of Occupational Therapy 65, 4-12.
| Crossref | Google Scholar |
Giles GM, Edwards DF, Baum C, Furniss J, Skidmore E, Wolf T, Leland NE (2020) Making Functional Cognition a Professional Priority. American Journal of Occupational Therapy 74, 7401090010p1-7401090010p6.
| Crossref | Google Scholar | PubMed |
Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N (2006) Lost in knowledge translation: time for a map? Journal of Continuing Education in the Health Professionals 26, 13-24.
| Crossref | Google Scholar | PubMed |
Hobson E, Lannin NA, Taylor A, Farquhar M, Morarty J, Unsworth C (2016) Determining client cognitive status following mild traumatic brain injury. Scandinavian Journal of Occupational Therapy 23, 138-146.
| Crossref | Google Scholar | PubMed |
Huijg JM, Gebhardt WA, Dusseldorp E, Verheijden MW, van der Zouwe N, Middelkoop BJC, Crone MR (2014) Measuring determinants of implementation behavior: psychometric properties of a questionnaire based on the theoretical domains framework. Implementation Science 9(1),.
| Crossref | Google Scholar |
Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD (2012) Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Systematic Review 6, Cd000259.
| Crossref | Google Scholar | PubMed |
Jolliffe L, Lannin NA, Cadilhac DA, Hoffmann T (2018) Systematic review of clinical practice guidelines to identify recommendations for rehabilitation after stroke and other acquired brain injuries. BMJ Open 8, e018791.
| Crossref | Google Scholar | PubMed |
Jolliffe L, Hoffmann T, Lannin NA (2019) Increasing the uptake of stroke upper limb guideline recommendations with occupational therapists and physiotherapists. A qualitative study using the Theoretical Domains Framework. Australian Occupational Therapy Journal 66, 603-616.
| Crossref | Google Scholar | PubMed |
Kapoor A, Lanctôt KL, Bayley M, Kiss A, Herrmann N, Murray BJ, Swartz RH (2017) “Good outcome” isn’t good enough: cognitive impairment, depressive symptoms, and social restrictions in physically recovered stroke patients. Stroke 48, 1688-1690.
| Crossref | Google Scholar | PubMed |
Koh CL, Hoffmann T, Bennett S, McKenna K (2009) Management of patients with cognitive impairment after stroke: a survey of Australian occupational therapists. Australian Occupational Therapy Journal 56, 324-331.
| Crossref | Google Scholar | PubMed |
Lannin NA, Scarcia M (2004) Multidisciplinary screening of cognitive impairment following acquired brain impairment. Is there repetition? Journal of Cognitive Rehabilitation, 23, 19-25.
| Google Scholar |
Lannin N, Carr B, Allaous J, Mackenzie B, Falcon A, Tate R (2014) A randomized controlled trial of the effectiveness of handheld computers for improving everyday memory functioning in patients with memory impairments after acquired brain injury. Clinical Rehabilitation 28, 470-481.
| Crossref | Google Scholar | PubMed |
Manee FS, Nadar MS, Alotaibi NM, Rassafiani M (2020) Cognitive Assessments Used in Occupational Therapy Practice: A Global Perspective. Occupational Therapy International 2020, 8914372.
| Crossref | Google Scholar | PubMed |
McMahon D, Micallef C, Quinn TJ (2022) Review of clinical practice guidelines relating to cognitive assessment in stroke. Disability and Rehabilitation 44, 7632-7640.
| Crossref | Google Scholar | PubMed |
Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, “Psychological Theory” Group (2005) Making psychological theory useful for implementing evidence based practice: a consensus approach. BMJ Quality & Safety 14, 26-33.
| Crossref | Google Scholar | PubMed |
Nott MT, Barden HLH, Chapparo C, Ranka JL (2020) Evidence based practice and knowledge translation: a survey of Australian occupational therapy practice with clients experiencing neurocognitive impairments. Australian Occupational Therapy Journal 67, 74-82.
| Crossref | Google Scholar | PubMed |
Ozdemir F, Birtane M, Tabatabaei R, Ekuklu G, Kokino S (2001) Cognitive evaluation and functional outcome after stroke. American Journal of Physical Medicine & Rehabilitation 80, 410-415.
| Crossref | Google Scholar | PubMed |
Romero-Ayuso D, Castillero-Perea Á, González P, Navarro E, Molina-Massó JP, Funes MJ, Ariza-Vega P, Toledano-González A, Triviño-Juárez JM (2021) Assessment of cognitive instrumental activities of daily living: a systematic review. Disability and Rehabilitation 43, 1342-1358.
| Crossref | Google Scholar | PubMed |
Romney W, Bellows DM, Tavernite JP, Salbach N, Deutsch JE (2022) Knowledge Translation Research to Promote Behavior Changes in Rehabilitation: Use of Theoretical Frameworks and Tailored Interventions: A Scoping Review. Archives of Physical Medicine and Rehabilitation 103, S276-S296.
| Crossref | Google Scholar | PubMed |
Skoien W, Page K, Parsonage W, Ashover S, Milburn T, Cullen L (2016) Use of the Theoretical Domains Framework to evaluate factors driving successful implementation of the Accelerated Chest pain Risk Evaluation (ACRE) project. Implementation Science 11(1),.
| Crossref | Google Scholar |
Stigen L, Bjørk E, Lund A, Cvancarova Småstuen M (2018) Assessment of clients with cognitive impairments: a survey of Norwegian occupational therapists in municipal practice. Scandinavian Journal of Occupational Therapy 25, 88-98.
| Crossref | Google Scholar | PubMed |
Stolwyk RJ, Mihaljcic T, Wong DK, Chapman JE, Rogers JM (2021) Poststroke Cognitive Impairment Negatively Impacts Activity and Participation Outcomes: A Systematic Review and Meta-Analysis. Stroke 52, 748-760.
| Crossref | Google Scholar | PubMed |
Straus SE, Tetroe J, Graham I (2009) Defining knowledge translation. Canadian Medical Association Journal 181, 165-168.
| Crossref | Google Scholar | PubMed |
Stroke Foundation (2019) ‘Clinical Guidelines for Stroke Management.’ (Melbourne, Vic, Australia) Available https://informme.org.au/guidelines/living-clinical-guidelines-for-stroke-management# [accessed 25 August 2023]
Swiatek DP, Jewell VD (2020) Cognitive Evaluation: Is It Time to Require Standardized Assessments? Internet Journal of Allied Health Sciences and Practice 18, Article 12.
| Crossref | Google Scholar |
