Guidelines, training and quality assurance: influence on general practitioner MRI referral quality

KEYWORDS: General practitioner; magnetic resonance imaging; direct access; guidelines; training; quality assurance

Musculoskeletal disorders comprise at least 25% of New Zealand’s annual health spending,¹ paralleling international data,^2–5 and resulting in significant morbidity and social costs^1,6,7 for approximately one-in-four New Zealand adults.¹ Musculoskeletal disorder management often occurs in primary care, but in some cases, patients require additional high-tech imaging that is accessible only by secondary care referral, creating barriers to definitive care. To evaluate service delivery effect, invited general practitioners (GPs) working in one region participated in a study altering usual treatment pathways, allowing trained GPs to refer patients directly to publicly funded magnetic resonance imaging (MRI).

Limited evidence exists of the diagnostic effectiveness of direct GP MRI referrals. Several studies indicate MRI requests from GPs and other non-orthopaedic specialists lack diagnostic relevance.^8–10 Lehnert and Bree⁹ suggest that unsuitable MRI and computed tomography (CT) referrals occur because of medical liability fear, economic motivation, regional differences, patient demand and inadequate doctor training or experience. They found that individuals with inappropriate imaging referrals were 3.5-fold more likely to have negative results than patients with appropriate referrals.⁹ Much of the available evidence surrounding direct MRI referral pathways recommends use of guidelines, training or both to facilitate correct referral practices.^10–12

MRI is an established secondary care tool for diagnosing selected musculoskeletal conditions. Over the past 20 years, as part of a drive to manage patient journeys more efficiently and to improve use of health system resources, several countries have trialled direct access MRI referrals.^8,10,13,14 This paper reports on the New Zealand experience of direct access MRI, specifically focusing on whether pre-training and quality assurance frameworks can lead to appropriate GP access to deliver suitable patient management outcomes. Given uncertainty around introducing earlier GP access to funded MRI, the primary aim of this study was to examine whether a decision support framework of guidelines, training and quality assurance resulted in clinically appropriate use of MRI. Further, this study assessed post-scan patient management as a consequence of MRI use in primary care.

An observational study was conducted in ProCare, a primary health organisation,¹⁵ with 825,052 enrolled patients.¹⁶ GP participation initially began as a 38-practice pilot study and gradually expanded to 72 practices, almost 25% of all primary care clinics in urban Auckland.

Guidelines were based on a modified collective version of Accident Compensation Corporation (ACC), New Zealand and Department of Health, New South Wales, Australia diagnostic imaging and musculoskeletal management guidelines.^17,18 These were adjusted to be general practice-centric, supported with expert and sector review. Guidelines focused on four body sites: knee, shoulder, cervical and lumbar spine.

GPs were invited to participate in a musculoskeletal training and MRI referral accreditation programme. GP inclusion began after attending two separate 2-h musculoskeletal assessment education sessions accredited by ProCare’s Clinical Quality and Education Committee. Education consisted of sessions delivered by a clinical lead (S. Kara) and small group ‘hands-on’ primary care-centric clinical examinations focusing on the relevant clinical indications (Table 1). A ratio of one educator to six GPs ensured group participation. Educators were experienced sports medicine practitioners and musculoskeletal physiotherapists.

On average, 25 GPs attended each clinical education session. GPs were continually entering the training programme in tranches throughout this study. Ninety-six percent of practices supported sending their GPs for annual re-training. Re-training consisted of initial training material further contextualized from learnings through the study period. Ongoing annual accreditation is currently expected, but may alter in content delivery.

Following education sessions, GPs referred consenting patients for MRI, or followed usual care practice. A single radiology provider conducted all MRIs. Radiologist reports detailing MRI findings and clinical recommendations based on these findings were electronically sent to referrers. Radiologists were available to discuss reports and to upskill GPs on body site-specific findings.

GP MRI referral data and subsequent patient management outcomes were collected. The clinical lead reviewed all data for quality assurance purposes. Data were received from ProCare and radiology provider databases. Cases were individually analysed, with imaging results and clinical referral diagnoses matched against guidelines compliance. Individual case data included age, sex, GP clinical notes, clinical indications for referral and MRI findings. Data were graded as shown in Table 2. Referrals not reviewed (grading system V) included instances where patients did not attend appointments or were non-contactable (n = 5), had scans completed elsewhere (n = 8), were declined ACC cover (n = 1), had imaging contraindications (n = 4) or missing scan reports (n = 5). MRI requests that were inconsistent with guidelines (III and IV) were discussed with the referring GP, facilitating GP accountability and closing quality assurance feedback loops. Patient management post-MRI was assessed by reviewing clinical notes (Table 3) to evaluate the direct access pathway effect on subsequent treatment.

Following collection of 550 MRIs, a second reviewer from a different region independently assessed all data for referral alignment with guidelines. This clinician and the clinical lead each had 20 years’ experience in primary care and musculoskeletal and sports medicine. Reviewers had access to all clinical information and radiology reports, but were blinded to each other’s assessment findings.

Consistency of reviewer interpretation was evaluated by calculating overall reviewer agreement in referrals for three groups: grades I–II, grade III and grade IV. Cohen’s weighted kappa statistic¹⁹ was calculated, using quadratic weights to allow a disagreement of grade I–II vs. grade III to be given partial credit (weight = 0.75) compared to the more discordant combination of grade I–II vs grade IV (weight = 0). Referrals not reviewed (grade V) were excluded from the agreement analysis. Differences in type of referral according to body site were assessed using Fisher’s exact test. The odds of patient management outcomes according to scan status were derived using logistic regression. Analyses were conducted in Stata, version 15 (StataCorp, College Station, TX, USA).

Approval for an evaluation of the high-tech imaging pathway was obtained from the ACC Ethics Committee on 8 March 2017 (no approval number).

Overall, 273 GPs from 72 practices attended training sessions and received MRI referral accreditation. Over an 18-month period (Feb 2017–July 2018), 150 GPs requested 550 MRIs of four body sites (Table 1), averaging 30 MRIs per month from all eligible GPs. These GPs comprised 55% of accredited GPs, and they requested between one and 16 MRIs (mean 3.7), with 88% percent of MRI requests for knee or lumbar spine pathology (Table 4). A total of 23 requests fitted under grade V and were not analysed. Monitoring did not indicate that later-trained GPs delivered different quality clinical outcomes than earlier-trained GPs.

Between reviewers, on average, 79% of reviewed scans (Table 4) were considered consistent with guidelines or as recommended by musculoskeletal medicine specialists (grade I and II). A further 7% of MRI requests were clinically indicated, but fell outside guidelines (grade III) and 14% percent of reviewed MRI requests were considered unnecessary. Overall agreement between reviewers was 75%. Cohen’s weighted kappa statistic was 0.38 (95% CI: 0.28–0.48, P < 0.001), indicating weak-to-moderate inter-rater agreement; with a proportion of consistent ratings of 0.7, there was 90% power to detect kappa ≥0.4.²⁰ Kappa measures the amount of agreement beyond that expected by chance, so one reason kappa appears low is due to the proportion of expected agreement and overall prevalence of consistent ratings both being high.²⁰

For reviewers, MRI request alignment with guidelines differed by body site (reviewer 1, P = 0.04; reviewer 2, P < 0.001). There was less agreement between reviewers for shoulder imaging requests (Table 2), reflecting the diagnostic complexity shoulders pose.^21,22 Most grade IV shoulder scans were inside the required time for guideline compliance (<6 months). Irrespective of shoulder referral grading, 75% resulted in specialist intervention.

Analysis of patient management outcomes post-MRI (Fig. 1) showed 78% of all MRI requests resulted in specialist referral; 49% benefiting from specialist intervention by surgical management (outcome A) or non-surgical intervention (outcome B), with 29% advised rehabilitation (outcome C). The remaining patients did not receive specialist review; their GPs instead prescribed active rehabilitation. Data for 27 (5%) patient management outcomes could not be found.

Further sub-group analysis by body site (Fig. 1) showed 76% of shoulder referrals resulted in specialist intervention. Lumbar spine and knee referrals resulted in lower but similar rates of specialist intervention (outcomes A or B).

Patients with scans that were consistent with guidelines, specialist recommended or otherwise clinically indicated (grades I–III) were more likely to receive subsequent specialist interventional management, according to reviewers (P < 0.001; Table 5). MRI requests consistent with guidelines or recommended by specialists were between 2.64- (95% CI 1.51–4.62) and 4.44- (95% CI 2.47–7.99) fold more likely to lead to specialist intervention (outcomes A or B) than grade IV scans. MRI referrals that were inconsistent with guidelines but clinically indicated (grade III) were also more likely to lead to specialist services, although this effect reached statistical significance for only one reviewer.

Conversely, the odds of patients not receiving specialist referral, but instead being managed by active rehabilitation alone (outcome D) were significantly higher (between 2.70 (95% CI 1.56–4.65) and 3.39 (95% CI 2.02–5.69)) where GP MRI referrals were not clinically indicated and inconsistent with referral guidelines (P < 0.05 for all associations according to both reviewers; Table 5).

During the study period, 55% of trained eligible GPs requested MRI scans. On average, 79% of MRI requests were consistent with guidelines, increasing to 86% if all clinically indicated scans were included. Compliance with imaging guidelines increased the likelihood of specialist intervention for patients by at least 2.5-fold, compared to patients who did not meet the clinical criteria for MRI scanning. Although inter-rater reliability was weak to moderate, outcomes supported use of the guidelines independent of reviewers.

This New Zealand study reports on the effect on patient care of expanding MRI access to GPs. The patient population is representative of the Auckland region of 1.66 million, with eligible GPs’ enrolled patients making up 36% of the region’s total population. Study data were gathered over 18 months, giving GPs time to learn from the training provided and enabling implementation monitoring. Use of real-world practice settings with accurate measurement of post-MRI outcomes is a strength in identifying potential benefits.

Limiting factors include the lack of a control group and the unknown size of the eligible pathway patient cohort (denominator). Patients may not have been referred for MRI post clinical assessment because of appropriate use of the pathway; collecting data on this was not feasible. Consequently, the study group may not be representative of all MRI eligible patients. However, the study’s aim was to assess appropriate use of MRI, rather than uptake against population baselines.

GP participants had varied backgrounds, educations and ages. Only 55% of accredited GPs requested MRIs, possibly indicating differences in decision-making between GPs who did and did not request MRIs. This could reflect improvements in GP confidence post-training programme participation, desire to use these pathways, or differences in practice demographics.

Generalisability of this service delivery model to other New Zealand areas has not been tested, and we did not monitor GP MRI referral decisions, either by human or artificial intelligence. Evaluating how GP decision-making changes over time could establish whether the effectiveness of GP MRI referrals could be attributed to the Hawthorne Effect.²³ Direct access MRI cost-effectiveness was not considered.

There is limited agreement in existing literature on the benefit of GP direct access MRI pathways. The DAMASK (Direct Access to Magnetic Resonance Imaging: Assessment for Suspect Knees) trial assessed the effectiveness of GP referral to early MRI and a provisional orthopaedic appointment, compared with referral to an orthopaedic specialist without prior MRI for patients with continuing knee problems over a 2-year period, for 553 patients recruited from 163 general practices.¹⁴ The authors suggest increased GP diagnostic and therapeutic confidence because of imaging access. However, they state access to MRI did not influence GP decisions for specialist referral post-imaging, based on the fact that only 10% of patients having pre-booked specialist appointments were cancelled by their GPs; this was used as a surrogate marker to support no change in GP referral patterns. However, this did result in an effective use of public resources through providing a small, but significant, improvement in patient quality-of-life measures. In a retrospective chart review, Roberts et al.²⁴ found specialist-generated MRI resulted in more appropriate interventions for symptomatic patients and more surgically amenable pathology compared to GPs. Debaters against GP direct access MRI pathways often identify situations where GPs lack dedicated pre-use training and quality assurance, and operate within an unlimited condition open access environment, factors which are critical to success.^10,24 These studies contrast with findings from The Netherlands, indicating changes in GP referral patterns following MRI referrals.²⁵ Roberts et al.²⁴ admit to the value of an educational primer as testament to effectiveness of careful clinical evaluation and prudent MRI use. Other studies support educational interventions and patient pre-selection as means of preventing increases in GP referral rates.^9,12,26

Debate exists regarding unaltered patient outcomes following GP MRI use. A large Netherlands-based multi-centre, non-inferiority randomized control trial (TACKLE Trial = TraumAtic Complaints of the Knee — Leiden University Medical Center (LUMC) and Erasmus MC Trial) provides new evidence that GP MRI referrals (356 patients aged 18–45 years) for traumatic knee complaints following clinical pathways resulted in neither worse nor better outcomes than usual care for quality-of-life knee-related daily function during a 1-year follow up.²⁷ Our study shows decisions for specialist referral and appropriate intervention are improved when guidelines were consistently followed. The TACKLE Trial could be seen as a difference in management of operative versus non-operative care, rather than the role of MRI in outcomes. Their inclusion criteria of generalised knee complaint due to trauma or sudden onset adds weight to the use of specific condition clinical guidelines, such as those used in our study. Kisser et al.¹² and the TACKLE Trial team²⁷ both emphasise the importance of radiologist involvement in decision-making; something that our trial encouraged and was well-received by GPs.

Study results show that when GPs participate in training and re-training programmes following established evidence-based guidelines, quality utilisation of diagnostic imaging can be achieved. International direction and New Zealand health policy emphasise the value of delivering services in primary care, reducing secondary care reliance and enabling GPs to work at ‘top of scope’. This study demonstrates an opportunity to improve use of primary care services and expand GP capabilities. Findings show that where GPs refer patients for MRI in alignment with clinical guidelines, more appropriate use of specialist services occur. GPs can be confident that should conditions fall outside guidelines, patient rehabilitation is likely without MRI. Education and guidelines use inside a quality assurance framework present an opportunity to deliver continued evidence-based practice. Testing this on a larger scale may be the next step.

Only 55% of eligible GPs referred patients for MRIs. Reasons for the remainder not using this referral pathway were not explored, but are the subject of future research. Anecdotal feedback suggests clinical guidelines were followed, with MRI not required for diagnosis.

These results suggest clinical training, quality assurance procedures and subsequent consistent GP guideline use led to appropriate direct specialist referral. Without discouraging specialist opinion for clinicians when diagnostic uncertainty exists, use of timely imaging for selected conditions before secondary care review can be supported with guideline use and monitoring.

The authors declare no competing interests.