Measuring care alignment in general practice consultations for people with long-term conditions: an exploratory study

KEYWORDS: Chronic illness; self-management; primary health care

The Chronic Care Model (CCM)¹ aims to make the care of people with long-term conditions (LTCs) planned, proactive, and patient-centred. Two instruments have been developed to assess how well care aligns with the CCM: the Assessment of Chronic Illness Care (ACIC),² designed to evaluate provider perspectives of care delivery, and the Patient Assessment of Chronic Illness Care (PACIC),³ developed to measure patient perspectives on their LTC care. To understand and evaluate the effect of practice change, a more comprehensive approach is to incorporate both patient and provider perspectives on LTC care delivery within the same practice setting.

A longitudinal study of disease management programmes in the Netherlands by Cramm and Nieboer⁴ and a cross-sectional study of primary care clinics in the US⁵ concluded that the ACIC and PACIC provide complementary views on chronic illness care, highlighting the need to take both patient and provider perspectives into account when evaluating care quality. Although the ACIC and PACIC have been developed to reflect CCM principles, they are unlikely to be directly comparable, as the ACIC is completed by a practice team, focuses on the systems of care delivery, and describes care provided to patients in general. Individual patients complete the PACIC, regarding the care they personally receive from a practitioner or team of practitioners. To make the two instruments comparable, the PACIC has been modified to assess practitioners’ responses to the same items.⁶ The exact the same questions are asked but from the separate perspectives of patient (PACIC) and practitioner (MPACIC).

Here, we explore perceptions of LTC care within the context of actual patient/practitioner dyads. This approach exemplifies ways the PACIC and MPACIC tools can be used; first, to assess how well care provision is guided by the CCM, second to examine concordance between perceptions of practitioners and patients regarding provision and receipt of care, and third to see if aspects of care provision vary according to the general practitioner (GP) and practice nurse (PN) roles. A case study is included to demonstrate the potential for using the two instruments in combination to understand LTC support for individual patients.

Patients and practitioners were recruited through advertisements in general practices and via Māori consultation networks. Inclusion criteria required patients to be enrolled in general practices and have at least two LTCs compromising their quality of life. Fourteen patients had previously been involved in a doctoral study with the nurse interviewer and 13 patients and their GPs and PNs volunteered, and provided written consent, to participate in the current additional research. Two patients nominated the same GP and PN, so the practitioner sample included 24 individuals: 12 GPs and 12 PNs. For this exploratory study, a relatively small convenience sample was considered appropriate.

The PACIC is a 20-item self-report measure designed for patients to rate specific activities or qualities of care that occur during their encounters with health practitioners. The items are descriptions of care behaviours; the frequency they are provided is rated on a five-point scale. Response options were ‘never/almost never’ (scored as 1), ‘generally not’ (2), ‘sometimes’ (3), ‘most of the time’ (4) and ‘almost always/always’ (5). Responses to the 20 items can be averaged to form a total score or combined to create subscales approximating five of the CCM domains: Patient Activation, Delivery System Design, Goal Setting, Problem Solving and Follow up. Reliability analyses have focused on internal consistency, although moderate test-retest reliability has also been established.^3,7 Validity evaluations have been primarily factorial,^8–12 but Rick et al. considered convergent validity to be reasonable.¹⁰

The MPACIC⁶ was used with the GPs and PNs to collect the same information from a provider perspective. The modification included changing the question stem from ‘when I received care for my chronic conditions, I was ...’ to ‘when I provide care for (patient) I ...’, and changing the wording of each item to reflect the provision of care.

Ethical approval was obtained from the Massey University Human Ethics Committee (MUHEC 14/19). Responses were written onto paper forms by a registered nurse interviewer during separate face-to-face interviews with patients and their GP and PN providers of LTC care. Patients responded to the PACIC items twice: once for their doctor and once for their nurse. The MPACIC was used with GPs and PNs to rate the care they provided for individual study patients. This matched sample technique has not previously been used and while it provides more focused information, it requires more time and effort to collect data. The process resulted in four sets of data per patient (see Box 1) in the form of dyads; a patient/GP dyad (P/GP) and a patient/PN dyad (P/PN), with some additional comments also being provided.

Descriptive analyses were conducted to provide ranges and mean and median scores for the patient and practitioner groups. To compare scores across groups, difference scores were calculated by subtracting practitioner scores from patient scores. A positive result reflects the patient score being higher than the practitioner score, suggesting that patients perceive the standard of care to be better/more frequently provided than their practitioners. Conversely, negative scores reflect the opposite, whereby patients perceive the standard of care to be poorer/less frequently provided than practitioners. The closer the difference score is to zero, the greater the degree of congruence. The same approach was taken in presenting a case study, but this was based on four sets of scores: ‘Mrs Q’s’ ratings of her PN; the PN’s ratings of the care provided to ‘Mrs Q’; ‘Mrs Q’s’ ratings of her GP; and the GP’s ratings of care provided to ‘Mrs Q’.

Within the patient group, five participants were male and eight female, seven were European and six were Māori. Age ranged from 29 to 78 years (mean (M) = 60.8 years). Participating patients had four to seven chronic conditions. Patients had been enrolled with their current general practice between 6 and 51 years (M = 20.2 years). The time they had been consulting their current GP ranged from 4 to 30 years (M = 14.5 years) and their PN from 1 to 28 years (M = 10.3 years).

The 12 GPs were aged 36–65 years (M = 52.1 years); eight were male and four were female. They had been GPs for 7–40 years (M = 24.4 years) and the length of time they had been in their current practice ranged from 7 to 37 years (M = 19.9 years). The 12 female PNs were aged 41–67 years (M = 54.9 years) and had been employed for 7 – 43 years (M = 28.4 years) and in their current practice for 1–28 years (M = 12.4 years).

Table 1 provides the range, mean, and median domain and total PACIC scores for patients’ ratings of care received from their GPs and PNs separately, and GPs’ and PNs’ ratings of care provided to individual patients. Patient mean scores generally suggest that care is received sometimes to most of the time, but goal setting occurred with slightly less frequency, on average. There was variation in the range and level of the patient scores across the five domains. ‘Goal Setting’ and ‘Follow up’ had a greater range than the others, and the ‘Delivery System Design’ and ‘Problem Solving’ subscales had the smallest range, with scores towards the upper end of the five-point scale. Comparison of patient and practitioner scores suggests that while they are fairly close; the GPs’ ratings are marginally higher and the PNs’ are marginally lower.

Table 1. Ranges, mean, median and difference scores for PACIC domain and total scale ratings

Further information about agreement is provided by difference scores, also provided in Table 1, calculated by subtracting the practitioners’ ratings from the patients’ ratings. Mean difference scores for the Patient/GP dyads ranged from –0.8 for ‘Goal Setting’ to 0.1 for ‘Delivery System Design’, and for Patient/PN dyads from –0.1 for ‘Patient Activation’ to 0.5 for ‘Delivery System Design’. For four domains, the patient/PN scores were more congruent than the patient/GP scores, the exception being the ‘Delivery System Design’ scores. The mean difference scores for the total scale (–0.2 for Patient/GP and 0.2 for Patient/PN dyads) suggest overall similarity between patient and practitioner scores, and indicates that the discrepancies are of similar magnitude for the two types of practitioner, albeit in opposite directions.

When differences were considered at the item level, 66% of Patient/GP and 62% of Patient/PN scores were the same or one response category apart, 19% of Patient/GP and 23% of Patient/PN were two categories apart, and 15.0% of both Patient/GP and Patient/PN scores were three to four response categories apart. In practice, it would be unreasonable to expect absolute agreement, so these figures suggest that there is fairly strong agreement between patient and both types of practitioner approximately two-thirds of the time.

However, there was considerable difference for 15% of scores, and this was most pronounced for ‘Goal Setting’ where there were three or four categories difference between patient scores and 26% of GP scores and 18% of PN scores. The following case study demonstrates the use of the PACIC and MPACIC at an individual patient level.

The mean domain and total ratings for Mrs Q and her two practitioners are provided in Figure 1.

Figure 1. Domain scores associated with Mrs Q and her providers

The scores suggest that Mrs Q’s ratings for the PN were consistently higher than those for the GP. The same applied at item level where the PN was scored as high as, or higher than, the GP across all items. However, the practitioners’ self-ratings were the opposite; the PN rating herself lower than the GP rated himself across the five domains. Also evident is that the patient’s ratings of GP care were lower than the GP’s ratings of care provision, and the patient’s ratings of PN care were higher than the PN’s ratings of the care provided. The highest possible mean score (5) was awarded by the patient to the PN for the ‘Patient Activation’ items; this was the lowest GP score at 1.7. Two of the ‘Patient Activation’ questions were about gathering information from the patient: the nurse commented that she did not need to ask as the patient volunteered, thus ‘Patient Activation’ items were scored as rarely happening.

With respect to the magnitude of score differences, for the 20 items, complete agreement was achieved on five occasions for the Patient/GP dyad and on three occasion for the Patient/PN dyad. Ratings were one category apart for seven Patient/GP comparisons and five Patient/PN comparisons, and two or more categories apart for eight Patient/GP ratings and 12 Patient/PN ratings. When the direction of the differences was considered, 11 P/GP comparisons were negative and four were positive, whereas for the P/PN comparisons, three were negative and 14 were positive. These differences suggest that Mrs Q perceived her GP to be providing support less often than the GP felt he was providing it in relation to more than half of the care examples. Conversely, she perceived the PN to be providing care more frequently than the PN rated her provision for almost three-quarters of the items.

As GPs and PNs should be working as a team in the delivery of LTC care, another approach is to take the highest score of the patient’s ratings of GPs and PNs to assess the care received overall. In this case, the score remains the same (4.2) as the nurse was rated so positively, but this would not always be the case.

Comments made during the data collection for Mrs Q highlighted the different roles of the doctor and nurse. For example, when asked about Goal Setting, the patient observed that she had this sort of discussion with the nurse, and the nurse indicated that giving treatment choices was part of the doctor’s role.

The PACIC assesses care for people with LTCs aligned with the CCM, so it is important for patients’ views to be part of the quality assessment of any interventions. However, tools are still being developed to establish whether there is agreement between patients and providers on the quality, frequency and desirability of care aspects. By matching patients with their actual primary care providers, we examined differences between their PACIC and MPACIC scores. In practice, there is likely to be some measurement error resulting in apparent difference in perceptions. Therefore, for this preliminary study, we accepted no or one scale point difference in scores to reflect congruity. On this basis, for approximately two-thirds of the scores, there was agreement between patients and providers on care frequency. A considerable difference, when scores were three or four scale points apart, was found for approximately 15% of score dyads.

The strongest agreement was found with respect to ‘Delivery System Design’, particularly for the P/GP data, and the least agreement could be seen with ‘Goal Setting’, again more distinctly for the P/GP dyad. With the exception of ‘Goal Setting’, differences in agreement levels between the two patient/practitioner groups were not particularly notable overall, but the direction of the total mean differences indicated that nurses rated care to be provided slightly less often than patients perceive it to occur and doctors slightly more. The reason for the GP versus PN difference in relation to the patient perspective is unclear, and it would be interesting to see if this finding was replicated in a larger sample. In a previous study,¹³ a comparison between patient and provider ratings in unmatched samples found the difference between PACIC and MPACIC scores to be considerably greater, with mean practitioner scores consistently higher than mean patient scores, and nurses’ ratings consistently higher than doctors’ ratings. The closer scores identified in the current study are reassuring, as they suggest that when actual individual patient/practitioner partnerships are assessed, the level of support appears to be more congruent than the earlier study would suggest.

Cramm and Nieboer¹⁴ describe the patient-centred approach as including joint decision-making based on patients’ preferences and achieved through open communication, co-operation and respect. They call this productive interaction and in a longitudinal study of more than 1000 people with LTCs found that it mediated the relationship between care quality and wellbeing. Their participants rated the quality of their relationships with practitioners more highly than they rated their communication, and the authors concluded that chronic care practitioners should be trained in relational as well as functional competencies; relational competence being the ability to see the perspectives of others, empathise with their situations and respect their needs and choices.^14,15

As they are relatively brief measures, specifically designed to assess LTC care in line with current best practice, the PACIC and MPACIC are well-suited for use in primary care to evaluate productive interactions within consultations. By focusing on patient/practitioner partnerships, there is scope to gauge perceptions of care provided for specific individuals and, by including other members of the general practice team, the added value of adopting a team approach to care delivery can be identified. This study has highlighted the importance of ensuring that whoever is answering, whether patient or practitioner, understands the intention of each question before answering.

From the ‘Mrs Q’ scenario, an observation of disparity in the patient and practice nurse ‘Patient Activation’ scores could be made when considering the accompanying comments. The three patient scores of 5 indicated that Mrs Q felt the nurse was asking for input into her treatment plan, giving choices about treatment and asking for information about medicine concerns. The nurse indicated that the patient volunteered information without being prompted, and that offering treatment choices was not part of the nursing role. Taken at face value, the low nurse’s scores could suggest that these important aspects of care were not provided. However, the nurse’s comments indicate an awareness of the need, but stated that, in this case, the issues were already being addressed by the patient herself or by another team member. Importantly, the patient was getting an opportunity for input, and perhaps the nurse did indeed engage the patient in discussions around treatment when it is defined more holistically. These conclusions support the need to access team members’ individual perspectives to make sense of the broader care picture and to encourage detailed rationales for scores to add insight into behaviour.

The sample size for this study limited the use of aggregated data, and we can make no claims that the results reflect a broader group. However, the participants provided sufficient information to demonstrate the potential for using the PACIC and MPACIC in combination in general practice to examine LTC care from different perspectives. There is scope not only to compare patient/practitioner ratings and identify areas of perceived deficit or apparent misunderstanding, but also to use this information to alter practice and monitor progress over time. Face-to-face administration can result in the collection of full datasets and can provide patients (and practitioners) with opportunities to clarify their understanding of the questions, thus adding to the validity of their responses. The research nurse collecting the data was already known to the samples through previous engagement with them. Using these tools within a standard consultation scenario may influence patients’ openness, but if the purpose is clearly and sensitively explained, they should see the value and potential for all parties to benefit from frank discussion.