Risk stratification of New Zealand general practice patients for emergency admissions in the next year: adapting the PEONY model for use in New Zealand

KEYWORDS: General practice; emergency hospital admissions; risk stratification

New Zealand, like many other Organization for Economic Cooperation and Development (OECD) countries, is considering health system policy and new initiatives involving integrated health care to tackle the problem of an ageing patient population with long-term health conditions.^1–7 New Zealand’s population aged > 45 years grew by 29.2% from 1.30 million to 1.68 million over the 10-year period from 2001 to 2010,⁸ and during this time, the number of emergency admissions to New Zealand public hospitals increased by more than 30%.⁹ This growth in demand for unplanned hospital care has an effect on the ability of hospital services to reduce waiting lists for surgery and other arranged admissions.

Acute exacerbations and complications arising from chronic disease often require hospital treatment and it is likely that this is at least partially responsible for the increase in emergency hospital admissions. Many emergency admissions may be avoidable; there is evidence that coordinated patient care programmes delivered in the community to patients at high risk of hospitalisation can reduce the need for unplanned hospital treatment. Such initiatives may include primary care interventions,^10,11 better clinical pathways and co-ordination between primary and secondary care,^12,13 co-ordination of primary and social care in the community,¹⁴ case management for individual diseases,¹⁵ and patient education aimed at improving self-management.^16,17 A pragmatic approach when establishing programmes aiming to improve patient care and health outcomes is to identify patients for whom the greatest benefits may be achieved. Risk stratification of general practice patients with regard to their likelihood of future emergency admissions provides one method for achieving this.

Predictive statistical models are widely used for risk stratifying patients, with models using both primary and secondary care data tending to perform better than models based solely on hospital records.^18–26,5 Our aim was to assess the performance of the Predicting Emergency Admissions Over the Next Year (PEONY) model when applied to New Zealand general practice patients, and to improve on its predictive power, where possible, to provide risk stratification of patients aged ≥ 40 years for emergency admissions in the next year. Originally developed in Scotland, the PEONY model algorithm estimates risk for future admissions based on each patient’s hospital utilisation and use of prescribed medicines in the previous three years.²³ We assessed this model due to its excellent performance in risk stratifying Scottish patients, and because its predictor variables were quantifiable from data recorded in New Zealand’s national databases of hospital admissions, dispensed prescription medicines, and practice patient registers.

We analysed 2008–2011 data from four national data collections to calculate patient risk at 1 January 2011 for a subsequent emergency admission in that year. The study population was drawn from the Primary Health Organisation Enrolment Collection of patients registered in New Zealand general practices in the first quarter of 2011. Approximately 4.19 million New Zealanders (95.3% of an estimated total population of 4.39 million)⁸ were registered with a practice in 2011. Data available included patient date of birth, sex, prioritised ethnic group,²⁷ month of last consultation, and small area deprivation index score derived from the NZDep2006 census-based index of deprivation.²⁸ Ethnicity and deprivation status were assigned as unknown where not recorded.

The National Minimum Dataset for Hospital Events provided records of all admissions to public hospitals in New Zealand from 2008 to 2011. Data included admission and discharge dates, admission type (acute, arranged, or waiting list), and the principal diagnosis (International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Australian Modification; ICD-10-a.m.). Length of stay was calculated as the number of days between admission and discharge dates.

Information on patients’ medicine use was collated from the Pharmaceutical Collection containing records of subsidised medicines dispensed from New Zealand community pharmacies. For each patient, we quantified the number of medicines dispensed from 2008 to 2010 in the following therapeutic groups: antihypertensives, diuretics, nitrates, anticoagulants, antiplatelets, gastrointestinal, respiratory, hypnotics, anxiolytics, antipsychotics, antidepressants, analgesics, antiparkinson agents, antibacterials, antiosteoporotics, antianaemics and diabetes medicines. Many of these medicines provide treatment for long-term conditions.

The Mortality Collection provided records of deaths occurring from 2008 to 2011. Information in the four national datasets relating to each patient was linked using their encrypted National Health Index (NHI) code.

The modelling cohort included patients aged ≥ 40 years as at 1 January 2011 who were registered at a general practice in all four years, had a general practice consultation in both 2008 and 2011, and had not died between 2008 and 2011. Although this excluded patients who died in the follow-up year, this criterion was consistent with the original PEONY model, which excluded patients with less than one year of follow-up data.²³ We calculated the proportion of patients with and without an emergency admission in 2011 by patient demographic group, by drug therapeutic group, and for patients with and without emergency admissions from 2008 to 2010. The number of emergency admissions and total bed days for all admissions for each patient from 2008 to 2010 were also summed.

The patient cohort was randomly split into two equal halves, with one half used for the New Zealand model’s derivation and the other used as a validation dataset to assess the model’s performance. Regression coefficients were estimated using binomial logistic regression, with any emergency admission in 2011 as the patient outcome. A risk was then estimated for each patient in the derivation and validation datasets from the regression coefficients of all variables in the derived model. Medicine use from 2008 to 2010 in each of the 16 therapeutic groups (excluding diabetes medicines, hypnotics and anxiolytics) was modelled as a binary variable. The number of prescriptions for respiratory drugs, analgesics, antibacterials, hypnotics and anxiolytics, and diabetes medicines were modelled as continuous variables. Eight interaction terms included in the Scottish model were retained (see footnote to Table 1) and patients’ ethnicity was included as a new variable.

Table 1. Patients with and without an emergency admission in the follow-up year (2011) for model variables with adjusted odds ratios from the final model dataset*

Model performance was assessed by measuring the positive predictive value (PPV), sensitivity and specificity at incremental risk thresholds, and by calculating the area under the receiver operating curve (the c statistic) for patients in the validation dataset. We also calculated c statistics for patients registered at practices within each district health board (DHB) area of New Zealand to determine the consistency of the model’s performance across geographic regions. To explore the influence of each model variable on estimated patient risk, we calculated the proportion of patients by demographic group, drug therapeutic group and hospital use in the previous three years for four patient risk groups; ≥ 70%, 50–69%, 30–49% and < 30% probability of an emergency admission in 2011. We also examined the range of principal hospital diagnoses for emergency admissions in 2011 for patients with a high estimated risk (≥ 50%).

A total of 1,928,266 patients aged ≥ 40 years were listed on the general practice registers. Of these, 179,731 patients (9.3%) were excluded as they were not registered in all four years, with a further 334,612 (17.4%) excluded due either to no record of visiting a practice in 2008 and 2011 or inconsistent recording of date of birth and sex. A further 4417 patients (0.2%) died during 2011, leaving a total of 1,409,506 patients (73.1%) for inclusion in the model. Ethnic group was unknown for 8164 patients (0.6%) and deprivation decile for 96,452 patients (6.8%).

Table 1 shows the proportion of patients with and without an emergency admission in 2011 for model variables and the adjusted odds ratios for this outcome from the model dataset. A total of 154,892 patients (11.0%) were admitted in 2011. The odds of an emergency admission in 2011 for patients with an emergency admission in the previous three years were over threefold higher than that for patients with no previous emergency admissions. Mean total of bed days from 2008 to 2010 was four times higher for patients with a subsequent emergency admission. Patients of Māori and Pacific Island ethnicity were more likely to have an emergency admission in 2011 than patients from other ethnic groups, and patients living in the most deprived areas were more likely to be admitted than patients in less deprived areas. Use of medicines in all 16 therapeutic groups was more prevalent in patients with an emergency admission in the following year. All variables in the original PEONY model, with the exception of medicines used for respiratory conditions, were significant in the New Zealand model.

Performance indicators for the final model applied to the validation dataset are shown in Table 2 for descending risk thresholds. The following results are from the validation dataset, unless stated otherwise. The PPV for patients with an estimated risk ≥ 50% for an emergency admission in 2011 was 58.2%, indicating that almost 6 out of 10 of these high-risk patients subsequently went on to have an emergency admission. Nearly 7 out of 10 patients identified as being at very high risk (≥ 70%) had a subsequent emergency admission. The c statistic for the model was 72%, representing the probability that a randomly selected patient with an emergency admission in 2011 had a greater estimated prior risk than a randomly selected patient with no emergency admission. The c statistic for the derivation dataset was marginally less at 71%. Performance of the model was consistent when applied to patients in each of the 20 DHB regions, with c statistics ranging from 71.1 to 72.8. Although different regions had different patient profiles in terms of age, sex, ethnicity, and deprivation, these variables were included in the risk-adjusted model.

Table 2. Model performance for the validation dataset (n = 704,753) by predicted risk group for an emergency admission in the next year

Figure 1 shows the association between predicted risk for an emergency admission in the next year and use of medicines in multiple therapeutic groups. Mean predicted risk increased with the number of different medicine groups used by patients, after controlling for other variables in the model. The corresponding increase in the proportion of patients who subsequently had emergency admissions indicates that the PPV of the model increases with increasing patient risk.

Figure 1. Predicted risk for an emergency admission in the next year for the validation dataset by number of medicine groups

Table 3 shows the influence of each model variable on predicted patient risk. Although more than 95% of patients with risk ≥ 30% had emergency admissions in the preceding three years, the mean number of emergency admissions and associated bed days in hospital for patients with a risk higher than 70% was twice that of patients with a 50–69% risk and fourfold that of patients with a 30–49% risk. This reflects the influence of previous emergency admissions on estimated risk for future admissions. Over 16% of patients with risk ≥ 50% were of Māori ethnicity, compared to 7.9% of patients with risk < 30%. The mean age for patients with risk ≥ 30% was 75 years compared with 59 years for patients at lower risk.

Table 3. Patient demography, hospital and medicine use in the previous 3 years for the validation dataset by predicted risk of an emergency admission in the next year

With the exception of patients using respiratory medicines, analgesics and antibacterials, the proportion of patients with risk ≥ 30% using medicines in all drug groups was more than twice as great as patients with risk < 30%. For diuretics, nitrates, anticoagulants, antipsychotics, antianaemics, antiosteoporotics and antiparkinson medicines, the proportion was more than fourfold greater. Table 3 highlights the differences in demographic characteristics, medicine use and hospital use between patients at high and low risk for future emergency admissions.

Table 4 lists the most common reasons for high-risk patients having emergency admissions in 2011. This patient group may be considered the most suitable target group for initiatives aimed at reducing future emergency admissions. Chronic obstructive pulmonary disease, heart disease and symptoms, and warning signs relating to circulatory and respiratory disease were the most frequent reasons for admission. There were also high rates of admission for pneumonia, abdominal and intestinal problems, and complications arising from previous surgical and medical care.

Table 4. Reasons for emergency admission in the follow-up year for validation dataset patients with a predicted high risk (≥ 50%) of emergency admission (n = 7902)

Our results indicate that the PEONY model performs well when adapted and applied to New Zealand patients. Our aim was to identify a methodology for determining which patients were most at risk of future emergency admissions, and to inform potential interventions aimed at curbing growth in demand for emergency hospital care. The model provided risk stratification for 73% of all general practice patients aged ≥ 40 years in New Zealand in 2011. To our knowledge, this represents the first research to estimate individual patient risk for future emergency admissions at the national level.

The developers of the original PEONY model concluded that its superior performance over previous models quantifying individual patient risk for future admissions was probably due to the inclusion of community prescribing measures as indicators of chronic disease and other conditions.²³ Our findings confirm the significance of including medicine use within the 16 therapeutic groups in the model, in addition to each patient’s historical use of hospital services.

The c statistic for our model (0.72) was lower than that for the Scottish PEONY model (0.80), largely due to its lower sensitivity in high-risk groups; a smaller proportion of patients with an emergency admission in the next year were correctly identified as high risk. At the 50% risk threshold, sensitivity was 5.9% for our model but just under 7.9% for the Scottish model. However, PPVs were similar at this threshold level (58.2% for our model and 59.0% at the 49% risk threshold for Scottish patients), indicating that a similar proportion of patients identified as high risk were admitted in the next year. It has been argued that traditional measures of performance, like sensitivity, mask the real value of models in targeting preventive intervetions.²⁵ Our rationale for developing a New Zealand model was not to identify every patient with emergency admissions in the next year, but to identify those at highest risk for future admissions.

We acknowledge certain limitations in our study. First, 514,343 New Zealand patients (26.7%) aged ≥ 40 years were excluded from the model; they were not registered with a general practice in all four years or had no record of visiting a practice in 2008 and 2011, or there was inconsistent recording of date of birth and sex. Our intention was to include only patients with evidence of New Zealand residency in all four years so that examining hospital admissions and medicine use was possible in all study years. The exclusion of patients dying in the follow-up year will have misrepresented use of the model for risk prediction in the real world; we applied this criterion to be consistent with the derivation of the original PEONY model. Ethnicity and/or deprivation status were also unknown for 103,661 patients (7.4%), and these missing demographic data will detract from the model’s accuracy and performance.

While our results indicate that high-risk patients are most likely to be elderly and with multiple chronic conditions, among patients with risk ≥ 30%, 27.5% of all emergency admissions from 2008 to 2011 were for patients aged < 65 years and 12.6% for patients aged < 55 years. Furthermore, 21.0% of all patients with risk ≥ 50% were aged < 65 years. Thus, a significant proportion of high-risk patients were middle-aged and an important patient group for health service consideration. Measures targeted at reducing emergency admissions in this younger cohort may help reduce demand for emergency services in later years.

Our adaptation of the PEONY model provides New Zealand with a regionally consistent algorithm for identifying people at risk of hospital admission. Recalculating the model coefficients annually and making patient risk scores available to primary care teams would identify patients at risk of hospitalisation. Providing highly targeted care to these patients by improved coordination of primary care providers may be critically important in averting hospital admission.²⁹ A case management programme to address risks could be used to provide a patient-centred model for reducing hospital admissions.¹

Interpractice admission rate variation when standardised for age, gender, ethnicity and deprivation is often explained by differences in levels of morbidity or different models of care. The PEONY model may be an effective proxy measure of morbidity and partially explain admission variation between general practices. This could assist in identifying outlying practices with admission rates that likely represent different models of care; this in turn may provide opportunities and directions for general practice reconfiguration to reduce hospital admissions. The model may also be used to estimate future demand and costs for emergency admissions because changing demand from year to year as the population ages and the prevalence of chronic conditions grows will be taken into account when recalibrating model coefficients with each successive year.

The New Zealand PEONY model represents an efficient methodology for providing a national risk prediction tool that is applicable to all regions of New Zealand. Elderly patients with multiple chronic conditions constitute the majority of high-risk patients for an emergency admission in the next year, but there are also a considerable number of younger patients at high risk.

This research was conducted as a component of the usual employment of the authors by BPAC^NZ and the University of Otago.

The authors are aware of no conflicting interests with regard to this research.