Reporting of Aboriginal and Torres Strait Islander peoples on the NSW Admitted Patient Data Collection: the 2010 Data Quality Survey

Improving the health of Aboriginal and Torres Strait Islander peoples is a priority under the National Partnership Agreement on Closing the Gap on Indigenous Health Outcomes.¹ This Agreement aims to reduce the disadvantage experienced by Aboriginal and Torres Strait Islander peoples with respect to life expectancy, child mortality, access to early childhood education, educational achievement and employment outcomes.¹ The correct reporting of Aboriginal and Torres Strait Islander peoples on health data collections is essential to measure the effectiveness of policies and programs at reducing the health disadvantage that they experience.

The National best practice guidelines for collecting Indigenous status in health data sets note that there are continuing problems with the under-reporting of Aboriginal and Torres Strait peoples on many health-related data collections, and encourages states and territories to ‘establish mechanisms for monitoring, improving and maintaining the quality of Aboriginal and Torres Strait Islander data…’.²

A national survey coordinated by the Australian Institute of Health and Welfare (AIHW) estimated the level of correct reporting of Aboriginal and Torres Strait Islander peoples on New South Wales (NSW) public hospital admitted patient data in 2007 to be 88%.The level of correct reporting ranged from 48% to 97% across Australian states and territories.³

Inpatient information collected by hospitals in NSW is compiled into the NSW Admitted Patient Data Collection (APDC). The APDC covers demographic and episode-related data for every inpatient separated from any public and repatriation hospital, and public same day procedure centre. Similar data are collected on patients admitted to private hospitals. Separation can result from discharge, transfer, death or change in care type. The APDC is maintained by the Data Collections and Reporting Unit in the Demand and Performance Evaluation Branch of the NSW Ministry of Health. The APDC is a major source of information on indicators of serious morbidity in the population.

To monitor the quality of reporting of Aboriginal and Torres Strait Islander peoples in the APDC, we carried out an audit using survey methods, estimated the level of correct reporting in 2010 and compared the results with those of the 2007 survey conducted by the AIHW.

This study used the methods of the national survey conducted by the AIHW in 2007. The methods have been described in detail elsewhere,² however a summary is provided here.

As for the 2007 survey, the aim of the 2010 survey was to achieve a sample size of approximately 2800 people drawn from NSW public hospitals. Hospitals were stratified by their geographic remoteness using the Accessibility/Remoteness Index of Australia (ARIA+) categories.⁴ A proportion of the total sample of people was assigned to each geographic area based on the proportion of the NSW Aboriginal and Torres Strait Islander population resident in each remoteness area. Within each ARIA category, a purposive sample of public hospitals was selected, based on high throughput and/or a relatively high number of separations of Aboriginal or Torres Strait Islander people. Within each hospital, people were selected from medical and surgical wards and renal units. Certain groups of patients were excluded, such as those: aged less than 18 years, too ill to speak, in intensive care or where clinical staff advised at the time that it was not appropriate to interview a patient. To allow comparison, hospitals selected for the 2007 survey were selected again in 2010. Three hospitals that were sampled in 2007 did not participate, reducing the number of participating hospitals from 20 to 17. Of the 17 hospitals sampled: five were selected from 61 hospitals located in major cities, six were selected from 71 hospitals in inner-regional areas, two were selected from 117 hospitals in outer-regional areas, and four were selected from 29 hospitals in remote and very remote areas.

Patients were interviewed in June 2010 by hospital staff members who, where possible, were Aboriginal and Torres Strait Islander liaison officers and not involved in maintaining personal information on the hospital Patient Administration System. The questionnaire included a script for each question so that questions were asked in a standard way and order. Patients were eligible to be interviewed if they were aged 18 years or over and well enough to be interviewed when the interviewer was present. Patients were provided with an information sheet about the survey and informed consent was obtained. Patients were asked a series of questions concerning their sex, place of birth in Australia (yes/no), date of birth, whether they were Aboriginal or Torres Strait Islander and usual place of residence. Information on whether each survey participant was reported as Aboriginal or Torres Strait Islander was also obtained from the hospital Patient Administration System. De-identified data were provided to the Ministry of Health for analysis.

The analysis followed the same method used for the 2007 survey. The proportion of Aboriginal or Torres Strait Islander people correctly reported on the Patient Administration System was calculated for each hospital stratum, and correction factors derived. These correction factors were then applied to the number of reported separations for Aboriginal and Torres Strait Islander peoples for each hospital remoteness category for June 2010, to give a number of expected separations for Aboriginal and Torres Strait Islander peoples for each remoteness category, which were then summed to give the total number of expected separations for Aboriginal and Torres Strait Islander peoples for NSW. The numbers of observed and expected separations for Aboriginal and Torres Strait Islander peoples for NSW were then compared to give the estimated proportion of Aboriginal and Torres Strait Islander peoples in NSW who were correctly reported on the Patient Administration System. Binomial confidence intervals for the remoteness category and total state level completeness were calculated.

Trends in monthly counts of reported separations among Aboriginal and Torres Strait Islander peoples for survey hospitals were obtained from the APDC and aggregated to the remoteness level of the hospitals.

As this review is an audit of a data collection and conforms to the standards established by the National Health and Medical Research Council for ethical quality review,⁵ ethics committee approval was not sought.

Of the 2581 patients who were asked to participate, 61 (2.4%) declined and a further 58 (2.2%) records were excluded as the patients were interviewed twice. Of the 2462 (95.4%) valid survey responses, 136 (5.5%) were among people who identified themselves as Aboriginal and Torres Strait Islander (Table 1).

Table 1.  Valid survey responses and Aboriginal and Torres Strait Islander peoples reported by Accessibility/Remoteness Index of Australia (ARIA+) remoteness category of hospital, NSW, 2010

The percentage of Aboriginal and Torres Strait Islander peoples correctly reported on the Patient Administration System in 2010 was estimated to be 91% (Table 2), compared to 88% (95% CI 84%–93%) reported in the 2007 survey. This improvement in reporting was not statistically significant. The percentage of correct reports increased with increasing geographic remoteness of the hospital, from 83% for hospitals in major cities to 100% for hospitals in remote and very remote areas. This pattern is similar to that reported for six jurisdictions combined in the 2007 survey.

Table 2.  Correct reporting of Aboriginal and Torres Strait Islander peoples on hospital admission records by Accessibility/Remoteness Index of Australia (ARIA+) remoteness category of hospital, NSW, 2010 and Australia, 2007

There was no noticeable increase in the number of reported separations among Aboriginal and Torres Strait Islander peoples at the time of the survey in June 2010 compared to previous months (Figure 1).

Figure 1.  Separations from selected hospitals for Aboriginal and Torres Strait Islander peoples aged 18 years and over by month and location of hospital, NSW, July 2006–June 2010.
The series comprises facilities sampled in the 2010 survey.
Source: NSW Admitted Patient Data Collection (HOIST), Centre for Epidemiology and Research, NSW Ministry of Health.

Most hospital separations among Aboriginal and Torres Strait Islander peoples were correctly recorded on the Patient Administration System. There was an absolute increase of 3% compared to the 2007 survey, as mentioned. This improvement was not statistically significant.

The survey was stratified by geographic remoteness of the hospital to allow adjustment for differences in levels of reporting caused by the remoteness. The strength of this approach is that it addresses any bias that might be introduced through differences in the population caused by the geographic location. It should be noted, however, that the original sample sizes were calculated to allow estimation of the quality of the reporting for each remoteness level at a national rather than state level. For this reason, the estimates of correct reporting for each level of hospital remoteness for NSW are not precise and this imprecision is reflected by wide confidence intervals (see Table 2). Despite this, correct reporting was found to increase with increased geographic remoteness of the hospital.

There are several aspects to the study design that may affect the generalisability of the results to the NSW hospital population:

• The sample population was restricted to people aged 18 years or over.

• The sample population was restricted to NSW public hospitals.

• The sample population was restricted to those well enough to be interviewed. There is likely to be under- representation of very sick patients who could not be interviewed, or day-only patients.

• Within each geographic stratum, the selection of hospitals for the survey was purposive rather than random. Purpose sampling was used to ensure that a sufficient sample of Aboriginal and Torres Strait Islander peoples were interviewed. Due to the small numbers of Aboriginal and Torres Strait Islander people sampled, we were unable to check the representativeness of the sample. The possibility of sampling bias therefore cannot be excluded.

While the number of Aboriginal and Torres Strait Islander people is small in the survey, the percentage is more than twice the estimated 2.2% of Aboriginal and Torres Strait Islander peoples in the population reported at the 2006 Census.⁶ This reflects the purposive nature of the sampling. Other potential explanations are that hospitals were necessarily aware that an audit was taking place and, during the audit period and administrative staff made a conscious effort to improve reporting of Aboriginal and Torres Strait Islander peoples on the Patient Administration System. Trend information obtained from the APDC, however, provided no evidence to suggest this occurred.

The survey determined the level of correct reporting for those Aboriginal and Torres Strait Islander peoples who agreed to participate in the survey and who chose to identify themselves as Aboriginal or Torres Strait Islander at interview. It is possible that some Aboriginal or Torres Strait Islander patients chose not to identify themselves at interview and on admission.

While reporting of Aboriginal and Torres Strait Islander peoples was reasonably high overall at 91%, there remains room for improvement. Future interventions to increase reporting should target hospitals in major cities, where the level of reporting, at 83%, remains relatively low. The increase in reporting from 88% in 2007 to 91% in 2010 did not achieve statistical significance. Further surveys are needed to confirm that reporting of Aboriginal and Torres Strait Islander peoples is truly improving over time. Future surveys should have an increased sample size and cover a larger number of hospitals to better assess the significance of small increases in reporting and provide a more representative sample. A greater coverage of hospitals and a larger patient sample size will allow the level of correct reporting to be reliably estimated within levels of geographic remoteness.

JPB was employed as part of the NSW Biostatistical Officer Training Program funded by the NSW Ministry of Health.