Indicators for continuous quality improvement for otitis media in primary health care for Aboriginal and Torres Strait Islander children

Additional keywords: Indigenous health.

Acute otitis media (AOM) is a common childhood illness and is seen in almost 1 in 10 of all general practice encounters for children aged 0–18 years (Gunasekera et al. 2007). Although AOM is generally self-limiting in developed countries, it can progress to severe disease such as chronic suppurative otitis media (CSOM) and persistent otitis media with effusion (OME). Aboriginal and Torres Strait Islander children have among the highest prevalences of these diseases in the world (Verhoeff et al. 2006; Australian Institute of Health and Welfare and Australian Institute of Family Studies 2014). They have been associated with hearing loss and subsequent speech and language delays (Aithal et al. 2008) and poor educational outcomes (Williams and Jacobs 2009; Senate Community Affairs References Committee 2010). The hearing loss associated with otitis media (OM) is thought to have an effect across the life course, resulting in behavioural problems and social isolation; poor school attendance and low levels of literacy and numeracy; poor employment opportunities and increased poverty (Senate Community Affairs References Committee 2010; Leach 2016). Hearing loss affects the criminal justice system at multiple levels (Howard et al. 1993) and has been documented in up to 90% of Aboriginal and Torres Strait Islander inmates, leading to questions about its contribution to the over-representation of Aboriginal and Torres Strait Islander people in that system (Vanerpoll and Howard 2011). AOM and its sequelae are thus major causes of morbidity for Aboriginal and Torres Strait Islander people.

Although AOM and its sequelae are influenced by several environmental factors, appropriate medical intervention can improve the disease course. This is achieved by either reducing the progression to severe disease or else identifying and treating hearing loss early, thus minimising its effect on speech and language development. This requires appropriate screening and treatment in primary health care and subsequent testing and treatment by audiologists and surgeons.

Continuous quality improvement (CQI) is now a requirement of all primary healthcare services receiving Australian Government funding for care for Aboriginal and Torres Strait Islander people. A soon-to-be-released National CQI Framework for Aboriginal and Torres Strait Islander Primary Health Care identifies regular Plan–Do–Study–Act cycles, as an essential component of continuous quality improvement. Through the development of the National Key Performance Indicators for Aboriginal and Torres Strait Islander primary health care, there are now robust indicators for use by services for continuous quality improvement for antenatal care, immunisations, smoking, alcohol consumption, type 1 and type 2 diabetes, cardiovascular disease and cervical screening (Sibthorpe et al. 2016). Separately, indicators have been developed within the community-controlled sector for sexually transmitted infections. However, there are currently no indicators to support services to review their treatment of otitis media (OM) and its sequelae among Aboriginal and Torres Strait Islander children. There is evidence that indicators extracted from electronic health records, as part of continuous quality improvement processes, can improve the quality of care (Peiris et al. 2015). We aimed to develop a suite of indicators on the diagnosis and treatment of OM that could be extracted from electronic health records to assist primary healthcare practitioners monitor the quality of their otitis media care.

This work had its genesis in a university, community-controlled sector continuous quality improvement research partnership. We used an expert group consensus process, managed by one of the authors (BS), to develop the indicators.

A small, nationally dispersed group of professionals with relevant knowledge and experience was established by invitation (n = 8). Each has worked for a decade or more (some for many decades) in Aboriginal and Torres Strait Islander health, including in community-controlled organisations. Their combined OM-related expertise is in specialist paediatric otolaryngology, general practice, nursing and health worker practice, audiology, epidemiological research and program development. Their general expertise includes continuous quality improvement, primary healthcare indicator development and primary healthcare data management and reporting. Three members of the group were centrally involved in the development of the definitive Recommendations for Clinical Care Guidelines on the Management of Otitis Media in Aboriginal and Torres Strait Islander Populations 2010 (Department of Health and Ageing and Menzies School of Health Research 2011). Members variously drew on input from colleagues throughout the development process. Expert reviewers (n = 7) identified by expert group members and covering all of the relevant professional expertise, reviewed a draft set of indicators. As well, correspondence was maintained with community-controlled ear health program staff in two states. A late stage field test of the indicators in community-controlled settings was done by a member of the expert group (MW).

The group process was facilitated by clear purpose, use of a conceptual framework, strong shared commitment and mutual respect for different views about how the system should work to support optimal ear health outcomes for Aboriginal and Torres Strait Islander children. The primary purpose of the indicators is their use by Aboriginal Community-Controlled Health Services and other primary healthcare organisations to monitor the quality of their OM care through continuous quality improvement Plan–Do–Study–Act cycles. Designed as a menu of indicators to flexibly meet local needs, the goal for the final set was six to eight indicators. Based on the criteria published by Crampton et al. (2004), the indicators were to: be quantitative; reflect important aspects of OM; reflect evidence-based or best practice; be meaningful for clinicians in everyday primary healthcare practice and reasonably readily influenced by their actions; be responsive over time periods that make them useful for monitoring quality within a continuous quality improvement framework (6 months to 1 year); be feasible to collect; and minimise perverse incentives.

The conceptual framework used in the development process was the Framework for Performance Assessment in Primary Health Care (FPA_PHC) (Sibthorpe and Gardner 2007). Based on the ‘structure’, ‘process’ and ‘outcome’ model for assessment of quality of care by Donabedian (1988), it specifies measurement at four levels: the stewardship role of governments (Level 1); local health services’ organisational structures and processes (Level 2); processes of care (Level 3); and intermediate outcomes (Level 4). Importantly, processes are split across two levels – processes of care delivered belong with structures in Level 2, whereas processes of care received belong in Level 3. Thus, the denominator at Level 3 always relates to clients not providers. This framework has been successfully used for other indicator development processes, including for the now superseded National Quality and Performance Indicators for Divisions of General Practice. So that the indicators were consistent with the approach of the National Key Performance Indicators, it was agreed by the group that the indicators would all relate to regular clients, as defined by the Australian Institute of Health and Welfare (see http://meteor.aihw.gov.au/content/index.phtml/itemId/436653, accessed 1 August 2016) and be confined to Levels 3 and 4 (Sibthorpe et al. 2016).

The indicator development process is summarised in Tables 1 and 2. Recognising the importance in quality improvement processes of the perceived benefit of indicators to people working at the coalface, we worked from the ground up, beginning with a broad canvassing of members’ indicator ‘ideas’. This resulted in a list of 81 ideas that were collated into summary 1. A first cull was undertaken (by BS) to remove all Level 2 ideas and ideas at Levels 3 and 4 for which there was no data source, leaving 34 ideas (summary 2). Both summaries were sent to expert group members and a face-to-face meeting convened, during which both summaries were reviewed and a short list of 10 indicator ideas was agreed. These were then developed as indicators through a series of drafts during which they were repeatedly cross-referenced against the Recommendations and other evidence to ensure they were consistent with best practice. They were then assessed for their ability to be extracted from medical software by one of the major software providers, subjected to final field review (MW), and before the final set was agreed upon. The process was run interspersed with other project activities over 18 months, but could easily be compressed into a shorter timeframe.

Table 1.  Summary of the expert group consensus process for development of primary healthcare indicators for otitis media (OM) in Aboriginal and Torres Strait Islander children
FPA_PHC, Framework for Performance Assessment in Primary Health Care

Table 2.  Indicator ideas remaining following the first cull and reasons for their removal at the second cull
FPA_PHC, Framework for Performance Assessment in Primary Health Care; N/R, not removed; OM, otitis media; CSOM, suppuratives otitis media; ENT, ears, nose and throat; OME, otitis media with effulsion; HL, hearing loss; NPDC, National Perinatal Data Collection; nKPIs, National Key Performance Indicators; AIR, Australian Immunisation Register; TMs, tympanic membranes

The final set of indicators is shown in Table 3 and definitions of the terms used in Table 4. Table 3 highlights the indicators’ level within the FPA_PHA and the level of evidence supporting its implementation. The final adjustments that were made following external review and field testing are highlighted in bold. Most notably OM_1, which was developed as a single screening indicator using both otoscopy and tympanometry, was split into OM_1a and OM_1b, because the two screening rates may vary widely (otoscopy higher than tympanometry) for several reasons, including access to a tympanometer, its ongoing maintenance and training in its use. Also, because of the recurrent nature of OM, the numerator was changed from children with OM to episodes of OM among children.

Table 3.  Primary healthcare indicators for otitis media (OM) in Aboriginal and Torres Strait Islander children
Technical specifications for the final indicators are available from Dr Jason Agostino, ANU Medical School. These are draft, pending coding, data entry and extraction development for the relevant electronic health record systems. National Health and Medical Research Council (NHMRC) additional levels of evidence and grades for recommendations for developers of guidelines are available at http://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/developers/nhmrc_levels_grades_evidence_120423.pdf (accessed 12 December 2016). GPP, good practice point

Table 4.  Definitions for terms used in primary healthcare indicators for otitis media (OM) in Aboriginal and Torres Strait Islander children

Technical specifications for the indicators have been developed, but are in draft form pending coding, data entry and extraction development for the relevant electronic health record systems. They are available from one of the authors (JA) on request (see details Table 3).

As shown, all the indicators relate to regular clients. Consistent with the National Key Performance Indicators, a regular client is defined as a client who has had a minimum of three visits in the previous 2 years. The focus on regular clients’ support services in assessing how well they are doing in caring for the clients they are seeing on an ongoing basis. Consistently defined, it also allows cross-services comparisons of achievement, which are likely to become an important and highly valued part of continuous quality improvement processes at regional and state levels within the community-controlled sector. The focus on regular clients is not meant to imply that other children are not important or worthy of best practice care and some data extraction tools allow services to vary the client population to capture all clients for internal continuous quality improvement processes.

A robust set of OM indicators for primary care continuous quality improvement has been developed. They focus predominantly on processes of care received (six indicators) that cover screening, prescribing, care planning, follow up, referral and testing for sequelae (hearing loss). One indicator addresses outcomes (disease incidence). Although the initial list of indicators contained several other outcome indicators, some are collected elsewhere (e.g. maternal smoking is included in the National Key Performance Indicators), some have no electronic health record data source (e.g. resolution following antibiotic treatment) and the others were predominately influenced by factors outside of primary health care (e.g. closure of air-bone gap). Although high quality of care may affect disease incidence, it is acknowledged that improvements in environmental factors such as overcrowding will also be required to improve outcomes in hearing health.

OM is a disease that has far-reaching implications that encompass many aspects of social life. Non-client outcomes – for example, those relating to OM literacy among carers, educators and the wider community – are an important part of a comprehensive approach to prevention and care. Indicator ideas relating to such outcomes put forward for the initial list are therefore also important, but they could not be included in this set because of lack of data. Indicator ideas at Level 2, relating to providers, were also put forward. These were culled to retain the client focus, which is so important in monitoring quality, but this can have the effect of making workers feel that important things they do are not being monitored. In both cases, indicators cannot be interpreted in isolation, so service dialogues will necessarily encompass discussions of issues and approaches that draw on provider behaviours and relationships with the broader community into focus.

All indicators rely on good quality data and the ability to extract and display it in meaningful ways. As for many (if not most) other measures in primary health care, this is the major challenge facing this set of indicators. Within Australia, primary healthcare electronic health record systems are currently unregulated. There are multiple systems (at least eight just in general practice) and each has been developed independently, with no common standards across them all (Gordon et al. 2016), resulting in different approaches to data recording and coding, and different extraction tools and reporting functionalities. Underpinning the electronic health records are several terminology classification systems (the International Classification for Primary Care (ICPC-2) is commonly used in Australia but not mandated (Gordon et al. 2016); and several terminology interfaces that do not all map to each other or to ICPC-2. As well, there are limitations in the OM codes in the ICPC-2.

In recent years, attempted expanded use of electronic health record data for quality improvement and other purposes has brought these deficiencies into sharp focus, leading to calls for nationally agreed and implemented standards for electronic health records in Australia in three areas: data model; data element names and definitions; and clinical terminology and classifications (Gordon et al. 2016). Work is underway on many fronts that will inform these developments. In the meantime, our experience has shown that with appropriate indicators, individual services can make very effective use of their electronic health records for quality improvement purposes, with initial data extracts addressing a given health issue often highlighting data quality problems that then become the subject of the early Plan–Do–Study–Act cycles. These processes are supported by the ongoing expansion by software providers in the sophistication of tools for electronic health record data extraction and display, and expanding in-house capacity in this area.

It is also widely recognised within the sector that a great deal of staff training is needed to improve OM data quality in primary health care. In a study of the value of their electronic health records for assessing the effectiveness of primary health care in an Aboriginal community in Western Australia, Davis et al. (2015) found that 32% of children with no coded diagnosis of OM had evidence of OM on examination of the notes. They suggested that:

…while busy clinicians may see filling [diagnosis] in as important when a condition requires ongoing management (such as [type 1 and type 2] diabetes), this may not be the case for an acute and/or self-limiting condition, which clinicians may find easier to record in free text.

They concluded that ‘acute disease data were not accurate enough to be clinically useful’ for assessing quality of care. They emphasised the importance of the clinician’s role in accurately entering data and noted that a clinician engagement strategy had been implemented to help rectify the problems observed.

Although the data issues for OM management in primary care seem daunting, the improvements in data quality for chronic diseases show that these problems can be overcome. Aboriginal Community Controlled Health Services have become familiar with quality data entry through the National Key Performance Indicators, and the paper from Davis et al. (2015) found that 100% of diagnoses related to existing National Key Performance Indicators were correctly entered into the electronic health record. Service-level improvements need to be supported at a system level to ensure consistent and appropriate coding of conditions across different electronic health record systems.

OM care regularly needs to draw in providers across the full spectrum of primary, secondary and tertiary care. In almost all instances, the data from these providers are still housed in providers’ silos, rather than a single silo built around the child, so the health record complexities in primary health care proliferate out across the system as a whole. For example, in 2011, only 38% of specialists used a computer for viewing and recording patient information (Department of Health and Ageing 2011). The exception to this is Northern Territory Department of Health’s soon-to-be-released, custom-built Hearing Health Information Management System that stores specialist ear and hearing clinical data and supports multidisciplinary care and outreach. Not all jurisdictions will go down this path, so data sharing will depend on information system interoperability solutions. Once these catch up with health system goals of client focus and continuity of care, we will have the data capabilities to add performance indicators relating to wider outcomes for OM, to monitor and to use to improve care and outcomes for clients.

To ensure the indicators are of the greatest use to health, they will need to be reviewed. We anticipate that once they have been in use for 12 months, we will work with the Aboriginal community-controlled health sector and other stakeholders to obtain feedback on their implementation. The feedback will be reviewed by the expert group, with reference to the literature on good indicators and effective CQI. Any proposed changes will be reviewed by the sector and other stakeholders and any amendments made will be published in the peer-reviewed literature.

A focus on continuous quality improvement for OM in primary health care using this small set of indicators has the potential to greatly improve ear health outcomes for Aboriginal and Torres Strait Islander children. A comprehensive approach will address not only OM screening and care, but also the social determinants of health that are recognised risk factors for OM, including smoking, bottle feeding, poor nutrition and the effects of overcrowding (Kong and Coates 2009; Department of Health and Ageing and Menzies School of Health Research 2011; Australian Institute of Health and Welfare and Australian Institute of Family Studies 2014), thereby having a broader effect on closing the health gap for Aboriginal and Torres Strait Islander children.

The authors declare that no conflicts of interest exist.