Building a strategic approach to improve Aboriginal health research and evaluation in NSW

Using research evidence to develop and evaluate health policy and practice has the potential to improve health outcomes and resource allocation.¹ More effective mechanisms are needed for generating policy-relevant research evidence, and for translating it into policy and practice.^2,3 In Aboriginal health, the limited evidence-base for informing policy and practice is likely to contribute to the ongoing health disparity between Aboriginal and non-Aboriginal Australians.^4,5 The evaluation and monitoring of programs and services that are delivered to Aboriginal peoples are important for developing evidence that can inform future policy and practice.

In March 2010, the Population and Public Health Division, NSW Ministry of Health, established an Aboriginal health research and evaluation function, recognising the need to increase the generation and use of research evidence, and for the rigorous evaluation of programs and services.⁶ Increasing investment in Aboriginal health in recent years through the Council of Australian Governments (COAG) National Indigenous Reform Agenda has also encouraged more evaluation to produce evidence of effective strategies to improve health outcomes.

In 2011, a strategic plan was developed to guide the Division's work in Aboriginal health research and evaluation. The plan focuses activities on contributing towards improving the health of Aboriginal people by facilitating evidence-based best practice, and meeting national and state government reporting commitments. This paper describes the development of the NSW Ministry of Health's Aboriginal Health Research and Evaluation Strategic Plan 2011–2015, its objectives and strategies, and highlights initial achievements in its implementation. The paper also draws on approaches nationally and internationally for future directions in policy-relevant research and evaluation.

The plan was developed in early 2011 over a 6-month period. A review of published research evidence was used to identify possible strategies and activities that could contribute towards meeting the objectives and overall vision. As well as being evidence-based, the plan reflects existing national and state policy commitments and is achievable within resources.

Strategic planning days were held to identify the aims, objectives and activities to be included in the plan. Interviews were conducted with senior staff in the Population and Public Health Division and other branches within the NSW Ministry of Health responsible for implementing state-wide Aboriginal health programs. Information was collected on:

• perceptions regarding the role of Aboriginal health research and evaluation in NSW Health

• strategies which could increase the use of evidence in policy and program decision making

• strategies which could improve the evaluation of programs in order to develop rigorous evidence on effectiveness

• how to build research and evaluation capacity.

Strategic health research documents were sourced by searching websites of health departments, research institutions and relevant non-government organisations in Australia and internationally, including a targeted search for policies and strategies in Indigenous health, and reviewed.

Draft versions of the plan were developed and feedback from the Population and Public Health Division was used to finalise the document.

For successful implementation of the Strategic Plan, the following guiding principles are a necessity:

• A strong partnership with Aboriginal communities and the Aboriginal Community Controlled Health Services, with the other Divisions of the NSW Ministry of Health and Local Health Districts, as well as the research community particularly the Aboriginal health research community

• An organisational culture within NSW Health that values evidence, evaluation and research

• Staff who have the research capacity to use evidence to guide policy and practice

• A policy direction that supports and drives evidence-based practice

• A mechanism to lead, drive and coordinate statewide research and evaluation projects

• A mechanism that enables reporting and accountability to occur at local, state and national levels

• An effective communication and advocacy strategy across the health system.

Strategies included in the plan and progress with its implementation are outlined below.

Increasing the use of evidence is a commitment under the National Indigenous Reform Agreement⁷ and the NSW Health Population Health Research Strategy.⁸ The activities to be undertaken are based on evidence of what is most likely to be effective: improving access to evidence from research; greater interaction between researchers and policy agencies; and improving organisational capacity for using research including policies, culture and skills.³

Knowledge translation meetings with leading researchers in Aboriginal health have been held to facilitate greater interaction between researchers and policy-makers. The meetings focused on evidence relevant to specific Aboriginal health programs. Evidence reviews have been completed to inform the development of programs, including a review of evidence-based chronic care models, a review of chronic disease interventions, and a review of appropriate evaluation methodologies to generate robust evidence on program effectiveness.

To improve access to published research evidence, the first phase in developing a web-based evidence repository was undertaken in consultation with Aboriginal health policy-makers and health-service providers. To improve organisational capacity, staff have received training and presentations on using research evidence, and receive weekly updates on newly published research in Aboriginal health, with access to full-text journal articles.

Evaluation of Aboriginal health programs is a commitment under the National Indigenous Reform Agreement.⁹ Rigorous evaluation of health services and programs to generate evidence to inform policy is essential for improving health outcomes, and requires significant investment.¹⁰

In 2010 the Ministry began a review of existing and planned evaluations of all of its funded programs in Aboriginal health, and considered possible models available to bring research expertise to these programs. New evaluation projects have commenced using a range of models and scientific methods, including multiple-baseline interrupted time-series design,¹¹ randomised controlled trials and mixed-method approaches. The Centre has worked with other branches within NSW Health to develop evaluations in the areas of maternal and child health, maternal smoking cessation, injury prevention and chronic care and is currently evaluating the Chronic Care Service Enhancements Program. It has run training on evaluation techniques to staff, and to chronic care workers in Local Health Districts and Aboriginal Community Controlled Health Services (ACCHS).

The implementation of research findings into practice have contributed to increases in life expectancy and in quality of life for all Australians. Despite this, there is limited research evidence on effective strategies for improving the health of Aboriginal people.^4,5 Collaborative, locally driven research partnerships are needed to drive research priorities and to undertake research. Developing the infrastructure, capacity and governance required to facilitate such research is an objective of the Ministry of Health, and a priority under the Population and Public Health Division’s Promoting the generation and effective use of population health research in NSW strategy 2011–2015.⁸

The Ministry has made progress in supporting high quality, policy relevant intervention research projects to contribute to developing the evidence base in Aboriginal health. The Ministry is working in partnership with leading research institutions to trial the effectiveness of interventions in priority areas including maternal and child health, chronic disease prevention and management, injury prevention and improving cultural competency and Aboriginal identification within NSW hospitals.

Public reporting on population health and health service delivery can improve patient clinical outcomes and benefit the system as a whole.¹² To encourage effective reporting, the NSW Government has committed to produce an annual progress report on closing the gap between Aboriginal and non-Aboriginal people. The first of these reports will be released in 2012, and a summary report card was launched on National Close the Gap day, 22 March 2012 (see http://www.health.nsw.gov.au/pubs/2012/aboriginalhealthreportcar.html ).

The Ministry of Health contributes to state and national Aboriginal health reporting through Health Statistics NSW,¹³ the National Aboriginal and Torres Strait Islander Health Performance Framework,¹⁴ the National Key Performance Indicators for Indigenous Specific Primary Health Care Services,¹⁵ and the Productivity Commission’s Overcoming Indigenous Disadvantage Key Indicators Report.¹⁶ It has contributed to the development of national trajectories for closing the gap targets and to refining the methodologies used for calculating targets for the COAG National Indigenous Reform Agreement. A number of epidemiological and health services studies that use routinely collected data are being supported, including a data linkage project as part of the 48-hour follow-up evaluation under the Chronic Care for Aboriginal People program which linked program participation data and hospital admissions data to assess the impact of the program on readmissions rates.

Achievements against the strategic plan will be assessed against increasing the use of evidence in policy-making, increasing in the number of high quality evaluations and research projects that address Aboriginal health priorities, an increase in the number of publications and reports that contribute to the evidence base in Aboriginal health, as well as measures of increased capacity, organisational change and collaboration with key stakeholders.

The aim of the NSW Ministry of Health's Aboriginal Health Research and Evaluation Strategic Plan 2011–2015 is to focus the activities towards facilitating evidence-based practice by ensuring policies and programs are evidence-based and evaluated rigorously, and health indicators are monitored and reported accurately. The plan aligns with the NSW Health Promoting the generation and effective use of population health research in NSW. A Strategy for NSW Health 2011–2015.⁸

The implementation of the strategic plan will aim to continuously build on approaches for bringing together research, evidence, policy and health service delivery in order to improve health outcomes. In the area of evidence translation into policy (Strategy 1), emerging models such as that being developed through CIPHER, and the Canadian McMaster Health Forum may provide guidance in this area. The Canadian model coordinates forums with policy-makers, leaders, citizens and researchers for structured deliberations around specific health challenges using an evidence-based medicine and problem-based learning approach.^17,18

The evaluation of Aboriginal health policies and programs (Strategy 2) is consistent with a global trend to increase investment in rigorous designs for research and evaluation to provide recommendations for policy and practice. A useful approach has been developed by the Medical Research Council in the UK, which has produced guidelines for developing and evaluating complex interventions to guide policy agencies on how to design and rigorously evaluate evidence-based interventions.¹⁹

The Ministry will continue to maintain a strong focus on working with health services and population health programs to increase rigorous research and evaluation, and is trialling more feasible and collaborative designs which maintain high standards of scientific rigour.¹¹ The partnership between researchers, policy-makers, practitioners and Aboriginal communities and organisations will be critical for allowing evaluation designs to fit policy and program implementation, and to meet community priorities.

The aim to facilitate strategic research to develop evidence for improving Aboriginal health outcomes (Strategy 3) will contribute towards expanding the limited evidence-base in Aboriginal health on effective interventions.⁵ In Australia, the majority of funded research is investigator driven, therefore the evidence produced may not always provide useful information for policy makers who need to know what works in different settings to improve health service delivery and health outcomes;²⁰ and not how to best allocate resources towards competing priorities. Existing strategic documents relevant to Aboriginal health research also support the need for more strategic approaches for increasing policy-relevant research that provides evidence of what works.^21,22 In October 2011 the NSW Aboriginal Health Partnership between the Aboriginal Health and Medical Research Council of NSW and NSW Health established a Research and Evaluation Sub-Committee to provide strategic direction to research and evaluation activities in Aboriginal health across NSW.

NSW has considerable expertise in the translation of evidence into policy and practice, and in population health intervention research. There is great potential to draw on this expertise to implement the Strategic Plan by supporting the development of effective and collaborative relationships between Aboriginal communities and the Aboriginal Community Controlled Health Services, researchers, policy-makers and health service providers to encourage greater use of evidence, stronger evaluations of services and programs, and the development of rigorous evidence of what works to improve the health of Aboriginal people in NSW.