Health policy evaluation in rural and remote Australia: a qualitative exploration and lessons from the Northern Territory
Kate Raymond
A B * , Sally Nathan B C , Reema Harrison B C and Lois Meyer B
+ Author Affiliations
- Author Affiliations
A Northern Territory Department of Health, Darwin, NT, Australia.
B School of Population Health, University of New South Wales, Sydney, NSW, Australia.
C Australian Institute of Health Innovation, Macquarie University, Sydney, NSW, Australia.
* Correspondence to: kate.raymond@nt.gov.au
Australian Health Review 47(2) 197-202 https://doi.org/10.1071/AH22255
Submitted: 8 November 2022 Accepted: 5 February 2023 Published: 28 February 2023
© 2023 The Author(s) (or their employer(s)). Published by CSIRO Publishing on behalf of AHHA.
Abstract
Objective This study explored approaches of government policymakers to health policy evaluation (HPE) in Australian rural and remote settings.
Methods Semi-structured interviews captured experiences and insights of 25 policymakers in the Northern Territory Department of Health. Data were thematically analysed using an inductive approach to coding and theme development.
Results We identified five main themes about HPE in rural and remote settings: (1) centring the rural and remote context; (2) balancing ideology, power and evidence; (3) working with community; (4) strengthening policy workforce capabilities in monitoring and evaluation; and (5) valuing evaluation through leadership.
Conclusion HPE is complex in any setting, however, policymakers navigate unique complexities in rural and remote health contexts. HPE can be enabled by developing policymaker and leadership capabilities in rural and remote settings and supporting co-design with communities.
Keywords: health policy, performance and evaluation, population health, rural and remote health, workforce.
References
[1] Wakerman J, Bourke L, Humphreys J, Taylor J. Is remote health different to rural health? Rural Remote Health 2017; 17 3832| Is remote health different to rural health?Crossref | GoogleScholarGoogle Scholar |
[2] Wakerman J. Defining remote health. Aust J Rural Health 2004; 12 210–4.
| Defining remote health.Crossref | GoogleScholarGoogle Scholar |
[3] Scott JW, Schwartz TA, Dimick JB. Practical Guide to Health Policy Evaluation Using Observational Data. JAMA Surg 2020; 155 353–4.
| Practical Guide to Health Policy Evaluation Using Observational Data.Crossref | GoogleScholarGoogle Scholar |
[4] Sheingold S, Bir A, editors. Evaluation for Health Policy and Health Care: A Contemporary Data-driven Approach. SAGE Publications; 2019.
[5] Evans BA, Snooks H, Howson H, Davies M. How hard can it be to include research evidence and evaluation in local health policy implementation? Results from a mixed methods study. Implement Sci 2013; 8 17
| How hard can it be to include research evidence and evaluation in local health policy implementation? Results from a mixed methods study.Crossref | GoogleScholarGoogle Scholar |
[6] Freund M, Zucca A, Sanson-Fisher R, Milat A, Mackenzie L, Turon H. Barriers to the evaluation of evidence-based public health policy. J Public Health Policy 2019; 40 114–25.
| Barriers to the evaluation of evidence-based public health policy.Crossref | GoogleScholarGoogle Scholar |
[7] Bourke L, Humphreys JS, Wakerman J, Taylor J. Understanding rural and remote health: a framework for analysis in Australia. Health Place 2012; 18 496–503.
| Understanding rural and remote health: a framework for analysis in Australia.Crossref | GoogleScholarGoogle Scholar |
[8] Bourke L, Humphreys JS, Wakerman J, Taylor J. Understanding drivers of rural and remote health outcomes: a conceptual framework in action. Aust J Rural Health 2012; 20 318–23.
| Understanding drivers of rural and remote health outcomes: a conceptual framework in action.Crossref | GoogleScholarGoogle Scholar |
[9] Woolf SH. Necessary but not sufficient: Why health care alone cannot improve population health and reduce health inequities. Ann Fam Med 2019; 17 196–9.
| Necessary but not sufficient: Why health care alone cannot improve population health and reduce health inequities.Crossref | GoogleScholarGoogle Scholar |
[10] Baciu A, Negussie Y, Geller A, Weinstein JN. The root causes of health inequity. In: Weinstein J, Geller A, Negussie Y, Baciu A, editors. Communities in action: Pathways to health equity. National Academies Press (US); 2017. pp. 99–184.
[11] Gopalan M, Rosinger K, Ahn JB. Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Rev Res Educ 2020; 44 218–43.
| Use of quasi-experimental research designs in education research: Growth, promise, and challenges.Crossref | GoogleScholarGoogle Scholar |
[12] Barratt H, Campbell M, Moore L, Zwarenstein M, Bower P. Randomised controlled trials of complex interventions and large-scale transformation of services. Health Serv Deliv Res 2016; 4 19–36.
| Randomised controlled trials of complex interventions and large-scale transformation of services.Crossref | GoogleScholarGoogle Scholar |
[13] Datta J, Petticrew M. Challenges to evaluating complex interventions: a content analysis of published papers. BMC Public Health 2013; 13 568
| Challenges to evaluating complex interventions: a content analysis of published papers.Crossref | GoogleScholarGoogle Scholar |
[14] Basu S, Meghani A, Siddiqi A. Evaluating the health impact of large-scale public policy changes: classical and novel approaches. Annu Rev Public Health 2017; 38 351–70.
| Evaluating the health impact of large-scale public policy changes: classical and novel approaches.Crossref | GoogleScholarGoogle Scholar |
[15] Schram A, Townsend B, Mackean T, Freeman T, Fisher M, Harris P, Whitehead M, van Eyk H, Baum F, Friel S. Promoting action on structural drivers of health inequity: principles for policy evaluation. Evid Policy 2022; 18 761–75.
| Promoting action on structural drivers of health inequity: principles for policy evaluation.Crossref | GoogleScholarGoogle Scholar |
[16] Partridge E. Caught in the same frame? The language of evidence-based policy in debates about the Australian government ‘intervention’ into Northern Territory Aboriginal communities. Soc Policy Adm 2013; 47 399–415.
| Caught in the same frame? The language of evidence-based policy in debates about the Australian government ‘intervention’ into Northern Territory Aboriginal communities.Crossref | GoogleScholarGoogle Scholar |
[17] Nathan S, Lancaster K, Newman CE. Qualitative interviewing. In: Liamputtong P, editor. Handbook of Research Methods in Health Social Sciences. Springer; 2019. pp. 391–410.
[18] Liamputtong P. Qualitative data analysis: conceptual and practical considerations. Health Promot J Aust 2009; 20 133–9.
| Qualitative data analysis: conceptual and practical considerations.Crossref | GoogleScholarGoogle Scholar |
[19] Charmaz K. Premises, principles, and practices in qualitative research: Revisiting the foundations. Qual Health Res 2004; 14 976–93.
| Premises, principles, and practices in qualitative research: Revisiting the foundations.Crossref | GoogleScholarGoogle Scholar |
[20] Zhao Y, You J, Wright J, Guthridge SL, Lee AH. Health inequity in the Northern Territory, Australia. Int J Equity Health 2013; 12 79
| Health inequity in the Northern Territory, Australia.Crossref | GoogleScholarGoogle Scholar |
[21] Australian Bureau of Statistics. 2016 Census: Northern Territory. Canberra: ABS; 2017. Available at https://dbr.abs.gov.au/region.html?lyr=ste&rgn=7 [accessed August 2022].
[22] Northern Territory Department of Health. Northern Territory Health Strategic Plan 2018-2022. Darwin: Northern Territory Government; 2018. Available at https://digitallibrary.health.nt.gov.au/prodjspui/bitstream/10137/2729/3/Northern%20Territory%20Health%20Strategic%20Plan.pdf [accessed August 2022].
[23] Northern Territory Department of Health. Annual Report 2020-21. Darwin: Northern Territory Government; 2021. Available at https://digitallibrary.health.nt.gov.au/prodjspui/handle/10137/12314 [accessed January 2023].
[24] Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health 2015; 42 533–44.
| Purposeful sampling for qualitative data collection and analysis in mixed method implementation research.Crossref | GoogleScholarGoogle Scholar |
[25] Iivari N. Using member checking in interpretive research practice: A hermeneutic analysis of informants’ interpretation of their organizational realities. Inf Technol People 2018; 31 111–33.
[26] Terry G, Hayfield N. Reflexive thematic analysis. In: Ward MRM, Delamont S, editors. Handbook of qualitative research in education. Edward Elgar Publishing; 2020. pp. 430–41.
[27] Terry G, Hayfield N, Clarke V, Braun V. Thematic Analysis. In: Willig C, Rogers WS, editors. The SAGE handbook of qualitative research in psychology. Sage; 2017.
[28] Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Gibbs GR, editor. Analyzing qualitative data. Routledge; 2002.
[29] Hughes A, Gleeson D, Legge D, Lin V. Governance and policy capacity in health development and implementation in Australia. Policy Soc 2015; 34 229–45.
| Governance and policy capacity in health development and implementation in Australia.Crossref | GoogleScholarGoogle Scholar |
[30] Alston L, Nichols M, Allender S. Policy makers’ perceptions of the high burden of heart disease in rural Australia: implications for the implementation of evidence-based rural health policy. PLoS One 2019; 14 e0215358
| Policy makers’ perceptions of the high burden of heart disease in rural Australia: implications for the implementation of evidence-based rural health policy.Crossref | GoogleScholarGoogle Scholar |
[31] Australian Institute of Health and Welfare. Australia’s Health 2021: In brief. Canberra: Commonwealth of Australia; 2021. Available at https://www.aihw.gov.au/getmedia/2aa9f51b-dbd6-4d56-8dd4-06a10ba7cae8/aihw-aus-232.pdf.aspx?inline=true [cited 20 June 2022].
[32] Mackean T, Fisher M, Friel S, Baum F. A framework to assess cultural safety in Australian public policy. Health Promot Int 2020; 35 340–51.
| A framework to assess cultural safety in Australian public policy.Crossref | GoogleScholarGoogle Scholar |
[33] Haldane V, Chuah FLH, Srivastava A, Singh SR, Koh GCH, Seng CK, Legido-Quigley H. Community participation in health services development, implementation, and evaluation: A systematic review of empowerment, health, community, and process outcomes. PLoS One 2019; 14 e0216112
| Community participation in health services development, implementation, and evaluation: A systematic review of empowerment, health, community, and process outcomes.Crossref | GoogleScholarGoogle Scholar |
[34] Slattery P, Saeri AK, Bragge P. Research co-design in health: a rapid overview of reviews. Health Res Policy Syst 2020; 18 17
| Research co-design in health: a rapid overview of reviews.Crossref | GoogleScholarGoogle Scholar |
[35] Blomkamp E. The Promise of Co-Design for Public Policy. In: Howlett M, Mukherjee I, editors. Routledge handbook of policy design. New York: Routledge; 2018.
[36] Productivity Commission. Indigenous Evaluation Strategy. Canberra: Commonwealth of Australia; 2020. Available at https://www.pc.gov.au/inquiries/completed/indigenous-evaluation/strategy/indigenous-evaluation-strategy.pdf [cited 9 January 2023].
[37] Walter M, Lovett R, Maher B, Williamson B, Prehn J, Bodkin‐Andrews G, Lee V. Indigenous data sovereignty in the era of big data and open data. Aust J Soc Issues. 2021; 56 143–56.
| Indigenous data sovereignty in the era of big data and open data.Crossref | GoogleScholarGoogle Scholar |
[38] Gleeson DH, Legge DG, O’Neill D. Evaluating health policy capacity: Learning from international and Australian experience. Aust N Z Health Policy 2009; 6 3
| Evaluating health policy capacity: Learning from international and Australian experience.Crossref | GoogleScholarGoogle Scholar |
