Free Standard AU & NZ Shipping For All Book Orders Over $80!
Register      Login
Australian Health Review Australian Health Review Society
Journal of the Australian Healthcare & Hospitals Association
RESEARCH ARTICLE

An evaluation of a benchmarking initiative in extended treatment mental health services

Nicole Shepherd A , Thomas J. Meehan A B E , Fiona Davidson C and Terry Stedman B D
+ Author Affiliations
- Author Affiliations

A Service Evaluation and Research Unit, The Park, Centre for Mental Health, Sumner Park, QLD 4074, Australia.

B Department of Psychiatry, The University of Queensland, Herston, QLD 4072, Australia.

C Queensland Mental Health Benchmarking Unit, The Park, Centre for Mental Health, Sumner Park, QLD 4074, Australia.

D The Park, Centre for Mental Health, Sumner Park, QLD 4074, Australia.

E Corresponding author. Email: thomas_meehan@health.qld.gov.au

Australian Health Review 34(3) 328-333 https://doi.org/10.1071/AH09698
Submitted: 2 October 2008  Accepted: 26 November 2009   Published: 25 August 2010

Abstract

Aim. To evaluate the effectiveness of a benchmarking initiative in facilitating quality improvement and reform in extended care mental health services.

Method. Individual interviews and focus group discussions were conducted with 84 staff in 22 extended care mental health services that had previously participated in a State-wide benchmarking exercise in Queensland, Australia.

Results. Staff reported positive outcomes from participation in the benchmarking exercise. Information derived from benchmarking provided a different perspective on the strengths and weaknesses of individual services and an opportunity to learn from peer services. Staff in 86% of the services identified issues that needed to be addressed and 64% of services had implemented one or more service improvement projects in response to shortcomings identified through the benchmarking exercise.

Conclusions. The collection and reporting of performance data through a process of benchmarking was successful in facilitating service improvement in most of the participating facilities. Engaging services in all stages of the process was considered useful in converting benchmarking data into knowledge that was able to be applied at the local service level.

What is known about the topic? Implementing and maintaining service improvement initiatives in mental health services can be difficult, due to ongoing resource limitations in the sector. Although there is currently a drive to promote benchmarking for mental health services, to date, there has been limited evidence that it leads to sustained service improvements.

What does this paper add? This paper reports on an evaluation of a benchmarking initiative in extended treatment mental health services in Queensland, Australia. Six months after implementation, staff reported positive outcomes from participation in the project, and 64% of services had implemented one or more service improvement projects. Engagement of service staff in the process of benchmarking is important to ensure success.

What are the implications for practitioners? Benchmarking initiatives can lead to service improvement. It is important to involve staff in the process of benchmarking, from choosing performance indicators and collection of data, to interpretation of the results. Organising events where staff can present findings from the project to their peers provides the opportunity to reflect on the direction of their service and learn from others. In this way, information provided through benchmarking can be converted into knowledge that is applied at the service level.


Acknowledgements

We thank the service staff who graciously gave us their time to speak to us about the benchmarking project.


References


[1] Chisholm M,  Howard PB,  Boyd MA,  Clement JA,  Hendrix MJ,  Reiss-Brennan B. Quality indicators for primary mental health within managed care: a public health focus. Arch Psychiatr Nurs 1997; XI(4): 167–81.
Crossref | GoogleScholarGoogle Scholar |

[2] Carpinello S,  Felton CJ,  Pease EA,  DeMasi M,  Donohue S. Designing a system for managing the performance of mental health managed care: an example from New York State’s prepaid mental health plan. J Behav Health Serv Res 1998; 25(3): 269–78.
Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |

[3] Meehan TJ,  Stedman TJ,  Neuendorf KE,  Francisco ID,  Neilson MG. Benchmarking Australia’s mental health services: is it possible and useful? Aust Health Rev 2007; 31(4): 623–7.
Crossref | GoogleScholarGoogle Scholar | PubMed |

[4] Eager K , Burgess P , Buckingham B . Towards National Benchmarks for Australian Mental Health Services. ISC Discussion Paper. Report No.: 4 Canberra: Australian Government Department of Health and Ageing; 2003.

[5] NMHWG Information Strategy Committee Performance Indicator Drafting Group. Key Performance Indicators for Australian Public Mental Health Services. Discussion Paper. Canberra: Australian Government Department of Health and Ageing; 2005.

[6] Rosenheck R,  Cicchetti D. A mental health program report card: a multidimensional approach to performance monitoring in public sector programs. Community Ment Health J 1998; 34(1): 85–106.
Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |

[7] Ellis J. All inclusive benchmarking. J Nurs Manag 2006; 14(5): 377–83.
Crossref | GoogleScholarGoogle Scholar | PubMed |

[8] Bullivant JRN . Benchmarking for Continuous Improvement in the Public Sector. Harlow, Essex: Longman; 1994.

[9] Corrigan PW,  Steiner L,  McCracken SG,  Blaser B,  Barr M. Strategies for disseminating evidence-based practices to staff who treat people with serious mental illness. Psychiatr Serv 2001; 52(12): 1598–606.
Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |

[10] Rogers PJ , Williams B . Evaluation for practice improvement and organizational learning. In Shaw IF, Greene JC, Mark MM, editors. The Sage Handbook of Evaluation. London: Sage; 2006. pp. 76–97.

[11] Schick A. The performing state: reflection on an idea whose time has come but whose implementation has not. OECD Journal on Budgeting 2003; 3(2): 71–103.
Crossref | GoogleScholarGoogle Scholar |

[12] Cousins JB , Shulha LM . A comparative analysis of evaluation utilization and its cognate fields of inquiry: current issues and trends. In Shaw IF, Greene JC, Mark MM, editors. The Sage Handbook of Evaluation. London: Sage; 2006. pp. 266–91.

[13] Birleson P,  Brann P. Reviewing the learning organisation model in a child and adolescent mental health service. Aust Health Rev 2006; 30(2): 181–94.
Crossref | GoogleScholarGoogle Scholar | PubMed |

[14] Birleson P. Building a learning organisation in a child and adolescent mental health service. Aust Health Rev 1998; 21(3): 223–40.
Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |

[15] NVivo Qualitative Data Analysis Software. 8 edn. Melbourne: QSR International; 2008.

[16] Mental Health Council of Australia. Not for Service: Experiences of Injustice and Despair in Mental Health Care in Australia. Canberra; 2005.

[17] Clarke P,  McCarthy T. Mental health services. Called to the bench. Health Serv J 2000; 110(5722): 30–1.
PubMed |  CAS |

[18] Kaner E,  Steven A,  Cassidy P,  Vardy C. Implementation of a model for service delivery and organisation in mental healthcare: a qualitative exploration of service provider views. Health Soc Care Community 2003; 11(6): 519–27.
Crossref | GoogleScholarGoogle Scholar | PubMed |

[19] Kitson A,  Ahmed LB,  Harvey G,  Seers K,  Thompson D. From research to practice: one organizational model for promoting research-based practice. J Adv Nurs 1996; 23 430–40.
Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |

[20] Sutherland S. Creating a culture of data use for continuous improvement: a case study of an Edison Project School. Am J Eval 2004; 25(3): 277–93.
Crossref | GoogleScholarGoogle Scholar |

[21] Nutley SM , Walter I , Davies HTO . Using Evidence: How Research can Inform Public Services. Bristol, UK: The Policy Press; 2007.

[22] Rosenheck RA. Organizational process: a missing link between research and practice. Psychiatr Serv 2001; 52(12): 1607–12.
Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |

[23] Wenger E , McDermott R , Snyder WM . Cultivating Communities of Practice. Boston, MA, USA: Harvard Business School Press; 2002.