An evaluation of a benchmarking initiative in extended treatment mental health services
Nicole Shepherd A , Thomas J. Meehan A B E , Fiona Davidson C and Terry Stedman B DA Service Evaluation and Research Unit, The Park, Centre for Mental Health, Sumner Park, QLD 4074, Australia.
B Department of Psychiatry, The University of Queensland, Herston, QLD 4072, Australia.
C Queensland Mental Health Benchmarking Unit, The Park, Centre for Mental Health, Sumner Park, QLD 4074, Australia.
D The Park, Centre for Mental Health, Sumner Park, QLD 4074, Australia.
E Corresponding author. Email: thomas_meehan@health.qld.gov.au
Australian Health Review 34(3) 328-333 https://doi.org/10.1071/AH09698
Submitted: 2 October 2008 Accepted: 26 November 2009 Published: 25 August 2010
Abstract
Aim. To evaluate the effectiveness of a benchmarking initiative in facilitating quality improvement and reform in extended care mental health services.
Method. Individual interviews and focus group discussions were conducted with 84 staff in 22 extended care mental health services that had previously participated in a State-wide benchmarking exercise in Queensland, Australia.
Results. Staff reported positive outcomes from participation in the benchmarking exercise. Information derived from benchmarking provided a different perspective on the strengths and weaknesses of individual services and an opportunity to learn from peer services. Staff in 86% of the services identified issues that needed to be addressed and 64% of services had implemented one or more service improvement projects in response to shortcomings identified through the benchmarking exercise.
Conclusions. The collection and reporting of performance data through a process of benchmarking was successful in facilitating service improvement in most of the participating facilities. Engaging services in all stages of the process was considered useful in converting benchmarking data into knowledge that was able to be applied at the local service level.
What is known about the topic? Implementing and maintaining service improvement initiatives in mental health services can be difficult, due to ongoing resource limitations in the sector. Although there is currently a drive to promote benchmarking for mental health services, to date, there has been limited evidence that it leads to sustained service improvements.
What does this paper add? This paper reports on an evaluation of a benchmarking initiative in extended treatment mental health services in Queensland, Australia. Six months after implementation, staff reported positive outcomes from participation in the project, and 64% of services had implemented one or more service improvement projects. Engagement of service staff in the process of benchmarking is important to ensure success.
What are the implications for practitioners? Benchmarking initiatives can lead to service improvement. It is important to involve staff in the process of benchmarking, from choosing performance indicators and collection of data, to interpretation of the results. Organising events where staff can present findings from the project to their peers provides the opportunity to reflect on the direction of their service and learn from others. In this way, information provided through benchmarking can be converted into knowledge that is applied at the service level.
Acknowledgements
We thank the service staff who graciously gave us their time to speak to us about the benchmarking project.
[1] Chisholm M, Howard PB, Boyd MA, Clement JA, Hendrix MJ, Reiss-Brennan B. Quality indicators for primary mental health within managed care: a public health focus. Arch Psychiatr Nurs 1997; XI(4): 167–81.
| Crossref | GoogleScholarGoogle Scholar |
[2] Carpinello S, Felton CJ, Pease EA, DeMasi M, Donohue S. Designing a system for managing the performance of mental health managed care: an example from New York State’s prepaid mental health plan. J Behav Health Serv Res 1998; 25(3): 269–78.
| Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |
[3] Meehan TJ, Stedman TJ, Neuendorf KE, Francisco ID, Neilson MG. Benchmarking Australia’s mental health services: is it possible and useful? Aust Health Rev 2007; 31(4): 623–7.
| Crossref | GoogleScholarGoogle Scholar | PubMed |
[4]
[5]
[6] Rosenheck R, Cicchetti D. A mental health program report card: a multidimensional approach to performance monitoring in public sector programs. Community Ment Health J 1998; 34(1): 85–106.
| Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |
[7] Ellis J. All inclusive benchmarking. J Nurs Manag 2006; 14(5): 377–83.
| Crossref | GoogleScholarGoogle Scholar | PubMed |
[8]
[9] Corrigan PW, Steiner L, McCracken SG, Blaser B, Barr M. Strategies for disseminating evidence-based practices to staff who treat people with serious mental illness. Psychiatr Serv 2001; 52(12): 1598–606.
| Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |
[10]
[11] Schick A. The performing state: reflection on an idea whose time has come but whose implementation has not. OECD Journal on Budgeting 2003; 3(2): 71–103.
| Crossref | GoogleScholarGoogle Scholar |
[12]
[13] Birleson P, Brann P. Reviewing the learning organisation model in a child and adolescent mental health service. Aust Health Rev 2006; 30(2): 181–94.
| Crossref | GoogleScholarGoogle Scholar | PubMed |
[14] Birleson P. Building a learning organisation in a child and adolescent mental health service. Aust Health Rev 1998; 21(3): 223–40.
| Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |
[15]
[16]
[17] Clarke P, McCarthy T. Mental health services. Called to the bench. Health Serv J 2000; 110(5722): 30–1.
| PubMed | CAS |
[18] Kaner E, Steven A, Cassidy P, Vardy C. Implementation of a model for service delivery and organisation in mental healthcare: a qualitative exploration of service provider views. Health Soc Care Community 2003; 11(6): 519–27.
| Crossref | GoogleScholarGoogle Scholar | PubMed |
[19] Kitson A, Ahmed LB, Harvey G, Seers K, Thompson D. From research to practice: one organizational model for promoting research-based practice. J Adv Nurs 1996; 23 430–40.
| Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |
[20] Sutherland S. Creating a culture of data use for continuous improvement: a case study of an Edison Project School. Am J Eval 2004; 25(3): 277–93.
| Crossref | GoogleScholarGoogle Scholar |
[21]
[22] Rosenheck RA. Organizational process: a missing link between research and practice. Psychiatr Serv 2001; 52(12): 1607–12.
| Crossref | GoogleScholarGoogle Scholar | PubMed | CAS |
[23]
