Register      Login
Australian Health Review Australian Health Review Society
Journal of the Australian Healthcare & Hospitals Association
RESEARCH ARTICLE (Open Access)

The impact of self-assessment and surveyor assessment on site visit performance under the National General Practice Accreditation scheme

David T. McNaughton https://orcid.org/0000-0001-5842-877X A * , Paul Mara B and Michael P. Jones A
+ Author Affiliations
- Author Affiliations

A School of Psychological Sciences, Faculty of Medicine, Health and Human Sciences, Macquarie University, Balaclava Road, North Ryde, Sydney, NSW 2112, Australia.

B Quality Practice Accreditation, South Gundagai, NSW, Australia.

* Correspondence to: David.mcnaughton@mq.edu.au

Australian Health Review 48(3) 222-227 https://doi.org/10.1071/AH23235
Submitted: 7 November 2023  Accepted: 31 January 2024  Published: 19 February 2024

© 2024 The Author(s) (or their employer(s)). Published by CSIRO Publishing on behalf of AHHA. This is an open access article distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND)

Abstract

Objective

There is a need to undertake more proactive and in-depth analyses of general practice accreditation processes. Two areas that have been highlighted as areas of potential inconsistency are the self-assessment and surveyor assessment of indicators.

Methods

The data encompass 757 accreditation visits made between December 2020 and July 2022. A mixed-effect multilevel logistic regression model determined the association between attempt of the self-assessment and indicator conformity from the surveyor assessment. Furthermore, we present a contrast of the rate of indicator conformity between surveyors as an approximation of the inter-assessor consistency from the site visit.

Results

Two hundred and seventy-seven (37%) practices did not attempt or accurately report conformity to any indicators at the self-assessment. Association between attempting the self-assessment and the rate of indicator non-conformity at the site visit failed to reach statistical significance (OR = 0.90 [95% CI = 1.14–0.72], P = 0.28). A small number of surveyors (N = 9/34) demonstrated statistically significant differences in the rate of indicator conformity compared to the mean of all surveyors.

Conclusions

Attempt of the self-assessment did not predict indicator conformity at the site visit overall. Appropriate levels of consistency of indicator assessment between surveyors at the site visit were identified.

Keywords: accreditation, audit and feedback, general practice, health policy, health services, primary health care, quality and safety, reliability.

References

The Royal Australian College of General Practitioners. Standards for General Practices. 5th edn. RACGP; 2020. Available at https://www.racgp.org.au/running-a-practice/practice-standards/standards-5th-edition/standards-for-general-practices-5th-ed-1

Jones M, McNaughton D, Mara P. General practice accreditation–does time spent on-site matter? Aust Health Rev 2023; 47(6): 689-693.
| Crossref | Google Scholar | PubMed |

Khoury J, Krejany CJ, Versteeg RW, et al. A process for developing standards to promote quality in general practice. Fam Pract 2019; 36(2): 166-171.
| Crossref | Google Scholar | PubMed |

Service Australia. Practice Incentives Program. 2023. Available at https://www.servicesaustralia.gov.au/practice-incentives-program

Debono D, Greenfield D, Testa L, et al. Understanding stakeholders’ perspectives and experiences of general practice accreditation. Health Policy 2017; 121(7): 816-822.
| Crossref | Google Scholar | PubMed |

Braithwaite J, Greenfield D, Westbrook J, et al. Health service accreditation as a predictor of clinical and organisational performance: a blinded, random, stratified study. Qual Saf Health Care 2010; 19(1): 14-21.
| Crossref | Google Scholar | PubMed |

Benecke E. Improving Inter-Assessor Reliability for Health Service Accreditation A Literature Review. University of Tasmania, School of Business & Economics; 2014.

10  Greenfield D, Pawsey M, Naylor J, Braithwaite J. Are accreditation surveys reliable? Int J Health Care Qual Assur 2009; 22(2): 105-116.
| Crossref | Google Scholar | PubMed |

11  Greenfield D, Braithwaite J. Developing the evidence base for accreditation of healthcare organisations: a call for transparency and innovation. Qual Saf Health Care 2009; 18(3): 162-163.
| Crossref | Google Scholar | PubMed |

12  Australian Commission on Safety and Quality in Healthcare. Policy - Approval under the National General Practice Accreditation (NGPA) Scheme to conduct assessments. 2022. Available at https://www.safetyandquality.gov.au/publications-and-resources/resource-library/policy-approval-under-national-general-practice-accreditation-ngpa-scheme-conduct-assessments

13  StataCorp. Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC; 2021.

14  Erwin PC. A Self-Assessment Process for Accreditation Preparedness: A Practical Example for Local Health Departments. J Public Health Manag Pract 2009; 15(6): 503-508.
| Crossref | Google Scholar | PubMed |