Consumer involvement in health service research: a cross-sectional survey of staff in an Australian public hospital and health service

Laura Ryan; Laetitia Hattingh; Joan Carlini; Kelly A. Weir; Margaret Shapiro; Noela Baglot; Magnolia Cardona; Georgia Tobiano; Rachel Muir; Shelley Roberts; Sally Sargeant; Rachel Wenke

doi:10.1071/AH24186

RESEARCH ARTICLE (Open Access)

Consumer involvement in health service research: a cross-sectional survey of staff in an Australian public hospital and health service

Laura Ryan ^A ^* , Laetitia Hattingh ^A ^B ^C , Joan Carlini ^D ^E , Kelly A. Weir ^F ^G , Margaret Shapiro ^B , Noela Baglot ^B , Magnolia Cardona ^H ^I , Georgia Tobiano ^J ^K , Rachel Muir ^L ^M ^N , Shelley Roberts ^A ^O , Sally Sargeant ^P and Rachel Wenke ^A ^O ^Q

+ Author Affiliations

- Author Affiliations

^A Allied Health Research, Gold Coast Hospital and Health Service, Southport, Qld 4215, Australia.

^B School of Pharmacy and Medical Sciences, Griffith University, Qld 4222, Australia.

^C School of Pharmacy, The University of Queensland, Qld 4102, Australia.

^D Consumer Advisory Group, Gold Coast Hospital and Health Service, Southport, Qld 4215, Australia.

^E Department of Marketing, Griffith Business School, Griffith University, Gold Coast Campus, Qld 4111, Australia.

^F Melbourne School of Health Sciences, The University of Melbourne, Vic 3010, Australia.

^G The Royal Children’s Hospital Melbourne, Parkville, Vic 3052, Australia.

^H School of Population Health, The University of New South Wales, Sydney, NSW 2052, Australia.

^I Institute of Evidence Based Healthcare, Bond University, Robina, Qld 4226, Australia.

^J NHMRC CRE in Wiser Wound Care, Griffith University, Qld 4215, Australia.

^K Nursing and Midwifery Education and Research Unit, Gold Coast Hospital and Health Service, Southport, Qld 4215, Australia.

^L School of Nursing and Midwifery, Griffith University, Qld 4222, Australia.

^M Emergency Department, Gold Coast University Hospital, Gold Coast Hospital and Health Service, Southport, Qld 4215, Australia.

^N Florence Nightingale Faculty of Nursing Midwifery and Palliative Care, Kings College London, London, UK.

^O School of Health Sciences and Social Work, Griffith University, Gold Coast Campus, Southport, Qld 4222, Australia.

^P Faculty of Health, Southern Cross University, Gold Coast, Qld, Australia.

^Q Faculty of Health Sciences and Medicine, Bond University, Robina, Qld 4226, Australia.

^* Correspondence to: Laura.ryan2@health.qld.gov.au

Australian Health Review https://doi.org/10.1071/AH24186

Submitted: 4 May 2024 Accepted: 2 August 2024 Published: 26 August 2024

© 2024 The Author(s) (or their employer(s)). Published by CSIRO Publishing on behalf of AHHA. This is an open access article distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND)

Abstract

Objective

Consumer involvement is when patients (their families, friends, and caregivers) work with researchers on research projects. While health services are expected to support consumer–researcher collaborations, conducive environments still need to be developed, with limited research into how Australian health services support this practice. This study explores current consumer involvement in research activity and staff perceptions at an Australian tertiary hospital and health service.

Methods

Research-active employees at a health service were invited to participate in an online cross-sectional survey. It assessed experience with and confidence in involving consumers in research, the perceived value of consumer involvement, and considerations related to challenges, facilitators, and solutions to enhance consumer involvement in health service research.

Results

Among 83 respondents, including medical, nursing, and allied health professionals, 54 completed the survey. Approximately half had experience with involving consumers in research. Over 80% recognised that involving consumers enhanced research relevance. Identified barriers included challenges in finding (46.6%) and compensating (59.3%) consumers, as well as deficiencies in researcher skills, knowledge (32.2%), and time constraints (39%). Facilitators comprised access to experienced researchers and educational opportunities. Moreover, 87% advocated for increased support to involve consumers within health services, with the appointment of a designated consumer involvement officer deemed the most beneficial solution by 92.5% of respondents.

Conclusions

While individual barriers inhibit consumer involvement in research, more prominent institutional factors such as financial and technical support may determine successful and meaningful collaborations. As health services evolve towards co-design models in research, the outcomes of this study will guide initiatives aimed at enhancing consumer involvement in the research process within Australian health services.

Keywords: co-design, community participation, consumer, consumer engagement, consumer involvement, hospital, hospital and health service, patient involvement, patient participation.

Introduction

Consumer involvement in research is now considered fundamental for those pursuing knowledge generation in the health sector.¹ Consumer involvement in research is when research is conducted ‘by’ or ‘with’ health consumers (e.g. patients, friends, family, carers, paid service workers, or consumer groups) rather than ‘to’, ‘about’, or ‘for’ them.¹^–⁵ The role of consumers in health research is widely acknowledged, but terminology such as participation, involvement, engagement, and partnership is contested and used interchangeably.⁶^,⁷ In this Australian-based study, we use consumer involvement, reflecting its widespread usage.¹^,⁸^–¹² It refers to higher participation levels on the International Association of Public Participation (IAP2) Spectrum: involve, collaborate, and empower.¹³

Existing literature emphasises that involving consumers in research is important for three main reasons: (1) ensuring people’s right to contribute to knowledge affecting their lives, (2) improving research trustworthiness and relevance, and (3) enhancing research accountability and transparency.⁴^,¹⁴^–¹⁸ While consumer involvement is now endorsed by major Australian research councils and funding bodies, including the National Health and Medical Research Council (NHMRC),¹^,⁹^,¹⁹ translating this endorsement into practice necessitates a strategic commitment from organisations to involve consumers in research activities.

Our investigation is the initial phase of a larger project, encompassing a health service staff survey, group sessions with consumers and staff,²⁰ and a stakeholder workshop to gain multiple perspectives. The aim was to develop a framework for fostering meaningful consumer–researcher relationships within health service research initiatives. This paper presents findings from the first stage, focusing on surveying health service staff to explore their experiences and perspectives on involving consumers in research projects.

What consumer involvement in research activities is currently happening at the health service?
What are the staff perspectives about consumer involvement in research at the health service?

Method

Consumer involvement

Consumer involvement in this study is reported in consideration of the GRIPP2 reporting checklist²¹ (Supplementary material file S1). Three health service consumers were involved in all stages of this multi-stage project and are co-authors of this paper.

Study design

A cross-sectional survey design was used and is documented here as per the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting checklist for cross-sectional studies.²² A survey was chosen for data collection due to limited prior knowledge of the local context. Existing scholarship on consumer involvement in research guided survey development, eliminating the need for a preliminary exploratory step.²³

Participants and setting

The setting was a public hospital and health service in an Australian metropolitan area. A convenience sampling approach was used across all sites. The inclusion criteria allowed participation from current health service staff, including researchers, research-active clinicians, and those in management positions within relevant professional disciplines.

Recruitment

Promotional materials, including a project overview, survey links, and participant information sheets with ethical statements, were distributed electronically via health service platforms, professional networks, and directly to principal investigators of internal research projects. Participation was anonymous, and consent was obtained at the beginning of the survey.

Survey

Data was collected through an internal online survey. The comprehensive survey covered respondents’ experiences and perceptions of the value, facilitators, barriers, and improvement strategies within the health service. Demographics (gender, role, experience) were also included. The survey was created using SurveyMonkey, refined based on advisory team feedback, and pilot tested with two clinician-researchers for clarity. The survey featured Likert-scale and multiple-choice questions and included 27 questions: 7 on demographics, 16 on consumer involvement experiences, and 4 on participation in subsequent project stages. Questions on consumer involvement were guided by an established interview framework⁴ and the Theoretical Domains Framework (TDF).²⁴ Survey respondents were provided with a definition of consumer involvement in research to facilitate a clear understanding of the term throughout the survey (refer to Supplementary material file S2).

Data analysis

Survey responses were downloaded from SurveyMonkey and put into a Microsoft Excel spreadsheet for data analysis. Quantitative analysis (descriptive, frequencies, and percentages) was undertaken by two members of the research team and subsequently reviewed by remaining team members, including consumer advisors. Subgroup analyses were performed on professional groups. Statistical support was sought in consultation with the health service biostatistician for the quantitative components. Fisher’s exact test was used to understand relationships between professional discipline, knowledge, and previous experience involving consumers in research.

Ethics approval was obtained from Gold Coast Hospital and Health Service (GCHHS), Health Research Ethics Committee (HREC/2023/QGC/94895). Research governance approval was given by the Gold Coast Hospital and Health Service (SSA/2023/QGC/94895).

Results

Out of 90 initial respondents, 7 were removed for gross incompletion, 29 partially completed the survey, and 54 completed it in full. Partially completed surveys were included to maximise data utilisation and acknowledge respondent effort.²⁵ Completion rates varied, affecting the total responses per question. On average, respondents took 8 min and 32 s to complete the survey. Demographic details are provided in Table 1.

Table 1.Respondent demographics and professional experience.

Respondent demographics
Gender N = 76
Female	76.3% (58)
Male	23.68% (18)
Professional background N = 77
Medical	20.8% (16)
Nursing and Midwifery	29.9% (23)
Allied Health	19.5% (15)
Research	15.6% 12
Dual position (researcher and clinician)	3.9% 3
Other	10.4% 8
Health service experience N = 77
Less than 2 years	14.3% (11)
2–10 years	42.9% (33)
10–20 years	29.8% (23)
More than 20 years	13% (10)
Educational background (*multiple answers permitted) N = 77
Certificate/diploma	18.2% (14)
Bachelor’s degree	54.6% (42)
Honour’s degree	13% (10)
Master’s degree	42.9% (33)
Higher research degree	26% (20)
Other	13%
Research in current role N = 77
Full-time (38 or more hours per week)	16.9% (13)
Part-time (less than 38 h per week)	33.8% (26)
Occasionally (not on a weekly basis)	27.3% (21)
Never	16.9% (13)
Other (please specify)	5.2% (4)

Knowledge and experience of consumer involvement

The survey sought to determine the current landscape of involving consumers in research at the health service. Responses relating to self-reported knowledge about consumer involvement in research varied. Thirty-four respondents (48.6%) reported they knew how to involve consumers in their research, with 49.3% (n = 35) having previously engaged consumers. Thirty-six respondents (51.4%) either reported not knowing how to involve consumers in research (30%, n = 21) or were uncertain (21.4%, n = 15), with 46.5% (n = 33) having not previously involved consumers in their research. There were no statistically significant associations between professional discipline and either knowledge of (P = 0.437) or previous experience (P = 0.307) with the involvement of consumers in research.

Respondents identified their experience with specific components of the International Association for Public Participation (IAP2) Spectrum.¹³ Most often, research was driven by researchers, with consultation (62.1%, n = 18), involvement (62.1%, n = 18), and collaboration (34.5%, n = 10) with consumers. Only four respondents (13.8%) were involved in consumer-led projects.

In relation to the different stages of research, the majority of respondents had involved consumers in the conceptualisation and design of research (69%, n = 20). In comparison, fewer had experience in working with consumers to identify and prioritise research topics (21.1%, n = 7), implementation (20.7%, n = 6), and research evaluation (17.2%, n = 5).

Confidence in involving consumers

Findings relating to confidence in involving consumers in research are reported in Fig. 1. Areas of greater confidence when involving consumers in research included networking and consulting with consumers about research (49.2%, n = 32), identifying research issues and prioritising research goals (49.2%, n = 32), and disseminating research findings (50.8%, n = 33). Areas where respondents reported not feeling confident included recruitment of consumers (46.2%, n = 30), undertaking research funding applications with consumers (46.2%, n = 30), and monitoring and evaluating research impact with consumers (44.6%, n = 29).

Fig. 1.

Confidence in undertaking consumer involvement activities.

Valuing consumer involvement

Respondents placed a high value on the involvement of consumers in research and understood the benefits of involving consumers in their research (refer to Fig. 2). The majority (88.5%, n = 54) agreed that consumers brought ‘a lived experience’ and other expertise to the research team, enhancing the relevance of research to the target population. Furthermore, 86.9% (n = 53) reported that consumers increased the credibility of research within consumer groups, and 82% (n = 50) agreed that the involvement of consumers leads to more culturally responsive research. Finally, many respondents agreed that the involvement of consumers is often tokenistic (50.8%, n = 31).

Fig. 2.

Valuing consumer involvement.

Challenges to consumer involvement

Fig. 3 reports challenges to involving consumers in research. Key challenges in involving consumers in research included difficulty finding consumers (46.6%, n = 27), lack of funds to pay consumers for their time (59.3%, n = 35), and staff not having the relevant skills and knowledge to involve consumers in research (32.2%, n = 19). Eight (13.8%) respondents reported difficulty in managing power dynamics within the research team where they had involved consumers. Finally, 58.3% (n = 35) of respondents felt it was difficult to involve consumers in research at the health service where the study took place.

Fig. 3.

Challenges to consumer involvement.

Facilitators of consumer involvement

Perceived facilitators of involving consumers in research are reported in Fig. 4. Respondents were uncertain about the factors facilitating consumer involvement in research within the health service. This uncertainty was evidenced by the prevalence of neutral responses – indicating neither agreement nor disagreement – being frequently chosen in relation to the statements provided. Only 14.8% (n = 8) of respondents reported that the health service adequately supported researchers in involving consumers, while 86.8% (n = 46) thought that more should be done. Interestingly, 84.9% of respondents agreed that people whose opinions they valued approved of the involvement of consumers in their research. Key facilitators assisting respondents with involving consumers included having a more experienced researcher in the research team (51.9%, n = 27) and education and training (40.4%, n = 21).

Fig. 4.

Facilitators of consumer involvement.

Solutions to consumer involvement

Respondents supported all the proposed solutions to aid the involvement of consumers in research (refer to Fig. 5). However, a designated consumer involvement officer was viewed as the most useful solution for supporting them to involve consumers in research (92.5%, n = 49), closely followed by information linking researchers to external (87%, n = 47) and internal (85.2%, n = 46) training. Prompts in the health service’s research protocol template and ethics application form were viewed as less helpful solutions; however, 64.8% (n = 35) of respondents still agreed this would help.

Fig. 5.

Solutions to consumer involvement.

Discussion

This study provides insight into consumer involvement in research within an Australian hospital and health service context. The survey explored staff experiences and perceptions, including competencies, concerns, and opportunities regarding consumer involvement. This study focuses on a specific organisational context unlike prior studies covering diverse health organisations and institutions.³^,²⁶ It offers guidance for Australian health services to strategically address challenges in consumer–researcher collaborations, enhancing meaningful consumer involvement through targeted efforts and resource allocation.¹⁴

Approximately half of the surveyed staff had experience and felt knowledgeable about involving consumers in research, while the other half had limited exposure or understanding. This aligns with other recent research indicating a general lack of familiarity and expertise among health service researchers with involving consumers in research.³^,²⁷ This trend highlights the widespread challenge of limited exposure and expertise among health service staff in involving consumers in research. Addressing this is crucial for targeted interventions, training, and support systems to enhance consumer involvement in health service research, necessitating organisational frameworks and budgetary planning.

Our findings show that health service staff value consumer involvement in research and recognise the benefits. However, there was also an acknowledgment of tokenistic involvement, highlighting a growing awareness of the need for more meaningful engagement. These findings contribute to evidence showing a shift in health researcher attitudes towards greater consumer involvement.²⁷ While efforts to promote consumer involvement in health service research can leverage these acknowledged benefits,¹⁴^–¹⁶ it is important to consider that staff with reservations may not have participated in our survey. Addressing and possibly changing perspectives within this subgroup may require additional initiatives.

Survey respondents appreciated the benefits of involving consumers in research; however, significant challenges for health service staff were evident. These align with barriers observed in Australian health research, including limited project resources, recruitment difficulties, staff skills gaps, resistance to involvement, and organisational constraints.³^,¹⁸^,²⁸^–³⁰ Addressing these barriers demands systemic support, without which full consumer involvement in research appears improbable.

Power struggles can hinder consumer involvement in research when researchers resist sharing authority.¹⁸^,³⁰^–³² Power struggles may emerge when researchers feel threatened by the possibility of losing control over the research process and are reluctant to share authority in generating knowledge.³³ Some researchers include consumers superficially, while unconscious biases may undervalue their contributions.³⁴ Both parties benefit from recognising and leveraging consumers’ unique social capital to balance power dynamics and improve research outcomes.³⁵ Respondents to our survey may have overlooked power dynamics in teams with consumers, focusing instead on more pressing challenges such as consumer–researcher connection and compensation. This oversight might stem from a lack of awareness about power issues in these collaborations, hindering efforts to address and resolve such disparities.³⁵

The survey identified opportunities for health services to help staff overcome obstacles in consumer involvement. Respondents supported organisational backing, highlighting the need for enhanced support in building consumer relationships. These needs align with existing literature, underscoring the importance of general support, access to external training, consumer recruitment assistance, compensation funding, and extended project timelines.³^,⁴^,²⁷^,³⁶ Research into the efficacy of these strategies is evolving. For example, a 2016 study on a consumer involvement education program showed increased understanding among participants.³⁷ However, other studies have called for more tailored training programs for researchers and greater transparency in content development methods.³⁸ This emphasises the importance of adaptive strategies and ongoing assessment to meet evolving needs.

Limitations and strengths

Out of 90 initial participants, 54 completed the survey, with 30 dropping out before question 10 of 27. The survey, approved by research and advisory teams and pilot tested, averaged an 8 minute completion time, aligning with engagement guidelines,³⁹ although this includes incomplete responses. Respondents less familiar with consumer involvement may have found detailed questions challenging or irrelevant. Another limitation is the single-site focus, which limits generalisability. This study benefited from insights from health professionals from diverse specialties across various practice areas within the health service. It also had a diverse team of health service researchers, university academics, and consumers providing varied perspectives.

Conclusion

As Australian health staff and organisations face growing pressure to involve consumers in research, understanding the Australian health service context is crucial. The study provides considerations for Australian health services to strategically enhance efforts and allocate resources to address the evolving demands and challenges of consumer–researcher collaborations.

Supplementary material

Supplementary material is available online.

Data availability

The data that support this study cannot be publicly shared due to ethical or privacy reasons and may be shared upon reasonable request to the corresponding author if appropriate.

Conflicts of interest

The authors declares no conflicts of interest with respect to the research, authorship, and/or publication of this article.

Declaration of funding

This research was made possible thanks to the generous support provided by a Small Research Project Grant from the Gold Coast Health Study, Education, Research Trust Account (SERTA).

Acknowledgements

We acknowledge and pay respects to the people of the Yugambeh language region of the Gold Coast and all their descendants both past and present. We also acknowledge the many Aboriginal people from other regions, as well as the Torres Strait and South Sea Islander people who now live in the local area where this project took place. We thank the staff who participated in this study which enabled us to do this work. Our gratitude is extended to biostatistician Dr Ian Hughes for providing valuable consultation during aspects of this project. We acknowledge Gold Coast Health’s Allied Health Research Department for its in-kind support of the project. Finally, we acknowledge the use of ChatGPT-4, an AI language model developed by OpenAI and Grammarly, for grammar and spell-checking and to enhance the clarity and readability of the text.

Author contributions

All the named authors (LR, LH, JC, KW, MS, NB, MC, GT, RM, SR, SS, and RW) were involved in conceptualising and designing the study. LR collected the data. LR and LH analysed the data. All the named authors contributed to the interpretation of the data. LR drafted the manuscript with input from LH, JC, KW, MS, and RW. All authors reviewed the manuscript and gave approval for the final version of the manuscript.

References

1 National Health and Medical Research Council. Statement on consumer and community involvement in health and medical research. National Health and Medical Research Council; 2024.

2 Jackson T, Pinnock H, Liew SM, Horne E, Ehrlich E, Fulton O, et al. Patient and public involvement in research: from tokenistic box ticking to valued team members. BMC Med 2020; 18(1): 79.
| Crossref | Google Scholar | PubMed |

3 Ayton D, Braaf S, Jones A, Teede H, Gardam M, Bergmeier H, et al. Barriers and enablers to consumer and community involvement in research and healthcare improvement: Perspectives from consumer organisations, health services and researchers in Melbourne, Australia. Health Soc Care Community 2022; 30(4): e1078-91.
| Crossref | Google Scholar | PubMed |

4 Miller CL, Mott K, Cousins M, Miller S, Johnson A, Lawson T, et al. Integrating consumer engagement in health and medical research - an Australian framework. Health Res Policy Syst 2017; 15(1): 9.
| Crossref | Google Scholar | PubMed |

5 Ocloo J, Matthews R. From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Qual Saf 2016; 25(8): 626-32.
| Crossref | Google Scholar | PubMed |

6 Rise MB, Solbjør M, Lara Cabrera ML, Westerlund H, Grimstad H, Steinsbekk A. Same description, different values. How service users and providers define patient and public involvement in health care. Health Expect 2013; 16(3): 266-76.
| Crossref | Google Scholar | PubMed |

7 Tritter JQ. Revolution or evolution: the challenges of conceptualizing patient and public involvement in a consumerist world. Health Expect 2009; 12(3): 275-87.
| Crossref | Google Scholar | PubMed |

8 Cancer Australia and Cancer Voices Australia. National Framework for Consumer Involvement in Cancer Control. Canberra: Cancer Australia and Cancer Voices Australia; 2014.

9 Australian Government Department of Health and Aged Care. Medical Research Future Fund Consumer Reference Panel; 2023. Available at https://www.health.gov.au/committees-and-groups/medical-research-future-fund-consumer-reference-panel.

10 Australian Health Research Alliance. Consumer and Community Involvement in Health and Medical Research: An Australia-wide Audit. Australian Health Research Alliance; 2018.

11 Australian Clinical Trials Alliance. Why involve consumers? Australian Clinical Trials Alliance; 2023. Available at https://involvementtoolkit.clinicaltrialsalliance.org.au/toolkit/understanding/why-involve-consumers/

12 Gunatillake T, Shadbolt C, Gould D, Lam M, Hearst MG, Vleeskens C, et al. Embedding consumer and community involvement within an established research centre: moving from general recommendations to an actionable framework. Res Involv Engagem 2020; 6(1): 64.
| Crossref | Google Scholar | PubMed |

13 International Association for Public Participation (IAP2). Quality Assurance Standard. International Association for Public Participation; 2015.

14 Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot. Health Expect 2019; 22(4): 785-801.
| Crossref | Google Scholar | PubMed |

15 Modigh A, Sampaio F, Moberg L, Fredriksson M. The impact of patient and public involvement in health research versus healthcare: A scoping review of reviews. Health Policy 2021; 125(9): 1208-21.
| Crossref | Google Scholar | PubMed |

16 Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: A systematic review. BMC Health Serv Res 2014; 14(1): 89.
| Crossref | Google Scholar | PubMed |

17 Minogue V, Cooke M, Donskoy AL, Vicary P, Wells B. Patient and public involvement in reducing health and care research waste. Res Involv Engagem 2018; 4: 5.
| Crossref | Google Scholar | PubMed |

18 Anderst A, Conroy K, Fairbrother G, Hallam L, McPhail A, Taylor V. Engaging consumers in health research: A narrative review. Aust Health Rev 2020; 44(5): 806-13.
| Crossref | Google Scholar |

19 Australian Clinical Trials Alliance. Consumer Involvement in Clinical Trials: Consultation Report; 2019. Available at https://clinicaltrialsalliance.org.au/wp-content/uploads/2019/09/ACTA_Consumer-Involvement_Consultation-Report_final_050719.pdf

20 Ryan L, Wenke R, Carlini J, Weir KA, Shapiro M, Baglot N, et al. Exploring barriers and solutions to consumer involvement in health service research using a nominal group technique. Res Involv Engagem 2024; 10(1): 72.
| Crossref | Google Scholar | PubMed |

21 Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem 2017; 3: 13.
| Crossref | Google Scholar | PubMed |

22 von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. Epidemiology 2007; 18(6): 800-4.
| Crossref | Google Scholar | PubMed |

23 McGregor SLT Understanding and evaluating research: a critical guide. Los Angeles: SAGE Publications, Inc; 2018.

24 Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005; 14(1): 26-33.
| Crossref | Google Scholar | PubMed |

25 Graham JW. Missing data analysis: Making it work in the real world. Annu Rev Psychol 2009; 60(1): 549-76.
| Crossref | Google Scholar | PubMed |

26 Saunders C, Girgis A. Status, challenges and facilitators of consumer involvement in Australian health and medical research. Health Res Policy Syst 2010; 8(1): 34.
| Crossref | Google Scholar | PubMed |

27 Carlini J, Muir R, McLaren-Kennedy A, Grealish L. Researcher Perceptions of Involving Consumers in Health Research in Australia: A Qualitative Study. Int J Environ Res Public Health 2023; 20(10): 5758.
| Crossref | Google Scholar | PubMed |

28 McKenzie A, Bulsara C, Haines H, Hanley B, Alpers K. Barriers to community involvement in health and medical research - Researchers perspectives on consumer and community involvement in research: a qualitative study. The University of Western Australia School of Population Health, Telethon Kids Institute and The University of Notre Dame; 2016.

29 Payne JM, D’Antoine HA, France KE, McKenzie AE, Henley N, Bartu AE, et al. Collaborating with consumer and community representatives in health and medical research in Australia: results from an evaluation. Health Res Policy Syst 2011; 9(1): 18.
| Crossref | Google Scholar | PubMed |

30 Happell B, Gordon S, Bocking J, Ellis P, Roper C, Liggins J, et al. “Chipping away”: non-consumer researcher perspectives on barriers to collaborating with consumers in mental health research. J Ment Health 2019; 28(1): 49-55.
| Crossref | Google Scholar | PubMed |

31 Ocloo J, Garfield S, Franklin BD, Dawson S. Exploring the theory, barriers and enablers for patient and public involvement across health, social care and patient safety: a systematic review of reviews. Health Res Policy Syst 2021; 19(1): 8.
| Crossref | Google Scholar | PubMed |

32 Bird M, Ouellette C, Whitmore C, Li L, Nair K, McGillion MH, et al. Preparing for patient partnership: A scoping review of patient partner engagement and evaluation in research. Health Expect 2020; 23(3): 523-39.
| Crossref | Google Scholar | PubMed |

33 Hacker KE, Smith AB. Engaging Stakeholders and Patient Partners. Surg Oncol Clin N Am 2018; 27(4): 665-73.
| Crossref | Google Scholar | PubMed |

34 Richards DP, Poirier S, Mohabir V, Proulx L, Robins S, Smith J. Reflections on patient engagement by patient partners: how it can go wrong. Res Involv Engagem 2023; 9(1): 41.
| Crossref | Google Scholar | PubMed |

35 Locock L, Boylan AM, Snow R, Staniszewska S. The power of symbolic capital in patient and public involvement in health research. Health Expect 2017; 20(5): 836-44.
| Crossref | Google Scholar | PubMed |

36 McKenzie A, Bowden J, Zalcberg JR, Conroy K, Fallon-Ferguson J, Jesudason S, et al. A snapshot of consumer engagement in clinical trials in Australia: results of a national survey of clinical trial networks and research organisations. Res Involv Engagem 2022; 8(1): 3.
| Crossref | Google Scholar | PubMed |

37 McKenzie A, Alpers K, Heyworth J, Phuong C, Hanley B. Consumer and community involvement in health and medical research: evaluation by online survey of Australian training workshops for researchers. Res Involv Engagem 2016; 2(1): 16.
| Crossref | Google Scholar | PubMed |

38 Reeder S, Ayton D, Teede HJ, Skouteris H, Simmons M, Jones A. Opportunities to enhance consumer and community engagement training for researchers and healthcare providers: A qualitative study. BMJ Open 2023; 13(9): e073114.
| Crossref | Google Scholar |

39 Galesic M, Bosnjak M. Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey. Public Opin Q 2009; 73(2): 349-60.
| Google Scholar |