Perceptions of research capacity in public health organisations: comparison of NSW metropolitan and non-metropolitan Local Health Districts

Keywords: individual, organisational, perceptions, regional, research capacity, research culture, rural, team.

Populations in rural, regional and remote areas (non-metropolitan) are associated with poorer health outcomes, higher level of disease and injury, and shorter lives when compared to their metropolitan counterparts.¹ Almost 30% of the Australian population lives outside major cities,² with a higher proportion of people aged >65 years in inner and outer regional areas.¹ Living outside of major cities is associated with higher risk of adverse health behaviours.² These factors contribute to the challenges facing healthcare systems to service these communities, and highlight the role of research to inform programs to address adverse health behaviours.

Non-metropolitan health services differ greatly from those in metropolitan areas, presenting unique challenges for service delivery. These differences include, but are not limited to: (a) inferior access to, and use of, health services;^2–7 (b) planning and delivery models that may not be appropriate for the communities;⁴ (c) a need to deliver services and information remotely;⁸ and (d) workforce challenges, including recruitment and retention difficulties.^9,10 Both collectively and singularly, these issues highlight the importance of utilising research and innovative practices to confront the variations to deliver suitable health care.

Exploration of innovative approaches to the provision of health care in non-metropolitan areas require access to funding. A review conducted in 2018 showed the proportion of grant funding from the National Health and Medical Research Council (NHMRC) specifically aimed to deliver health benefits to people living in rural or remote Australia was only 2.4%.¹¹ Considering the service delivery challenges and poorer health outcomes of non-metropolitan area populations, Barclay et al. suggested this should be a priority for NHMRC funding.¹¹ Geographical challenges of forming collaborative relationships between healthcare services and universities and/or medical research institutes may be a factor.

There are key differences between metropolitan and non-metropolitan public healthcare organisations within Australia⁹ and internationally.¹² One is the capability and capacity to build and embed a research culture. Therefore, the aims of this study were to explore the perceptions of research capacity and culture (RCC) in New South Wales (NSW) Local Health Districts (LHDs) across the organisational, team and individual domains, and to determine whether there was a difference between metropolitan and non-metropolitan LHD locations.

This study was a cross-sectional survey study design that aimed to recruit staff from all LHDs across NSW, Australia. A broad range of public health services are provided by NSW LHDs through hospitals and community health centres. Collectively, NSW LHDs deliver public health care to 7.9 million residents and employs almost 120 000 full-time equivalent staff; the largest in Australia.¹³

Ethical approval for this study was obtained by the Greater Western Human Research Ethics Committee (2019/ETH03990) and Southern Cross University (ECN-19-049). Site research governance authorisation was obtained from each participating LHD.

The target population was all research-eligible NSW LHD employees. For the purposes of this study, the term research-eligible included staff who held either a clinical position and therefore their practice is guided by best available evidence (i.e. nursing, midwifery, allied health professionals (AHPs) and medical staff) or employed under the Health Manager (HM) award. The HM award is a broad range of appointments, which may include roles on specific programs and projects or within management/executive roles. There are 15 LHDs in the NSW public health system, eight in metropolitan areas and seven in non-metropolitan areas.

The Research Capacity and Culture Tool (RCCT)¹⁴ is a validated tool that has been widely used to explore the perceptions of RCC within the healthcare contexts. The RCCT measures participants’ subjective perspectives of RCC at the organisational (18 items), team (19 items) and individual (14 items) domains. Items in each domain are scored on a Likert response scale, with 1 representing the lowest rating of success/skill level and 10 the highest. Three studies have reported excellent reliability, with Cronbach’s α > 0.9 for all domains.^14–16 Exploratory factor analysis in the initial validation paper¹⁴ demonstrated one factor per domain. Subsequent studies, however, have identified two factors for each of the organisational and team domains.^15,16

This study was conducted across multiple NSW LHDs between 7 May and 31 July 2019. A key contact person from each LHD was identified to ‘champion’ the project. Links to the online survey (hosted by Qualtrics (Qualtrics.com)) for the RCCT were sent to these contacts to be distributed via that LHD’s promotional vehicle (e.g. email to staff, website or newsletter). Four reminder emails were sent to the key contacts during the recruitment period. Additional methods of promotion included social media (Facebook) and the NSW Nurses and Midwives’ Association (NSWNMA).

All data were analysed using Statistical Package for the Social Sciences (SPSS) version 27 for Mac.¹⁷ Likert-scale items were summarised for each domain by mean and standard deviation and the overall domain mean, including the 95% confidence intervals. Frequencies and percentages were used to present categorical variables and unsure responses. The means of items and the domains were categorised in accordance with previous users of the RCCT as low (<4.00), medium (4.00–6.99) and high (≥7.00).^{15,16,18–22} The differences between the domain means of the metropolitan and non-metropolitan subgroups were examined using the Independent-samples t-test (t-test).

Data from 1243 participants were available for investigation. Participant flow through the survey is shown in Fig. 1. Most participants (77%) were based in NSW non-metropolitan areas when compared to metropolitan areas (n = 953 vs n = 290). Nurses and AHPs accounted for most participants in the sample (74.7% combined), as shown in Table 1. Of the 839 participants who responded to the highest qualifications question, 14.7% had a research higher degree (27.6% metropolitan vs 10.3% non-metropolitan). Approximately double the percentage of participants from metropolitan locations than non-metropolitan reported they had research within their position description (40.6% vs 22.0%) or had participated in any research activities within the last 12 months (58.7% vs 27.7%).

Fig. 1.  Flow of participants through the survey.

Table 1.  Characteristics of the participants.

The sample means for the organisational and individual domains were classified as medium, whereas the team domain was low (Table 2). The t-tests indicated significantly higher means for all domains for participants in metropolitan compared with non-metropolitan LHDs (P < 0.001) (Table 2). The metropolitan LHD means were significantly higher for 13 of the 18 items for the organisational domain (Table 3), 17 of the 19 items for the team domain (Table 4) and all items in the individual domain (Table 5).

Table 2.  Independent samples t-test comparing subgroups – organisational domain.

Table 3.  Mean differences and t-tests between geographical subgroups – organisational domain.

Table 4.  Mean differences and t-tests between geographical subgroups – team domain.

Table 5.  Mean differences and t-tests between geographical subgroups – individual domain.

In summary, the domain means for the sample (NSW) and for each subgroup are displayed in Fig. 2. Overall, for all domains, non-metropolitan participants selected a higher proportion of unsure responses than their metropolitan counterparts. Additional details are available in the supplementary data tables (Supplementary Tables S1–S3). Further graphical representations of each of the domains and subgroups are available in the supplementary material (Supplementary Figs S1–S3).

Fig. 2.  Overall domain means for the sample (NSW) and by geographical subgroup (metropolitan/non-metropolitan).

Metropolitan NSW LHD staff had significantly higher perceptions of RCC than those from non-metropolitan areas across all three domains. Only two other studies have compared metropolitan with non-metropolitan areas in Victoria²³ and Queensland,²⁴ both finding a metropolitan location to influence RCC. Skills in research are vital in non-metropolitan areas to develop and test innovative solutions directly relevant to the local context, with the aim of addressing the disparity of health behaviours and outcomes between the two locations.²⁵ Therefore, these results further add to the evidence of the gap of perceived RCC and suggests this issue may be widespread across the Australian public healthcare context.

Formalised conjoint appointments between universities and healthcare organisations and dedicated embedded clinical research positions have been shown to provide access to experienced researchers, research skills development, opportunities to participate in research activities,^26–28 knowledge brokering activities²⁹ and the encouragement of post-graduate qualifications.³⁰ Additionally, collaboration between academia and industry is increasingly becoming a requirement of funding bodies such as the NHMRC and NSW Ministry of Health, with the view of improved relevance and translatability of findings.^31,32 Such relationships may afford access to additional funding opportunities and the delivery of training to build research capacity, culture, and capability within healthcare organisations. However, the measure of success for these positions is problematic, with reports focusing on traditional academic research outputs (e.g. building collaborations and increasing research activities), rather than the translation of findings into practice.³³ This may be because knowledge brokering,²⁹ translational and implementational activities are more complex to measure. Furthermore, concerns regarding reporting lines, role expectations and measure of research outputs have been raised.³³ Regardless of the type of appointment, these embedded positions have potential to deliver benefits for both organisations and ultimately patient outcomes, but should be clearly defined and structured at the outset.

Frameworks for research capacity building (RCB) invariably include the development or enhancement of research knowledge, skills, and confidence.^34–38 The skills required to conduct research lie at the individual level. However, to build those skills, strategies need to be implemented at the team and/or organisational level. Building skills and confidence of individuals may require a needs assessment to ascertain the current level and gaps.³⁹ At the team level, strategies to address gaps may include mentorship from more experienced researchers within the team to transfer knowledge and foster a culture of research.⁴⁰ Non-metropolitan areas, however, face challenges with geographical spread of staff and facilities³⁹ and sufficient experienced clinician-researchers to provide this mentoring function. Organisationally, a dedicated budget to develop targeted training based on gaps may be implemented to increase individual skills. Structured programs such as the Rural Research Capacity Building Program⁴¹ addresses many of the resource and mentoring limitations^25,42 for non-metropolitan healthcare workers; however, it is limited in its reach both geographically and to the number of participants it can accommodate.

Considering the findings from this study, both metropolitan and non-metropolitan participants perceived a lack of sufficient support for research in the form of funding, software, equipment and administration at both the organisational and team level. Participants also rated a career pathway in research to be low at the organisational level. At the team level, metropolitan participants perceived incentives and support for mentoring activities to be the third lowest, whereas non-metropolitan participants rated team level planning for research development as the third lowest. A focus on these identified strategies at both the organisational and team level may well help to build individual skills and foster a culture of research within teams and across the organisation.

This is the largest of the current literature to use the RCCT in Australia. All but three^27,43–45 studies using this tool were conducted in Australia. This study was significant in its reach across one state within Australia, NSW, and its perspective of capturing multiple professions and the comparison of metropolitan with non-metropolitan public healthcare organisations. Health system factors in other states may differ resulting in contrasting findings.

Limitations include the sampling method, which aimed to recruit all eligible staff but ultimately resulted in a non-random sample with self-selection of participants. Those LHDs that distributed the email invitation to all staff rather than relying on accessing the link from another site had higher participation levels. The NSWNMA was approached to broaden the reach of participants but no other professional associations were contacted, which may have limited recruitment.

Selection bias is a frequently reported limitation in studies using the RCCT,^20,22,24 with those with an interest in research more likely to respond. Furthermore, the response rate was unable to be calculated as the denominator of research-eligible staff exposure to the invitation was not able to be accurately determined. Additionally, not all metropolitan LHDs participated in this survey. The RCCT had recently been used within several metropolitan LHDs, which may account for the lack of interest from participating LHDs to subject staff to the survey again.^15,46,47

Future studies may explore alternative sampling methods for recruitment of participants to address the sampling bias. Targeting recruitment to select staffing groups and/or random sampling may enable the calculation of a response rate and may increase generalisability of results. However, across organisations, this method would likely require a greater level of local administrative support to the lead researcher due to privacy concerns and access to staff email lists.

Further exploration of the differences between and within professional groups may provide useful insight. This may assist targeting capacity-building strategies based on the individual needs of these groups. Future research should consider the use of interviews and/or focus groups to intensify the understanding of the concepts and findings from this study.

The findings of the current research may provide a benchmark for a longitudinal study across NSW LHDs of RCB strategies. This would be particularly helpful for non-metropolitan LHDs where strategies generally are still in their relative infancy in comparison to larger metropolitan LHDs. Additionally, studies may explore the influence of participating in research activities as a recruitment and retention strategy, which has recently been shown by Cosgrave.⁴⁸ Further and more broadly, longitudinal studies may provide insight into the relationships between research capacity and research activity over time.

Within this sample, NSW LHD staff rated their own level of research skill higher when compared to the perception of RCC at both the organisational and team level. Further, staff from metropolitan LHDs perceived the level of RCC across all domains to be higher than non-metropolitan LHDs. These important findings provide preliminary empirical evidence that a research culture in non-metropolitan is lagging behind metropolitan LHDs.

The implications of the findings of this research suggest that an overall approach to research capacity development strategies should be holistic in seeking to integrate and target individual, team, organisational and supra-organisational or systems (i.e. state) levels. Factors that were found to be consistently low across the state, such as resources to support research activities and career pathways, may be suggestive of prioritisation for further investigation and implementation at the state and/or national level.

Supplementary material is available online.

The data that support this study will be shared upon reasonable request to the corresponding author.

The authors declare that they have no conflicts of interest.

This research did not receive any specific funding.