The contribution of Primary Health Care Research, Evaluation and Development-supported research to primary health care policy and practice

Additional keywords: document review, health reform, general practice, research capacity building, rural health.

Primary health care (PHC) research informs clinical practice, improves system performance and patient care, promotes critical thinking, encourages multidisciplinary collaboration and supports health (Del Mar and Askew 2004; Mant et al. 2004). However, while family medicine and general practice publication rates are on the rise, they remain low compared with the size of the workforce (Del Mar and Askew 2004; Askew et al. 2008; Mendis et al. 2010). In addressing this discrepancy between the extensive PHC workforce and the limited research output from this cohort, the availability of support for PHC research must be addressed. The key components of support include both building research capacity and providing financial resources. For example, there is some indication that clinical, hospital and laboratory research receive greater funding than PHC (McAvoy 2005). From 2000 to 2008 the National Health and Medical Research Council provided Australian research funding of over $3.5 billion, only 1.9% of which was provided to PHC (McIntyre et al. 2011; National Health and Medical Research Council 2012). In order to advance PHC in terms of improving patient outcomes, developing the knowledge base, and informing health policy, it is vital that more support is offered to researchers.

In 2000, the Australian government Department of Health and Ageing established the Primary Health Care Research, Evaluation and Development (PHCRED) Strategy to build PHC research capacity and produce high-quality research. In the first 10 years the strategy comprised four key components: the Australian PHC Research Institute, leading and managing policy-relevant research; the PHC Research and Information Service (PHC RIS), responsible for supporting dissemination and knowledge exchange; a program of training awards and grants associated with the National Health and Medical Research Council; and the Research Capacity Building Initiative (RCBI), providing financial support and training opportunities in university departments.

Phase one of the PHCRED Strategy (2000–05) focussed on building capacity through development of researchers and infrastructure, and promoting evidence-based practice. Phase two (2006–10) sought to expand the pool of researchers and offer support to them to conduct high-quality research relevant to, and able to inform, policy and practice. In both phases, university departments of general practice and rural health were funded under the RCBI (http://www.phcris.org.au/phcred/rcbi.php) to support these objectives. Phase three (2010–14) will continue to build and communicate an evidence base around the National PHC Strategy themes led by the Australian PHC Research Institute and PHC RIS.

The National PHC Strategy is one of the Australian government’s schemes introduced in 2008 to promote National Health Reform, along with the National Health and Hospitals Reform Commission and the Preventative Health Taskforce. Kalucy and Jackson-Bowers (2010, 2009) investigated PHC citations included in the final reports of these three initiatives highlighting the valuable nature of credible, accessible literature for evidence-informed policy making. Similarly, Van Der Weyden (2008) described Australia’s need for health reform but the much greater need for comprehensive evidence relevant to, and in support of, reform. Based on the chronology of events it seems that some PHCRED-supported research may have informed key National PHC Strategy issues, while later papers may have been a result of the health reform topics. The themes of relevance to the National PHC Strategy include four main priority areas and five building blocks (Table 1).

Table 1.  National Primary Health Care (PHC) Strategy coding: labels and definitions
Adapted from the Department of Health and Ageing (2010, p. 12)

Key players in phases one and two of the PHCRED Strategy have been researchers based in Australian university departments of general practice and rural health. Funding from this strategy (the basis of the RCBI) was provided to departments to increase the number and range of people with knowledge and skills in PHC evaluation and research (McIntyre et al. 2010), including allied health professionals, early career researchers (Ried et al. 2007), and both academic and clinical practitioners (Yen et al. 2010). Initially each department was provided with a $200 000 grant (Oceania Health Consulting 2005). This funding promoted infrastructure and capacity-building support in these departments (McIntyre et al. 2007). It enabled researchers to conduct low-cost research projects and use the funding as leverage to obtain additional support.

The RCBI has increased research awareness and enabled a research culture that is missing in PHC in other parts of the world, where researchers experience a lack of training and opportunities to be involved in research activities (Glynn et al. 2009; Yen et al. 2010). Barriers to successful research have included lack of infrastructure, scheduled research time, remuneration, recognition, funding, education, skills, support, opportunity for and adequacy of supervision, and challenges with workforce shortages, recruiting researchers and retaining high-level academics (Mant et al. 2004; Hancock and Wilson 2006; Yallop et al. 2006; Ried et al. 2007; Glynn et al. 2009). Del Mar and Askew (2004) described successful interventions for building research capacity in a health context as those that

monitor the output of research … increase the number of journals, encourage and enable ... skills acquisition (including making it part of professional training), strengthen the academic base, and promote research networks and collaborations. (p. S35)

Additional sources have indicated that building research capacity includes a whole-system approach, appreciation for diversity, reducing barriers, promoting collaboration and mentoring, enhancing existing skills and teaching new research techniques (Dunbar et al. 2002; Farmer and Weston 2002; Hay et al. 2012). McGrail et al. (2006) discussed the benefits of writing courses, support and coaching to improve publication rates among researchers. In a climate of ‘publish or perish’ (McGrail et al. 2006), PHCRED-supported researchers in Australia are offered assistance with all of these aspects (Yen et al. 2010).

The current study investigates the impact of phase two of the PHCRED Strategy by analysing peer-reviewed publications from PHCRED-supported departments from 2006 to 2010. While bibliometric analyses are not uncommon in PHC, the current research offers a novel approach by extracting data from self-reported annual reviews. The study explored the volume of research output and determined which PHC issues the research focussed in relation to Australia’s current health reform and priority areas and building blocks of the National PHC Strategy (Table 1). In addition, the inclusion of social factors in models of PHC highlights the importance of considering differences between general practice (urban) and rural health units. While there has been some data published about the RCBI activities of PHCRED-supported departments (McIntyre et al. 2007), overall analysis of the content of the research has not been undertaken.

The authors performed a document review of annual reports from the 26 PHCRED-supported university departments of general practice (n = 14) and rural health (n = 12), for the years 2006 through 2010. These are self-reported, requirements of the funding agreement, set up for accountability purposes. Data was confined to phase two when a template for the annual reports was implemented that included providing information about outputs such as peer-reviewed publications. The departments involved gave permission for the data to be used in a non-identifiable, aggregated form. While data was not always complete and often required intensive data cleaning, information about published papers included various fields. First, bibliographic details were presented: author, organisation, publication title and citation. Second, organisations identified the role of the RCBI, e.g. funding, mentoring or writing group. Finally, an indication was provided as to whether the paper denoted a first publication for any of the authors.

Frequency analysis was performed on the demographic data of the publications. The first author also compiled an EndNote bibliographic database of all publications in peer-reviewed journals and conducted content analysis, categorising the publications according to type of publication or research method, role of RCBI, and National PHC Strategy themes based on title and abstract. Qualitative content analysis is useful in both using counting to describe the patterns that emerge in text data and encouraging interpretation of these patterns (Miller and Crabtree 1992; Morgan 1993). It is particularly beneficial when engaging in comparative analysis (Morgan 1993), as conducted for the rural and urban groups in the current research. To reduce the likelihood of coding errors (Glanville et al. 2011), a sample consisting of 12% of the publications was analysed by independent coders. These coders were provided with training and guidelines around the health reform themes to be selected in coding each paper. Coders were asked to analyse a blind database consisting only of the papers’ bibliographic details and abstracts. Moderate to strong agreement was recorded with 89% agreement on whether or not health reform categories could be observed in the publications, and 65% agreement on the specific codes.

In the period from 2006 to 2010, 661 papers were published in 212 journals (Fig. 1). Over half (53.5%) of all papers were published in 11 journals (Table 2). Fig. 2 displays the number of publications produced in each state, with New South Wales/Australian Capital Territory recording the highest figure (28.7% of all publications), but also the highest number of PHCRED-supported units (n = 7). The ‘urban’ areas reported 70.3% (n = 465) of the total publications in comparison with 29.7% (n = 196) from rural departments.

Fig. 1.  Number of publications between 2006 and 2010. Publications accepted during 2006–10 but not published until 2011 have been excluded.

Table 2.  Most popular journals (>10 publications) for Research Capacity Building Initiative publications

Fig. 2.  Australian states (no. of Primary Health Care Research, Evaluation and Development-supported departments) and percentage of publications.

Table 3 presents the nature of the research methods and types of publication. The greatest proportion of papers involved quantitative research (19.7%). There were similar methods used among rural and urban units although the rural groups recorded higher percentages of evaluations than the urban groups, while the latter had greater numbers of professional practice publications.

Table 3.  Types of publications in peer-reviewed journals

There were 124 first-time authors indicated in the available data. There were 100 articles that included a first-time author, 19 of which referred to multiple first-time authors on the one paper.

As displayed in Table 4, there were a range of RCBI supports identified by the organisations, with mentoring and supervision most commonly cited (43.6%). The rural groups mentioned financial support (funding, fellowships or bursaries) more often than any other type of assistance. For the urban groups, as for the total sample, mentoring and supervision were most frequently reported.

Table 4.  Types of Research Capacity Building Initiative (RCBI) support
There were 128 papers referring to multiple types of RCBI support combining to assist with the end product

The study’s main hypothesis related to the likelihood that PHCRED-supported publications would be in line with the National PHC Strategy priority areas and building blocks; overall the hypothesis was largely supported (Table 5).

Table 5.  National Primary Health Care Strategy themes

• 85.9% of publications were coded as relevant to a priority area, building block or both

• 40.4% of total publications illustrated content related to both aspects (priority areas and building blocks) of reform

• 79.6% of completed citations were able to be allocated to one of the priority areas

• 51.3% were able to be categorised according to building blocks

• 48.2% of the rural units’ papers received both priority area and building block codes

• 13.1% of urban papers did not have an underlying priority area or building block code

The percentages in Figs 3 and 4 demonstrate the proportion of each of the National PHC Strategy themes represented among the papers that could be coded.

Fig. 3.  Percentage of total coded publications relating to health reform priority areas.

Fig. 4.  Percentage of total coded publications relating to health reform building blocks.

• Of the 508 priority area papers 38.0% related to improving quality, safety, performance and accountability

• Of the 327 building block papers 45.9% reflected skilled workforce issues

There were a range of subject areas addressed by rural and urban groups (Table 6).

Table 6.  National Primary Health Care Strategy themes among urban and rural units

• Rural groups recorded increasing health promotion or prevention as their key priority area

• Improving quality, safety, performance and accountability was most important for urban groups

• Rural groups recorded greater proportions of papers around improving access and reducing inequity compared with urban groups

• Higher percentage of rural papers than urban publications associated with regional integration

• Urban groups had more financing and system performance-related papers than rural groups

Phase two of the PHCRED Strategy was implemented to provide support to researchers in conducting and disseminating research relevant to policy and practice. The current study recorded increasing research output in the period from 2006 to 2010 with publications in high-quality national and international journals. The majority of publications from PHCRED-supported departments addressed the building blocks and priority areas of the National PHC Strategy, highlighting the ability of this work to contribute to the PHC body of knowledge. There were three specific aims in the current study, namely to examine the volume of research output as a result of the RCBI, to assess the PHC issues addressed in PHCRED-supported research and to identify differences between the products from departments of general practice and rural health. Each of these is described below.

Demonstrating increasing publication rates and high numbers of first-time authors, the results presented are indicative of a capacity-building effort enabling development of research skills. Ovhed et al. (2005) suggested that there are ~400 PHC publications produced in Australia in each year, and it seems that the RCBI is able to account for a large fraction of this output (Yen et al. 2010). In terms of specific types of publications, previous findings have suggested there are a lack of randomised controlled trials (RCTs) and similar rigorous programs in PHC, most likely due to the funding available (Yallop et al. 2006; Birden 2007). Evidence in the current study reinforced this notion as RCTs represented only 6.6% of the total publications. This may be indicative of the short-term nature of the research fellowship positions funded and/or the amount of funding allocated to university departments, which may be conducive to smaller projects, but also may indicate that there are more suitable methods for the research that is needed in PHC. Almost 50% of the publications incorporated primary qualitative or quantitative research demonstrating that there is a body of work informing and delivering new ideas. The range of journals publishing PHCRED-supported papers incorporated PHC-related fields from public health to clinical practice. While 60% of the top 10 list were Australian journals, the high frequency with which publications appeared in influential international journals once again reflects the significant contribution from this cohort of Australian researchers, and parallels previous findings (Kalucy and Jackson Bowers 2010; Mendis et al. 2010). Askew et al. (2008) suggested that ‘more time and sustained investment in PHCRED are essential to see tangible outputs from funded research in general practice’ (p. 103). Considering results from the current study, it seems that phase two has both encouraged researchers in their endeavours and aided in production of tangible outputs.

With its current infrastructure Australia is well placed as a reputable source of PHC research relevant to both services and academia (Yen et al. 2010; Glanville et al. 2011). Similar work to the current research based on the General Practice Evaluation Program (the ‘predecessor’ to the PHCRED Strategy) found that research funded by this strategy reflected the PHC priority areas identified by the government (Beacham et al. 2003). This has been replicated in the current results with evidence that the underlying themes in these publications are indicative of core National PHC Strategy issues. The majority of papers related to a priority area of improving quality, safety, performance and accountability. This relates to establishing national standards for PHC performance, ensuring the safety of patients and the workforce, using evidence-informed decision making and practice, and promoting patient satisfaction and wellbeing. According to the Department of Health and Ageing (2010) this priority area also includes a commitment to research that can be applied to policy and practice, interesting considering it is the most frequently reported area among departments funded by the RCBI. In relation to building blocks, it was developing a skilled workforce that was most frequently addressed in PHCRED-supported publications. There are several regions in Australia experiencing PHC workforce shortages, hence research to encourage health professionals to expand their skills, conduct research or engage with rural communities is vital. It has been proposed that improving the number of skilled GPs and allied health professionals may lead to reduced rates of hospitalisation and better patient/GP ratios to meet the needs of patients throughout Australia (Department of Health and Ageing 2010).

The range of themes described throughout the current study reflects the contribution of both the university departments and the PHCRED strategy to the PHC evidence base. There were several articles that could not be coded according to national health reform topics, paralleling previous results that have reported that the research priorities noted by GPs are not necessarily matched with national priorities, but focus more on clinical issues (Dunbar et al. 2002). This reflects the diversity of PHC research. As described by Dunbar and colleagues (2002)

there is a need to balance a ‘bottom up’ ... approach among general practitioners and other PHC practitioners in their areas of special interest, with the ‘top down’ priorities of both the Commonwealth and universities. (p. 207)

McIntyre et al. (2007) also suggested that

gaps where research may not be happening … may indicate that there is an insufficient skill base or funding for research to be undertaken in these areas or the priorities may not be addressing areas where there is a PHC research skill set. (p. 9)

It has been observed that there is limited attention on information technology and chronic disease management among PHC researchers (Yen et al. 2010); perhaps chronic diseases are more thoroughly explored in specialised research while information technology is a rapidly expanding field that will likely record greater prevalence in coming years.

There were similar numbers of departments of rural health and general practice yet their RCBI experience differed. The urban units demonstrated a relatively linear increase in publication rates across the years. In contrast, the rural groups had much less variation and no obvious linear increase. Perhaps publication rates may not be an accurate reflection of RCBI outcomes for this latter group and assessing skill development, collaboration and mentoring may be more indicative of their enhanced research capacity. With regard to the nature of the research, the rural groups reported more evaluations than the urban groups, perhaps reflecting the needs of the communities. Methods need to be suited specifically to the setting and the resources available, as was apparent by the different kinds of research conducted in rural and urban spaces. The rural groups also cited financial support as the key RCBI contribution, which may be representative of the challenges of recruiting and retaining staff in addition to enabling skill development in remote areas. For the urban groups mentoring and supervision was the most frequent form of RCBI support, which may be indicative of the number of staff members in urban departments and the possibility of collaboration with researchers located in the same cities.

The rural groups consistently produced publications relating to improving access and reducing inequity, likely a reflection of the communities’ needs, attempts to improve health and the ease with which residents can access required support. In terms of building blocks, the rural papers focussed on skilled workforce and regional integration, which represent a type of ‘bottom up’ movement in these areas as opposed to the urban centres with more established research units and health services. Nevertheless, the urban groups did illustrate an interest in both skilled workforce and improving financing and system performance. For the urban groups it seems there were often different, non-reform related areas of attention with greater percentages of papers unable to be allocated a priority area or building block code. While the rural units often had fewer resources available in comparison with the much larger urban groups, they demonstrated steady research contributions that focussed on issues relevant to their specific space. As described in an evaluation of the first phase of the PHCRED Strategy, the RCBI ‘allowed departments to build on their strengths and to meet local needs’ (Oceania Health Consulting 2005, p. 3).

As discussed by McIntyre et al. (2007), the nature of self-reported annual reviews is that they rely strongly on the input of the research organisations. Reporting was not always complete nor were some details easily interpreted. Further, as is the case with any qualitative analysis, the subjective nature of coding is a potential limitation. When employing multiple coders the individual’s experience of PHC, their understanding of the codes and the detail in the guidelines are all potential confounds. In addition there have been some cautions regarding the use of publication rates as an accurate measure of research activity. It has been suggested that while peer-reviewed papers are one source of information around building research capacity, there may be benefit in considering conference presentations, grants, participation in projects and workshop attendance (McGrail et al. 2006; Askew et al. 2008; Jansen and Ruwaard 2012). Publication rates may be unable to capture the development of emerging researchers and collaborations, and do not necessarily reflect the societal impact of research on policy and practice (Hancock and Wilson 2006; Reed et al. 2011; Jansen and Ruwaard 2012). This suggests that the publication rates presented in the current study may underestimate the effectiveness of the RCBI. However, as discussed by Yen et al. (2010), publication is the first step of knowledge dissemination, therefore analysis of publication rates does provide some insight into the impact of the RCBI. Del Mar and Askew (2004) refer to definitions of quality as including ‘relevance to implementation by clinicians and policymakers’ (p. S36). It would seem from the consistent manner in which the publications in the current study are linked with the National PHC Strategy areas that this research definitely has relevance.

The results presented here support the notion that the RCBI has been able to improve research output among PHC researchers. The accessibility of non-competitive funding of this nature is rare (Oceania Health Consulting 2005) and the question must be raised as to where PHC would be in Australia if such a funding source had not been available. The answer may become clearer in coming years with a change in phase three of the PHCRED strategy from RCBI funding to competitive-based Centres for Research Excellence. In a similar approach to that conducted by Glanville et al. (2011), the future of Australian PHCRED-supported research may also be to assess its value on an international stage. While Glanville et al. (2011) suggested that Australia demonstrated slow growth in primary care publications as a function of research expenditure, the current data have suggested that the RCBI has enabled increasing rates of publication nationwide. With suggestion that the UK, the USA and the Netherlands are among the best performers internationally it would be beneficial to examine Australian PHC researchers’ potential (Mant et al. 2004; Glanville et al. 2011).

There has been suggestion that research capacity and output are strongly influenced by the field (Askew et al. 2008), with less research conducted in PHC than other health-oriented fields such as surgery and medicine. In line with this, for sustainability of the research community there will be a need to consider career progression in PHC, with evidence that PHC research is often limited to higher degrees (McGrail et al. 2006; Yen et al. 2010). It has been proposed that while the PHCRED Strategy developed research capacity there must be methods in place to sustain this development (Dunbar et al. 2002). Yen et al. (2010) propose that the translation of small research studies into policy is an area for future development to ensure that the linkage and exchange methods initiated under the PHCRED Strategy are supported. The goal of engaging PHC workers in research is that in the future

family physicians will not only respond to commissioned research but also set the research agenda themselves by pursuing their own research ideas or desires regardless of whether it is perceived as a priority. (Del Mar and Askew 2004, p. S36)

Continued support could promote objectives that have long been sought after for improving research capacity, and improving the health of the population by contributing to the body of knowledge in PHC. The way to continue such a legacy is ‘to identify, encourage and support the next generation of researchers’ (Hancock and Wilson 2006, p. 10). Recommendations for the third phase of the PHCRED Strategy include a need to focus on long-term research, conducting intervention and evaluation studies, in PHC settings, involving as many players as possible including, but not limited to, health practitioners, academics, Medicare Locals, policy makers and, importantly, patients (Yen et al. 2010; Hay et al. 2012).

The PHCRED Strategy’s RCBI has been able to enhance the research skills and output of PHC researchers in Australia. Not only have PHCRED-supported researchers produced consistent levels of publications but they have been able to inform and address priority areas and building blocks relevant to national health reform. Rural and urban researchers attend to different aspects of PHC but this serves to demonstrate the importance of directing research towards the needs of specific populations. The diversity in PHC is valuable and should have a strong impact on both clinical practice and health policy directions. The next phase of the PHCRED Strategy must target knowledge transfer and exchange to ensure the uptake of this valuable research into practice.

PHC RIS is funded in part through the PHCRED Strategy.