Validation of the research capacity and culture (RCC) tool: measuring RCC at individual, team and organisation levels
Libby Holden A D , Susan Pager B , Xanthe Golenko A and Robert S. Ware C
+ Author Affiliations
- Author Affiliations
A School of Medicine, Griffith University, University Drive, Meadowbrook, Qld 4131, Australia.
B Queensland Health, Level 2, 7 Clunies Ross Court, Brisbane Technology Park, Eight Mile Plains, Qld 4113, Australia.
C School of Population Health, University of Queensland, Herston Road, Brisbane, Qld 4006, Australia.
D Corresponding author. Email: l.holden@griffith.edu.au
Australian Journal of Primary Health 18(1) 62-67 https://doi.org/10.1071/PY10081
Submitted: 29 October 2010 Accepted: 2 May 2011 Published: 21 October 2011
Abstract
Research capacity building (RCB) in Australia has recently focussed on strategies that take a whole of system approach to developing research culture at individual, team and organisation levels. Although a theoretical framework exists, no tool has been published that quantitatively measures the effectiveness of RCB interventions aimed at these three levels. A sample of 134 allied health workers was used to validate the research capacity and culture (RCC) tool. Item level analysis was undertaken using Cronbach’s α and exploratory factor analysis, and test–retest reliability was examined using intra-class correlations (ICC). The tool had one factor emerge for each domain, with excellent internal consistency for organisation, team and individual domains (α = 0.95, 0.96 and 0.96 respectively; and factor loadings ranges of 0.58–0.89, 0.65–0.89 and 0.59–0.93 respectively). The overall mean score (total) for each domain was: 5.4 (inter-quartile range 3.9–7.7), 4.4 (IQR 2.6–6.1) and 3.9 (IQR 2.9–6) for the organisation, team and individual domains respectively. Test–retest reliability was strong for each domain: organisation ICC = 0.77, team ICC = 0.83 and individual ICC = 0.82. The RCC tool has three domains measuring research capacity and culture at organisation, team and individual levels. It demonstrates excellent internal consistency and strong test–retest reliability.
Additional keywords: evaluation, research culture, team based.
References
Albert E, Mickan S (2003) Closing the gap and widening the scope. New directions for research capacity building in primary health care. Australian Family Physician 32, 1038–1041.Argimon-Pallàs JM, Flores-Mateo G, Jimenez-Villa J, Pujol-Ribera E, Foz G, Bundo-Vidiella M, Juncosa S, Fuentes-Bellido CM, Perez-Rodriguez B, Margalef-Pallares F, Villafafila-Ferrero R, Fores-Garcia D, Roman-Martinez J, Vilert-Garroga E (2009) Study protocol of psychometric properties of the Spanish translation of a competence test in evidence based practice: the Fresno test. BMC Health Services Research 9, 37
| Study protocol of psychometric properties of the Spanish translation of a competence test in evidence based practice: the Fresno test.Crossref | GoogleScholarGoogle Scholar | 19239704PubMed |
Australian Institute of Health and Welfare (2008) Australia’s health 2008 cat. no. AUS99.
Bateman H, Walter F, Elliott J (2004) What happens next? Evaluation of a scheme to support primary care practitioners with a fledgling interest in research. Family Practice 21, 83–86.
| What happens next? Evaluation of a scheme to support primary care practitioners with a fledgling interest in research.Crossref | GoogleScholarGoogle Scholar | 14760051PubMed |
Blakeman TM, Comino EJ, Zwar NA, Harris MF (2001) Evaluating the introduction of the enhanced primary care 75+ annual health assessments. Australian Family Physician 30, 1004–1009.
Butler SF, Budman SH, McGee MD, Davis MS, Cornelli R, Morey LC (2005) Addiction severity assessment tool: development of a self-report measure for clients in substance abuse treatment. Drug and Alcohol Dependence 80, 349–360.
| Addiction severity assessment tool: development of a self-report measure for clients in substance abuse treatment.Crossref | GoogleScholarGoogle Scholar | 15994028PubMed |
Cook KE (2005) Using critical ethnography to explore issues in health promotion. Qualitative Health Research 15, 129–138.
| Using critical ethnography to explore issues in health promotion.Crossref | GoogleScholarGoogle Scholar | 15574720PubMed |
Cooke J (2005) A framework to evaluate research capacity building in health care. BMC Family Practice 6, 44
| A framework to evaluate research capacity building in health care.Crossref | GoogleScholarGoogle Scholar | 16253133PubMed |
Cooke J, Nancarrow S, Dyas J, Williams M (2008) An evaluation of the ‘designated research team’ approach to building research capacity in primary care. BMC Family Practice 9, 37
| An evaluation of the ‘designated research team’ approach to building research capacity in primary care.Crossref | GoogleScholarGoogle Scholar | 18588685PubMed |
D’Auria D (2000) Building a research capacity for occupational medicine. Occupational Medicine (Oxford, England) 50, 79
Department of Health and Ageing (2009) Evaluation of the primary health care research, evaluation and development strategy: findings summary report. Healthcare Planning and Evaluation Pty Ltd: Fullarton, SA.
DeVellis RF (2006) Classical test theory. Medical Care 44, S50–S59.
| Classical test theory.Crossref | GoogleScholarGoogle Scholar | 17060836PubMed |
Drennan J (2008) Postgraduate research experience questionnaire: reliability and factor structure with masters in nursing graduates. Journal of Advanced Nursing 62, 487–498.
| Postgraduate research experience questionnaire: reliability and factor structure with masters in nursing graduates.Crossref | GoogleScholarGoogle Scholar | 18476947PubMed |
Farmer E, Weston K (2002) A conceptual model for capacity building in Australian primary health care research. Australian Family Physician 31, 1139–1142.
Finch J (2006) Developing the best research professionals: qualified graduate nurses: recommendations for preparing and supporting the clinical academic nurses of the future: Report of the UKCTC Sub Committee for Nurses in Clinical Research (Workforce). London: UK clinical research collaboration. Available at http://www.ukcrc.org/recommendations-for-nursing-research-careers-published/ [Verified 8 September 2011]
Frontera WR, Fuhrer MJ, Jette AM, Chan L, Cooper RA, Duncan PW, Kemp JD, Ottenbacher KJ, Hunter Peckham P, Roth EJ, Tate DG (2006) Rehabilitation medicine summit: building research capacity. The Journal of Head Trauma Rehabilitation 21, 1–7.
| Rehabilitation medicine summit: building research capacity.Crossref | GoogleScholarGoogle Scholar | 16456387PubMed |
Hanusaik N, O’Loughlin JL, Kishchuk N, Eyles J, Robinson K, Cameron R (2007) Building the backbone for organisational research in public health systems: development of measures of organisational capacity for chronic disease prevention. Journal of Epidemiology and Community Health 61, 742–749.
| Building the backbone for organisational research in public health systems: development of measures of organisational capacity for chronic disease prevention.Crossref | GoogleScholarGoogle Scholar | 17630377PubMed |
Härdén M, Nystrom B, Kulich K, Carlsson J, Bengtson A, Edvardsson N (2009) Validity and reliability of a new, short symptom rating scale in patients with persistent atrial fibrillation. Health and Quality of Life Outcomes 7, 65
| Validity and reliability of a new, short symptom rating scale in patients with persistent atrial fibrillation.Crossref | GoogleScholarGoogle Scholar | 19604399PubMed |
Kalucy EC, Pearce CM, Beacham B, Lowcay BL, Yates RE (2006) What supports effective research links between divisions of general practice and universities? The Medical Journal of Australia 185, 114–116.
Mant D (1997) R & D in primary care. Final report (Report no. 97CC0138). Department of Health, Weatherby, UK.
Mazlan R, Kei J, Hickson L (2009) Test–retest reliability of the acoustic stapedial reflex test in healthy neonates. Ear and Hearing 30, 295–301.
| Test–retest reliability of the acoustic stapedial reflex test in healthy neonates.Crossref | GoogleScholarGoogle Scholar | 19322092PubMed |
Mendis K, Solangaarachchi I (2005) PubMed perspective of family medicine research: where does it stand? Family Practice 22, 570–575.
| PubMed perspective of family medicine research: where does it stand?Crossref | GoogleScholarGoogle Scholar | 16126822PubMed |
Mickan S (2005) Evaluating the effectiveness of health care teams. Australian Health Review 29, 211–217.
| Evaluating the effectiveness of health care teams.Crossref | GoogleScholarGoogle Scholar | 15865572PubMed |
National Health Service Scotland (2004) ‘Allied health professions: research and development action plan.’ NHS Scotland, Edinburgh.
North American Primary Care Research Group Committee on Building Research Capacity and the Academic Family Medicine Organisations Research Sub-Committee (2002) What does it mean to build research capacity? Family Medicine 34, 678–684.
Pickstone C, Nancarrow S, Cooke J, Vernon W, Mountain G, Boyce RA, Campbell J (2008) Building research capacity in the allied health professions. Evidence & Policy 4, 53–68.
| Building research capacity in the allied health professions.Crossref | GoogleScholarGoogle Scholar |
Queensland Health: Disability and Rehabilitation Research Unit (2006) Competencies for community rehabilitation in Queensland. Audit of the training and education needs of staff working in community rehabilitation in Queensland . Available at http://www.health.qld.gov.au/qhcrwp/docs/exesumm.pdf [Verified 4 June 2006]
Ried K, Farmer EA, Weston KM (2006) Setting directions for capacity building in primary health care: a survey of a research network. BMC Family Practice 7, 8
| Setting directions for capacity building in primary health care: a survey of a research network.Crossref | GoogleScholarGoogle Scholar | 16466583PubMed |
Ried K, Farmer EA, Weston KM (2007) Bursaries, writing grants and fellowships: a strategy to develop research capacity in primary health care. BMC Family Practice 8, 1–13.
Sarre G, Cooke J (2009) Developing indicators for measuring research capacity development in primary care organizations: a consensus approach using a nominal group technique. Health and Social Care in the Community 17, 244–253.
| Developing indicators for measuring research capacity development in primary care organizations: a consensus approach using a nominal group technique.Crossref | GoogleScholarGoogle Scholar | 19040697PubMed |
Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, Whelan C, Green M (2006) Instruments for evaluating education in evidence-based practice: a systematic review. Journal of the American Medical Association 296, 1116–1127.
| Instruments for evaluating education in evidence-based practice: a systematic review.Crossref | GoogleScholarGoogle Scholar | 1:CAS:528:DC%2BD28XptlKit74%3D&md5=46677e6602a0a46d52111771e63b9830CAS | 16954491PubMed |
Smith H, Wright D, Morgan S, Dunleavey J, Moore M (2002) The ‘research spider’: a simple method of assessing research experience. Primary Health Care Research and Development 3, 139–140.
| The ‘research spider’: a simple method of assessing research experience.Crossref | GoogleScholarGoogle Scholar |
Stineman M, Kennedy B (2005) Response to ‘metrics of rehabilitation research capacity’: within and beyond our borders. American Journal of Physical Medicine & Rehabilitation 84, 1020–1033.
| Response to ‘metrics of rehabilitation research capacity’: within and beyond our borders.Crossref | GoogleScholarGoogle Scholar |
Tabchnick B, Fidel L (2006) ‘Using multivariate statistics (5th edn).’ (Allyn & Bacon: Boston, MA)
Trostle J (1992) Research capacity building in international health: definitions, evaluations, and strategies for success. Social Science & Medicine 35, 1321–1324.
| Research capacity building in international health: definitions, evaluations, and strategies for success.Crossref | GoogleScholarGoogle Scholar | 1:STN:280:DyaK3s%2FpslWmsg%3D%3D&md5=6a4f1a1d8bdb7aacb34ad15849d332dfCAS |
Wagner AK, McElligott J, Wagner EP, Gerber LH (2005) Measuring rehabilitation research capacity: report from the AAPM & R Research Advisory Committee. American Journal of Physical Medicine & Rehabilitation 84, 955–968.
| Measuring rehabilitation research capacity: report from the AAPM & R Research Advisory Committee.Crossref | GoogleScholarGoogle Scholar |
Whyatt D, Scott J, Larson A (2006) ‘The research capacity matrix.’ Paper presented at the GP & PHC research conference abstracts 2006.
Yallop JJ, McAvoy BR, Croucher JL, Tonkin A, Piterman L (2006) Primary health care research – essential but disadvantaged. The Medical Journal of Australia 185, 118–120.
Zwar NA, Weller DP, McCloughan L, Traynor VJ (2006) Supporting research in primary care: are practice-based research networks the missing link? The Medical Journal of Australia 185, 110–113.
