Advancing bibliometric assessment of research productivity: an analysis of US Departments of Family Medicine
Winston Liaw 1 2 9 , Andrew W. Bazemore 3 4 , Bernard Ewigman 5 , Tanvir Chowdhury Turin 6 , Daniel McCorry 7 , Stephen Petterson 1 , Susan M. Dovey 81 Robert Graham Center, Policy Studies in Family Medicine and Primary Care, Washington DC, USA
2 Department of Health Systems and Population Health Sciences, University of Houston, College of Medicine, Houston, TX 77204, USA
3 American Board of Family Medicine in Lexington, Kentucky, USA
4 Center for Professionalism and Value in Health Care in Washington, DC, USA
5 Department of Family Medicine, University of Chicago & NorthShore University Health System, Chicago, IL 60637, USA
6 Department of Family Medicine, University of Calgary, Calgary AB T2N 4N1, Canada
7 Reid Hospital and Health Care Services, Richmond, Indiana, USA
8 Royal New Zealand College of General Practitioners, Wellington, New Zealand
9 Corresponding author. Email: winstonrliaw@gmail.com
Journal of Primary Health Care 12(2) 149-158 https://doi.org/10.1071/HC19098
Published: 7 May 2020
Journal Compilation © Royal New Zealand College of General Practitioners 2020 This is an open access article licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
Abstract
INTRODUCTION: Measurement of family medicine research productivity has lacked the replicable methodology needed to document progress.
AIM: In this study, we compared three methods: (1) faculty-to-publications; (2) publications-to-faculty; and (3) department-reported publications.
METHODS: In this cross-sectional analysis, publications in peer-reviewed, indexed journals for faculty in 13 US family medicine departments in 2015 were assessed. In the faculty-to-publications method, department websites to identify faculty and Web of Science to identify publications were used. For the publications-to-faculty method, PubMed’s author affiliation field were used to identify publications, which were linked to faculty members. In the department-reported method, chairs provided lists of faculty and their publications. For each method, descriptive statistics to compare faculty and publication counts were calculated.
RESULTS: Overall, 750 faculty members with 1052 unique publications, using all three methods combined as the reference standard, were identified. The department-reported method revealed 878 publications (84%), compared to 616 (59%) for the faculty-to-publications method and 412 (39%) for the publication-to-faculty method. Across all departments, 32% of faculty had any publications, and the mean number of publications per faculty was 1.4 (mean of 4.4 per faculty among those who had published). Assistant Professors, Associate Professors, Professors and Chairs accounted for 92% of all publications.
DISCUSSION: Online searches capture a fraction of publications, but also capture publications missed through self-report. The ideal methodology includes all three. Tracking publications is important for quantifying the return on our discipline’s research investment.
KEYwords: Professional development; primary health care; health research; bibliometric analysis
References
[1] Thomson W. Popular Lectures and Addresses. Vol. 1. London: Macmillan and Co.; 1889.[2] Dunikowski LG, Freeman TR. Impact of family medicine research. Can Fam Physician. 2016; 62 266–8.
| 26975920PubMed |
[3] McWhinney IR. General practice as an academic discipline: reflections after a visit to the United States. Lancet. 1966; 287 419–23.
| General practice as an academic discipline: reflections after a visit to the United States.Crossref | GoogleScholarGoogle Scholar |
[4] Starfield B. A framework for primary care research. J Fam Pract 1996; 42 181–5.
| 8606309PubMed |
[5] Zhang Y, Kou J, Zhang X-G, et al. The evolution of academic performance in nine subspecialties of internal medicine: an analysis of journal citation reports from 1998 to 2010. PLoS ONE. 2012; 7(10): e48290.
[6] Stone K, Whitham EA, Ghaemi SN. A comparison of psychiatry and internal medicine: a bibliometric study. Acad Psychiatry. 2012; 36 129–32.
| A comparison of psychiatry and internal medicine: a bibliometric study.Crossref | GoogleScholarGoogle Scholar | 22532204PubMed |
[7] Quinn N, Hensey O, McDowell DT. A historical perspective of pediatric publications: a bibliometric analysis. Pediatrics. 2013; 132 406–12.
| A historical perspective of pediatric publications: a bibliometric analysis.Crossref | GoogleScholarGoogle Scholar | 23979085PubMed |
[8] Hui Z, Yi Z, Peng J. Bibliometric analysis of the orthopedic literature. Orthopedics 2013; 36 e1225–32.
| Bibliometric analysis of the orthopedic literature.Crossref | GoogleScholarGoogle Scholar | 24093695PubMed |
[9] Zyoud SH, Al-Jabi SW, Sweileh WM, Awang R. A bibliometric analysis of toxicology research productivity in Middle Eastern Arab countries during a 10-year period (2003–2012). Health Res Policy Syst. 2014; 12 4
| A bibliometric analysis of toxicology research productivity in Middle Eastern Arab countries during a 10-year period (2003–2012).Crossref | GoogleScholarGoogle Scholar | 24443999PubMed |
[10] Zyoud SH, Al-Jabi SW, Sweileh VY. Worldwide research productivity in the field of electronic cigarette: a bibliometric analysis. BMC Public Health. 2014; 14 667
| Worldwide research productivity in the field of electronic cigarette: a bibliometric analysis.Crossref | GoogleScholarGoogle Scholar | 24975597PubMed |
[11] Kennedy C, Sullivan PO, Bilal M, Walsh A. Ireland’s contribution to orthopaedic literature: a bibliometric analysis. Surgeon. 2013; 11 267–71.
| Ireland’s contribution to orthopaedic literature: a bibliometric analysis.Crossref | GoogleScholarGoogle Scholar | 23380175PubMed |
[12] Zhang Y, Kou J, Zhang X-G, et al. The evolution of academic performance in nine subspecialties of internal medicine: an analysis of journal citation reports from 1998 to 2010. PLoS ONE 2012; 7 e48290
| The evolution of academic performance in nine subspecialties of internal medicine: an analysis of journal citation reports from 1998 to 2010.Crossref | GoogleScholarGoogle Scholar | 23300934PubMed |
[13] Van der Veer T, Baars JE, Birnie E, Hamberg P. Citation analysis of the ‘Big Six’ journals in internal medicine. Eur J Intern Med. 2015; 26 458–9.
| Citation analysis of the ‘Big Six’ journals in internal medicine.Crossref | GoogleScholarGoogle Scholar | 26066401PubMed |
[14] Ruffin MT, Sheets KJ. Primary care research funding sources. J Fam Pract. 1992; 35 281–7.
| 1517725PubMed |
[15] Pathman DE, Viera AJ, Newton WP. Research published in 2003 by US family medicine authors. J Am Board Fam Med. 2008; 21 6–16.
| Research published in 2003 by US family medicine authors.Crossref | GoogleScholarGoogle Scholar | 18178697PubMed |
[16] Merenstein J, Rao G, Wolfe S, Weaver-Agostoni J. Clinical research in family medicine: published articles 2000 and 2005. Fam Med. 2009; 41 7–8.
| 19132562PubMed |
[17] Post RE, Weese TJ, Mainous AG, Weiss BD. Publication productivity by family medicine faculty: 1999 to 2009. Fam Med. 2012; 44 312
| 23027112PubMed |
[18] Liaw W, Petterson S, Jiang V, Bazemore A, Pecsok J, McCorry D, et al. The scholarly output of faculty in Family Medicine Departments. Fam Med. 2019; 51 103–11.
| The scholarly output of faculty in Family Medicine Departments.Crossref | GoogleScholarGoogle Scholar | 30736035PubMed |
[19] Phillips RL, Pugno PA, Saultz JW, et al. Health is primary: family medicine for America’s health. Ann Fam Med. 2014; 12 S1–12.
| Health is primary: family medicine for America’s health.Crossref | GoogleScholarGoogle Scholar | 25352575PubMed |
[20] Ewigman B, Davis A, Vansaghi T, et al. Building research & scholarship capacity in departments of Family Medicine: a new joint ADFM-NAPCRG initiative. Ann Fam Med. 2016; 14 82–3.
| Building research & scholarship capacity in departments of Family Medicine: a new joint ADFM-NAPCRG initiative.Crossref | GoogleScholarGoogle Scholar | 26755789PubMed |
[21] Fiscella K. Reflections on primary care research. Post-Starfield Summit: thinking pragmatically, thinking big. Ann Fam Med. 2016; 14 382–3.
| Reflections on primary care research. Post-Starfield Summit: thinking pragmatically, thinking big.Crossref | GoogleScholarGoogle Scholar |
[22] Doohan N, Coutinho AJ, Lochner J, et al. “A paradox persists when the paradigm is wrong”: Pisacano Scholars’ reflections from the Inaugural Starfield Summit. J Am Board Fam Med. 2016; 29 793–804.
| “A paradox persists when the paradigm is wrong”: Pisacano Scholars’ reflections from the Inaugural Starfield Summit.Crossref | GoogleScholarGoogle Scholar | 28076263PubMed |
[23] Accreditation Council for Graduate Medical Education. Scholarly Activity Guidelines: Review Committee for Family Medicine [Internet]. Chicago, IL. [cited 2019 November 1]. Available from: https://www.acgme.org/Portals/0/PFAssets/ProgramResources/120_scholarly-activity-guidelines.pdf
[24] Accreditation Council for Graduate Medical Education. Family Medicine Milestone Project [Internet]. Chicago, IL; 2015. [cited 2016 May 10]. Available from: https://www.acgme.org/Portals/0/PDFs/Milestones/FamilyMedicineMilestones.pdf
[25] Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J. 2008; 22 338–42.
| Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses.Crossref | GoogleScholarGoogle Scholar | 17884971PubMed |
[26] ORCID. ORCID [Internet]. Newark, Delaware; [cited 2018 February 7]. Available from: https://orcid.org/
[27] Association of American Medical Colleges. Faculty Roster [Internet]. Washington, DC; [cited 2018 February 7]. Available from: https://www.aamc.org/data/facultyroster/
