Precision health through prediction modelling: factors to consider before implementing a prediction model in clinical practice
Mohammad Z. I. Chowdhury 1 , Tanvir C. Turin 1 2 31 Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Canada.
2 Department of Family Medicine, Cumming School of Medicine, University of Calgary, G012F, Health Sciences Centre, 3330 Hospital Drive NW, Calgary, Alberta, Canada.
3 Corresponding author. Email: chowdhut@ucalgary.ca
Journal of Primary Health Care 12(1) 3-9 https://doi.org/10.1071/HC19087
Published: 30 March 2020
Journal Compilation © Royal New Zealand College of General Practitioners 2020 This is an open access article licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
Abstract
INTRODUCTION: Precision medical practice emphasises early detection, improved surveillance and prevention through targeted intervention. Prediction models can help identify high-risk individuals to be targeted for healthy behavioural changes or medical treatment to prevent disease development and assist both health professionals and patients to make informed decisions. Concerns exist regarding the adequacy, accuracy, validity and reliability of prediction models.
AIM: The purpose of this study is to introduce readers to the basic concept of prediction modelling in precision health and recommend factors to consider before implementing a prediction model in clinical practice.
METHODS: Prediction models developed maintaining proper process and with quality prediction and validation can be used in clinical practice to improve patient care.
RESULTS: Aspects of prediction models that should be considered before implementation include: appropriateness of the model for the intended purpose; adequacy of the model; validation, face validity and clinical impact studies of the model; a parsimonious model with data easily measured in clinical settings; and easily accessible models with decision support for successful implementation.
DISCUSSION: Choosing clinical prediction models requires cautious consideration and several practical factors before implementing a model in clinical practice.
KEYwords: Prediction modelling; precision health; implement, clinical practice.
References
[1] Moons KG, Royston P, Vergouwe Y, et al. Prognosis and prognostic research: what, why, and how? BMJ. 2009; 338 b375| Prognosis and prognostic research: what, why, and how?Crossref | GoogleScholarGoogle Scholar | 19237405PubMed |
[2] Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009; 338 b606
| Prognosis and prognostic research: application and impact of prognostic models in clinical practice.Crossref | GoogleScholarGoogle Scholar | 19502216PubMed |
[3] National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) Third report of the National Cholesterol Education Program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III) final report. Circulation 2002; 106 3143–421.
| Third report of the National Cholesterol Education Program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III) final report.Crossref | GoogleScholarGoogle Scholar | 12485966PubMed |
[4] Stone NJ, Robinson JG, Lichtenstein AH, et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014; 63 2889–934.
| 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.Crossref | GoogleScholarGoogle Scholar | 24239923PubMed |
[5] Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ. 2009; 338 b604
| Prognosis and prognostic research: developing a prognostic model.Crossref | GoogleScholarGoogle Scholar | 19336487PubMed |
[6] Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009; 338 b605
| Prognosis and prognostic research: validating a prognostic model.Crossref | GoogleScholarGoogle Scholar | 19477892PubMed |
[7] Steyerberg EW, Moons KG, van der Windt DA, et al. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013; 10 e1001381
| Prognosis Research Strategy (PROGRESS) 3: prognostic model research.Crossref | GoogleScholarGoogle Scholar | 23393430PubMed |
[8] Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer Science & Business Media; 2009.
[9] Kuhn M, Johnson K. Applied Predictive Modeling. New York: Springer; 2013.
[10] Lee YH, Bang H, Kim DJ. How to establish clinical prediction models. Endocrinol Metab (Seoul). 2016; 31 38–44.
| How to establish clinical prediction models.Crossref | GoogleScholarGoogle Scholar | 26996421PubMed |
[11] Prosperi M, Min JS, Bian J, Modave F. Big data hurdles in precision medicine and precision public health. BMC Med Inform Decis Mak. 2018; 18 139
| Big data hurdles in precision medicine and precision public health.Crossref | GoogleScholarGoogle Scholar | 30594159PubMed |
[12] Gambhir SS, Ge TJ, Vermesh O, Spitler R. Toward achieving precision health. Sci Transl Med. 2018; 10 eaao3612
| Toward achieving precision health.Crossref | GoogleScholarGoogle Scholar | 29515002PubMed |
[13] Whelton PK, Carey RM, Aronow WS, et al. 2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/ PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol. 2018; 71 e127
| 29146535PubMed |
[14] Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019; 366 447–53.
| Dissecting racial bias in an algorithm used to manage the health of populations.Crossref | GoogleScholarGoogle Scholar | 31649194PubMed |
[15] D’Agostino RB, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care. Circulation. 2008; 117 743–53.
| General cardiovascular risk profile for use in primary care.Crossref | GoogleScholarGoogle Scholar | 18212285PubMed |
[16] Arnett DK, Blumenthal RS, Albert MA, et al. ACC/AHA guideline on the primary prevention of cardiovascular disease. J Am Coll Cardiol. 2019; 2019 26029
[17] Gail MH, Brinton LA, Byar DP, et al. Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst. 1989; 81 1879–86.
| Projecting individualized probabilities of developing breast cancer for white females who are being examined annually.Crossref | GoogleScholarGoogle Scholar | 2593165PubMed |
[18] Stevens RJ, Kothari V, Adler AI, et al. The UKPDS risk engine: a model for the risk of coronary heart disease in Type II diabetes (UKPDS 56). Clin Sci (Lond). 2001; 101 671–9.
| The UKPDS risk engine: a model for the risk of coronary heart disease in Type II diabetes (UKPDS 56).Crossref | GoogleScholarGoogle Scholar | 11724655PubMed |
[19] Lindström J, Tuomilehto J. The diabetes risk score. Diabetes Care. 2003; 26 725–31.
| The diabetes risk score.Crossref | GoogleScholarGoogle Scholar | 12610029PubMed |
[20] Hosmer DW, Jr, Lemeshow S, Sturdivant RX. Applied Logistic Regression. New York: John Wiley & Sons; 2013.
[21] Ogundimu EO, Altman DG, Collins GS. Adequate sample size for developing prediction models is not simply related to events per variable. J Clin Epidemiol. 2016; 76 175–82.
| Adequate sample size for developing prediction models is not simply related to events per variable.Crossref | GoogleScholarGoogle Scholar | 26964707PubMed |
[22] Steyerberg EW, Bleeker SE, Moll HA, et al. Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol. 2003; 56 441–7.
| Internal and external validation of predictive models: a simulation study of bias and precision in small samples.Crossref | GoogleScholarGoogle Scholar | 12812818PubMed |
[23] Bleeker SE, Moll HA, Steyerberg EW, et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003; 56 826–32.
| External validation is necessary in prediction research: a clinical example.Crossref | GoogleScholarGoogle Scholar | 14505766PubMed |
[24] Kappen TH, van Klei WA, van Wolfswinkel L, et al. Evaluating the impact of prediction models: lessons learned, challenges, and recommendations. Diagn Progn Res. 2018; 2 11
| Evaluating the impact of prediction models: lessons learned, challenges, and recommendations.Crossref | GoogleScholarGoogle Scholar | 31093561PubMed |
[25] Paynter NP, Cook NR, Everett BM, et al. Prediction of incident hypertension risk in women with currently normal blood pressure. Am J Med. 2009; 122 464–71.
| Prediction of incident hypertension risk in women with currently normal blood pressure.Crossref | GoogleScholarGoogle Scholar | 19375556PubMed |
[26] Ridker PM, Buring JE, Rifai N, Cook NR. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA. 2007; 297 611–9.
| Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score.Crossref | GoogleScholarGoogle Scholar | 17299196PubMed |
[27] Cook NR, Paynter NP, Eaton CB, et al. Comparison of the Framingham and Reynolds Risk scores for global cardiovascular risk prediction in the multiethnic Women’s Health Initiative. Circulation. 2012; 125 1748–56.
| Comparison of the Framingham and Reynolds Risk scores for global cardiovascular risk prediction in the multiethnic Women’s Health Initiative.Crossref | GoogleScholarGoogle Scholar | 22399535PubMed |
