Effectiveness and efficiency of training in digital healthcare packages: training doctors to use digital medical record keeping software
Nicola Benwell A , Kathryn Hird B , Nicholas Thomas A , Erin Furness A , Mark Fear C and Greg Sweetman A DA Medical Education Unit, Fiona Stanley Hospital, 11 Robin Warren Drive, Murdoch, WA 6150, Australia. Email: nicola.benwell@health.wa.gov.au; nicholas.thomas@health.wa.gov.au; erin.furness@health.wa.gov.au
B School of Medicine, University of Notre Dame Australia, 38 Henry Street, Fremantle, WA 6959, Australia. Email: kathryn.hird@nd.edu.au
C Burn Injury Research Unit, University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia. Email: mark.fear@uwa.edu.au
D Corresponding author. Email: Greg.Sweetman@health.wa.gov.au
Australian Health Review 41(5) 479-484 https://doi.org/10.1071/AH16090
Submitted: 20 April 2016 Accepted: 6 July 2016 Published: 5 September 2016
Abstract
Objective Fiona Stanley Hospital (FSH) is the first hospital in Western Australia to implement a digital medical record (BOSSnet, Core Medical Solutions, Australia). Formal training in the use of the digital medical record is provided to all staff as part of the induction program. The aim of the present study was to evaluate whether the current training program facilitates efficient and accurate use of the digital medical record in clinical practice.
Methods Participants were selected from the cohort of junior doctors employed at FSH in 2015. An e-Learning package of clinically relevant tasks from the digital medical record was created and, along with a questionnaire, completed by participants on two separate occasions. The time taken to complete all tasks and the number of incorrect mouse clicks used to complete each task were recorded and used as measures of efficiency and accuracy respectively.
Results Most participants used BOSSnet more than 10 times per day in their clinical roles and self-rated their baseline overall computer proficiency level as high. There was a significant increase in the self-rating of proficiency levels in successive tests. In addition, a significant improvement in both efficiency and accuracy for all participants was measured between the two tests. Interestingly, both groups ended up with similar accuracy on the second trial, despite the second group of participants starting with significantly poorer accuracy.
Conclusions Overall, the greatest improvements in task performance followed daily ward-based experience using BOSSnet rather than formalised training. The greatest benefits of training were noted when training was delivered in close proximity to the onset of employment.
What is known about the topic? Formalised training in the use of information and communications technology (ICT) is widespread in the health service. However, there is limited evidence to support the modes of learning typically used. Formalised training is often costly and there is little other than anecdotal evidence that currently supports its efficacy in the workplace.
What does the paper add? Assessment of accuracy when using the BOSSnet system over time revealed that daily use rather than formalised training appeared to have the most impact on performance. Formalised training was rated poorly, and this appeared to correlate with time between training and use. The present study suggests that formalised training, if required, should be delivered close in time to actual use of the system to benefit end-users. The study also shows that daily experience is more effective than formalised training to improve accuracy.
What are the implications for practitioners? Formalised training for ICT needs to be scheduled in close proximity to end-user use of the ICT. Current scheduling may be beneficial for ease of delivery, but unless it is delivered at a suitable time the benefits are minimal. Formalised training programs may not be critical for all staff and all staff improve with contextualised experience given time. Training may be better suited to optional rather than compulsory delivery programs with ongoing delivery to suit user schedules.
Additional keywords: education, e-Health, evaluation, health systems, information management, performance.
References
[1] Coiera E. Do we need a national electronic summary care record? Med J Aust 2011; 194 90–2.| 21241224PubMed |
[2] O’Leary KJ, Liebovitz DM, Feinglass J, Liss DT, Evans DB, Kulkarni N, Landler MP, Baker DW. Creating a better discharge summary: improvement in quality and timeliness using an electronic discharge summary. J Hosp Med 2009; 4 219–25.
| Creating a better discharge summary: improvement in quality and timeliness using an electronic discharge summary.Crossref | GoogleScholarGoogle Scholar | 19267397PubMed |
[3] Mandl KD, Kohane IS. Tectonic shifts in the health information economy. N Engl J Med 2008; 358 1732–7.
| Tectonic shifts in the health information economy.Crossref | GoogleScholarGoogle Scholar | 1:CAS:528:DC%2BD1cXksl2gsrs%3D&md5=a298fbf1cf5649fa9a446e70055d7919CAS | 18420506PubMed |
[4] Coiera E, Aarts J, Kulikowski C. The dangerous decade. J Am Med Inform Assoc 2012; 19 2–5.
| The dangerous decade.Crossref | GoogleScholarGoogle Scholar | 22116642PubMed |
[5] Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, Bayir H, Orr RA. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005; 116 1506–12.
| Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system.Crossref | GoogleScholarGoogle Scholar | 16322178PubMed |
[6] Sittig DF, Ash JS, Zhang J, Osheroff JA, Shabot MM. Lessons from unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2006; 118 797–801.
| Lessons from unexpected increased mortality after implementation of a commercially sold computerized physician order entry system.Crossref | GoogleScholarGoogle Scholar | 16882838PubMed |
[7] Dreyfus S. The five-stage model of adult skill acquisition. Bull Sci Technol Soc 2004; 24 177–81.
| The five-stage model of adult skill acquisition.Crossref | GoogleScholarGoogle Scholar |
[8] Vuk J, Anders ME, Mercado CC, Kennedy RL, Casella J, Steelman SC. Impact of simulation training on self-efficacy of outpatient health care providers to use electronic health records. Int J Med Inform 2015; 84 423–9.
| Impact of simulation training on self-efficacy of outpatient health care providers to use electronic health records.Crossref | GoogleScholarGoogle Scholar | 25746460PubMed |
[9] Nicklaus J, Kusser J, Zessin J, Amaya M. Transforming education for electronic health record implementation. J Contin Educ Nurs 2015; 46 359–63.
| Transforming education for electronic health record implementation.Crossref | GoogleScholarGoogle Scholar | 26247658PubMed |
[10] Nguyen L, Bellucci E, Nguyen LT. Electronic health records implementation: an evaluation of information system impact and contingency factors. Int J Med Inform 2014; 83 779–96.
| Electronic health records implementation: an evaluation of information system impact and contingency factors.Crossref | GoogleScholarGoogle Scholar | 25085286PubMed |
[11] Menachemi N. Barriers to ambulatory EHR: who are ‘imminent adopters’ and how do they differ from other physicians? Inform Prim Care 2006; 14 101–8.
| 17059699PubMed |
[12] Miller RH, Sim I. Physicians’ use of electronic medical records: barriers and solutions. Health Aff 2004; 23 116–26.
| Physicians’ use of electronic medical records: barriers and solutions.Crossref | GoogleScholarGoogle Scholar |
[13] Al-Azmi SF, Al-Enezi N, Chowdhury RI. Users’ attitudes to an electronic medical record system and its correlates: a multivariate analysis Health Inf Manage J 2009; 38 33–40.
[14] Jensen TB, Aanestad M. How healthcare professionals ‘make sense’ of an electronic patient record adoption Inf Syst Manage 2006; 24 29–42.
| How healthcare professionals ‘make sense’ of an electronic patient record adoptionCrossref | GoogleScholarGoogle Scholar |
[15] Gagnon MP, Desmartis M, Labrecque M, Car J, Pagliari C, Pluye P, Frémont P, Gagnon J, Tremblay N, Légaré F. Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals. J Med Syst 2012; 36 241–77.
| Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals.Crossref | GoogleScholarGoogle Scholar | 20703721PubMed |
