Review of medication errors that are new or likely to occur more frequently with electronic medication management systems

Electronic medication management systems (eMMS) have been demonstrated to improve medication safety by reducing medication errors.^1–3 However, they cause new types of errors, for example incorrect medication dose or form selected from drop-down menus and duplicate orders for the same medication due to fragmented screen design.^3,4

eMMS have also been shown to cause safety concerns and unintended consequences, including overreliance on technology and workflow changes.^4–6 Without sufficient staff engagement, education, training and feedback, the systems are not sustainable.^7,8 More recently, automation complacency and automation bias, in which users assume the electronic system will be correct rather than relying on their own decision making, have been identified as causes of new errors.^9,10

eMMS implementation in Australia is limited and there is little published evidence about the reduction of errors or potential new errors. A 2012 study on medication errors at two New South Wales (NSW) hospitals with different commercial eMMS demonstrated a reduction in incorrect documentation of medication orders, but considered that other prescribing errors relating to clinical decisions required effective clinical decision support to be integrated into the eMMS in order to prevent clinical errors.¹¹ The same authors published another paper in 2013 demonstrating new errors with eMMS relating to selection from drop-down menus, editing information within the system and performing new tasks.⁴

eMMS in four Victorian public hospitals have been funded by the Department of Health HealthSMART Clinical System and implemented to varying stages, whereas other Victorian hospitals have used different eMMS. We are not aware of other Australian multicentre studies that have investigated medication-related incidents considered to be a new error type or errors that were infrequent or of low significance in the paper-based system but may be more frequent, problematic or severe in the electronic system requiring corrective action. The present study investigated actual reported errors from Victorian public hospitals considered to be new since the implementation of eMMS or different somehow from errors reported in traditional pen-and-paper systems. We also investigated those errors considered likely to occur more frequently with eMMS.

This is a pilot study to gauge whether this adds something to our understanding, albeit knowing incidents are reported voluntarily and underreporting is well recognised.¹² A second part of the present study was an anonymous voluntary survey of frontline doctors, nurses and pharmacists who routinely use eMMS. We anticipate these survey data will build on our incident reporting data if the results align.¹³

The aim of the present study was to identify and quantify medication incidents reportedly related to eMMS and those considered likely to occur more frequently with eMMS. This included developing a new classification system relevant to eMMS errors. We hope this information will be used to inform organisations about potential new error types and to improve systems in order to reduce the likelihood of eMMS-related errors in the future.

The authors, three experienced medication safety pharmacists (members of the Victorian Therapeutics Advisory Group (VicTAG) Quality Use of Medicines) obtained funding from VicTAG, which is an independent, not-for-profit association with the purpose of promoting the quality use of medicines by sharing unbiased, evidence-based information about drug therapy in Victorian hospitals.

A letter from VicTAG was sent to all Victorian public hospitals requesting an expression of interest to be part of this project from any hospital in any stage of eMMS implementation. Ethics approval from one organisation was obtained initially to cover all sites, some of the participating organisations additionally required their own ethics approval with the requirement that all data remained anonymous. Standardised tools for reporting data were designed and piloted by the authors.

Participating hospitals appointed a site project officer to review all reported medication errors and near misses for a 3-month period from May to July 2014 and to identify those incidents they considered were a new error or an error likely to occur more frequently with eMMS based on the description in the incident report. The incidents were classified at each site on a standard spreadsheet developed by the authors with drop-down boxes to ensure responses were consistent. Guidance was provided on the classification system. These project officers usually review incidents, so they were familiar with standard classifications. If the project officer was uncertain about the classification, they included a narrative description to facilitate review.

Project officers were asked to review the Incident Severity Rating (ISR) and alter it if, in their opinion, the ISR did not reflect the actual or potential harm associated with the reported incident. The ISR is part of the Victorian Health Incident Management System and is based upon the degree of effect, the level of care and the treatment required. The ISR grades are as follows: ISR 1, severe or death; ISR 2, moderate; ISR 3, mild; and ISR 4, no harm or near miss.

Other data collected included the step in the medication management process at which the error occurred (e.g. prescribing), the type of medication order involved (e.g. in-patient order) and a brief description of what happened and why the error was new or likely to occur more frequently with eMMS.

The authors jointly reviewed the incidents and included them for further analysis if the incidents were of a clinical nature. Incidents describing a failure to comply with legislative requirements were excluded.

An error-classification system was developed following a literature review.

Traditional analysis of medication errors classifies the error by what went wrong (e.g. wrong drug or omitted dose). We only reviewed errors that were considered new or likely to occur more frequently with eMMS. The traditional classification, used initially, did not meet our needs because we wanted to understand what role eMMS had in each error. Schiff et al.¹⁴ reviewed medication errors from a US database with eMMS as a contributing factor using recruited pharmacists who classified the errors into what happened (e.g. ordered wrong dose or strength), and then evaluated why the error happened. Schiff et al.¹⁴ reported the top 25 codes for types, causes and prevention strategies. The top ‘Why did it happen?’ codes included multiple electronic or hybrid (paper–electronic) systems, user issues such as typing or pull-down menu errors and lack of training or knowledge. Meeks et al.⁵ conducted a review of voluntary reports on electronic health record-related safety concerns using a sociotechnical analysis to understand the nature and context of these concerns. The classification system in that study included human–computer interface, hardware and software and workflow and communication, and then described safety concerns as unmet display needs, software modifications, system–system interfaces and hidden dependencies.⁵

An Australian study classified the errors into system errors, including construction errors, editing errors and new tasks required as a result of electronic prescribing and procedural errors.⁴

Building on classifications described by others combined with our aim for a simplified taxonomy, we developed a new classification system. We wanted categories that could be easily understood and meaningful for organisations and vendors. Qualitative analysis was undertaken on the narrative data supplied by the site project officers using a framework approach. Microsoft (Bellevue, WA, USA) Excel was used to arrange and manage the data.

The authors reviewed each error using the incident reporting classifications, then discussed how eMMS contributed to the error. When necessary, we referred back to the original error report summarised by the project officer, classified the error and cross-checked that all errors could be contained within the categories proposed.

The hierarchy of effectiveness explains that forcing functions is the most sustainable method to prevent errors, with education being the least sustainable; therefore, the workflows in eMMS that are low in the hierarchy (e.g. reliance on human knowledge or memory) may be more prone to error. We describe these as ‘human factors, unfamiliarity or training’. Systems within eMMS that are not forcing functions (e.g. hardware or software issues) that can lead to errors became our categories of ‘hardware malfunction’, ‘system’ and ‘site build’. We chose a separate category for cross-encounter and hybrid system error because these were well known to us (our daily roles include review of medication errors in organisations with eMMS).

Thus, the final classifications are:

• human factors, unfamiliarity or training

• cross-encounter or hybrid system error

• hardware malfunction

• system build

• site build error.

We placed the cross-encounter or hybrid system error together because the contributing factors to some incidents involved issues with both factors. We were unable to further refine the classifications due to limited detail provided in the incident reports. This is possibly due to the lack of understanding and ability to describe the factors contributing to the errors in the new eMMS paradigm combined with the lack of a standardised taxonomy for eMMS.

Seven of the eight hospitals reported incidents relating to eMMS in the study period (May–July 2014). The hospital that did not identify any errors had implemented eMMS for discharge only. A summary of the extent of implementation is provided in Table 1.

Table 1.  Summary of the extent of electronic medication management system (eMMS) implementation and incident numbers
The total medication incidents reported in the study period are the total number of incidents for the entire organisation, not only the areas in which an eMMS was implemented. ED, emergency department; HIH, Hospital-in-the Home

The severity of reported errors was low; there was only one ISR 2 error. This was a cross-encounter or hybrid system error. The patient’s correct medications (charted on the in-patient medication chart) were transcribed into the electronic discharge prescription, but another patient’s medications were also included. The error was identified by the ward pharmacist on checking the discharge prescription, but the medications were not corrected in the discharge summary. Consequently, the medications were prescribed at the patient’s next admission and the patient suffered reversible harm before the mistake was recognised. This error has been investigated and recommendations developed, including checking that the discharge summary has been reconciled with the discharge prescription.

The order type relates to the scope of the eMMS implementation. Only Hospitals 1 and 3 have full implementation, but currently full implementation still requires medication orders on paper in some areas. The other organisations have partial implementation. More errors occurred in the in-patient areas; this is the area with the most complex systems and more staff interaction (Table 2).

Table 2.  Incidents by order type
The Incident Severity Rating (ISR) scale is as follows: ISR 1, severe or death; ISR 2, moderate; ISR 3, mild; and ISR 4, no harm or near miss

Half the reported errors were prescribing errors (n = 47) and the other were half administration errors (n = 46). There were no dispensing errors reported.

We created a new classification system (Tables 3, 4) and compared this with the traditional classification systems (Table 5). Most errors had a human factor component to the error (this includes lack of familiarity of the system or inadequate training).

Table 3.  Incidents by error type (as per the classification system developed herein)
The Incident Severity Rating (ISR) scale is as follows: ISR 1, severe or death; ISR 2, moderate; ISR 3, mild; and ISR 4, no harm or near miss

Table 4.  Error classifications developed for electronic medication management systems (eMMS) with examples
ED, emergency department; eMM, electronic medication management

Table 5.  Number of traditional incident classification types compared with our electronic medication management systems (eMMS) incident classifications

In the traditional classification, the most common error types were wrong patient, wrong timing and duplicate error. We classified all the ‘wrong patient’ errors and the majority of wrong timing and duplicate errors as ‘human factors’; these we also considered ‘likely to occur more frequently’ rather than as new errors because they are known in the paper system. It is a fundamental task for staff to confirm they have the correct patient, but these errors occurred because staff did not exit from one patient before typing in the next patient’s identification number or did not confirm that the patient’s record was open before prescribing or administering medications. The wrong timing errors occurred due to unfamiliarity with the scheduling of doses in eMMS, staff may not have refreshed the screen, not understood how to change a time or not correctly read the time of the previous administration. Duplicate errors were both prescribing and administration errors due to paper and electronic systems in different areas and inadequate handover or not checking the paper chart when a patient returned from surgery or an area without eMMS.

We believe the results of the present study will highlight to organisations the need for ongoing review of system design, refinement of workflow issues, staff education and training and reporting and monitoring of errors. The results reflect the perceptions of the safety of eMMS and medication errors demonstrated by our staff survey¹³ and other studies that have shown that eMMS led to new types of errors or existing errors that may be likely to occur more frequently.^6,9,10

There was a large variation in the percentage of errors reported between organisations (from 0 to 16.7%). However, this is not a true representation of the rate of eMMS incidents because we did not separately collect the total number of incidents in the areas that had implemented eMMS. Our focus was a qualitative analysis of new types of errors with eMMS.

The total number of incidents and those related to eMMS may have been affected by the size of the organisation, whether pharmacist interventions are reported in this system (which increases the total number of errors reported), the extent of implementation (e.g. limited implementation in a multisite organisation), area of implementation (e.g. discharge scripts are reconciled by pharmacists, reducing the risk of error), reporting culture and the timing of this study relative to time of implementation (e.g. a new system prompts staff to report concerns).

Many studies have only reviewed electronic prescribing errors.^3,11 The results of the present study show a similar number of prescribing and administration errors. Nurses are the staff most likely to report incidents, but because we could not identify who reported each incident, we cannot say whether administration errors were reported by nurses and prescribing incidents were reported by medical staff. It has been shown that even if the medication is correctly prescribed within eMMS, the order can be misunderstood and lead to an administration error (e.g. ‘Targin 15/7.5 mg’ being misunderstood and ‘Targin 5/2.5 mg’ being administered). We have received supportive feedback on our classification system from the NSW eMR Connect Program, which has used the system as the basis for its classification for the analysis of medication incidents (R. Worthington, pers. comm.).

A recent publication reviewing UK hospitals that have implemented eMMS used a classification system comprising three categories for new areas of potential safety risks,¹⁵ namely: (1) inadequacies in the system design; (2) inappropriate use of the system; and (3) problems in implementation strategies and infrastructure. These categories are similar to ours (inappropriate use of the system correlating with our ‘human factors’ category), so this is further support for the categories we developed.

It was pleasing to note that were very few reported errors relating to hardware malfunctions. Anecdotally, before eMMS implementation, staff were concerned about power failures and the system ‘going down’.

The two main types of errors we found were due to human factors (including unfamiliarity and training issues) and cross-encounters or hybrid systems. The eMMS removes errors due to poor legibility and incomplete orders, but there are new types of errors due to new tasks and changes to workflow. For example, on a paper medication chart, there are prompts to remind the prescriber to order warfarin, but in the electronic system if the dose changes daily, once the order has been administered it is no longer easily visible to the user. This has resulted in errors when the prescriber omitted to order warfarin the following day. This problem has now been largely rectified by ordering a warfarin reminder.

Staff may not have adequate training to understand how the system works or may have made a mistake due to lack of familiarity with the system instead of using their clinical judgement. This correlates with overdependence on technology.⁶ Our findings identifying human factors as one of the main contributors to errors reflect the recent research that has recognised that human factors engineering can be used to design the work system for safety.¹⁶ These results also correlate with automation complacency and automation bias, which have been described in other research and are now being considered in eMMS.^10,11

The second most commonly reported error type related to either a hybrid system or two encounters in the one admission (when a patient is transferred within an organisation from one unit or ward to another) or an error that occurs in a second admission carried over from a previous admission. The encounter issue is currently being addressed, but until electronic prescribing functionality is available for all areas (e.g. renal dialysis, operating theatres), there remains a risk of hybrid system errors.

Learnings from medication errors and staff experiences are fundamental tools to improve patient safety. The eMMS-related errors we identified are of value to other organisations and system providers for education and training. A system for ongoing review of medication-related errors by medication safety pharmacists and medication safety governance committees is important to detect eMMS issues and opportunities for improvement.

Organisations that have implemented eMMS have responded to identified errors by improvements, including the warfarin reminder and encounter issues described previously. Pharmacists are now employed on an ongoing basis for system maintenance (e.g. addition of new drugs) and to remedy issues (e.g. product shortages). Workflows have been developed and refined to overcome gaps (e.g. automatic discontinuation of some orders when a patient is transferred from a high acuity area to a ward). Wrong patient, wrong timing and duplicate orders are being addressed with training and education, and expanding eMMS to minimise the areas with paper.

This was a pilot project using data for a 3-month period; a longer time would be required to confirm these results. It is recognised that incidents are underreported and there is variation in what is reported. Reporting culture and staff training result in variations in the quality and quantity of errors reported at organisations. A more rigorous methodology would have been to employ one project officer to select and review incidents at all sites, but privacy and funding constraints precluded this. Direct observation would be an even more robust method to detect the true rate of incidents.

Restricted fields in incident reporting systems may limit the ability to determine whether eMMS was a factor in the incident. Analysis was further complicated by the various eMMS, as well as stages and scope of implementation at participating hospitals.

It was beyond the scope of the present study to compare pre- and post-eMMS incident data or to analyse the root cause of the reported individual incidents.

Although eMMS technology removes the opportunity for many medication error types, others may occur more frequently and new error types may emerge. Although the results of the present study suggest the errors reported were of low severity, continuous monitoring of errors is important. Organisations must remain vigilant to the risk of new errors and avoid the assumption that eMMS is the panacea to all medication error issues. Our new classifications for eMMS could be a model for others.

None of the authors have any conflicting interests to report.