Effects of eHealth on hospital practice: synthesis of the current literature

eHealth technologies, including electronic medical records (EMR), computerised provider order entry (CPOE), ePrescribing and computerised decision support systems (CDSS), are promoted for their financial and clinical benefits. This has led eHealth to become central to many government agendas worldwide,¹ with 75% of US hospitals implementing EMRs.² However, negative unintended consequences are being increasingly reported, with clinicians using eHealth technologies in unanticipated ways.³ Moreover, some clinicians resist using these systems⁴ and develop workarounds compromising patient care.⁵ This has prompted the US government to incentivise ‘meaningful use’ of eHealth technologies, but it is still uncertain how effective these incentives have been.

The transition to digital hospitals is fast moving. Although US hospitals are further ahead than some others in implementing eHealth technologies, their early experiences are not necessarily generalisable to contemporary healthcare because both the systems and technologies are rapidly evolving. This constantly changing environment, coupled with the heterogeneity in reported eHealth effects, means it is important to provide up-to-date assessments of the available evidence. The aim of this paper is to provide an assessment of the current literature on the effects to be expected from hospital implementations of eHealth technologies.⁶ This paper provides the detailed background evidence for our recently published Deeble Institute Evidence Brief.⁷

Keasberry et al.⁶ provided a narrative review of systematic reviews of the effects of EMR, CPOE, CDSS and ePrescribing published between 1 January 2010 and 31 October 2015. We update this review by replicating their search methods to review studies published from 1 November 2015 to 1 August 2017. We used the same search strategy, databases (PubMed, Medline, Cochrane) and inclusion and exclusion criteria as Keasberry et al.⁶ As in that study, remote health and patient-focused eHealth systems were excluded, as were reviews: (1) pertaining to a single discipline, investigation, medicine or vendor; (2) focusing on implementation only; and (3) conducted predominantly in non-hospital settings or developing countries. (For the detailed search strategy used for PubMed as well as the complete list of the inclusion and exclusion criteria used in the present study, readers are referred to the supplementary material in Keasberry et al.⁶)

Our search retrieved 563 articles. One author (RE) read all abstracts and coded them for inclusion and exclusion, whereas another author (ABJ) read and coded one-third of them. Due to high inter-rater reliability between the two authors’ codes (Cohen’s kappa 0.90), the first author’s (RE) coding was used for all abstracts. Through this process, 39 articles were identified as potentially relevant for inclusion. Two authors (RE, ABJ) then conducted a full-text review of each article for inclusion or exclusion. Their assessments were highly similar (Cohen’s kappa 0.84), with disagreements resolved by consensus. Through this process, seven articles were determined to be relevant. In four of the seven articles, the authors explicitly described their study as a systematic review. Three were not described as systematic reviews but were performed in a manner very similar to systematic reviews⁸ and thus were included. The 32 full-text articles that were excluded are listed in Table S1, available as Supplementary Material to this paper. Fig. 1 depicts the final Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram for inclusion and exclusion.

Fig. 1.  Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of study selection. EMR, electronic medical record.

Each article was assessed for quality using AMSTAR (a measurement tool to assess systematic reviews).⁹ In accordance with Long et al.,¹⁰ an AMSTAR score ≥8 was determined to be a high quality review, a score of 4–7 was taken to indicate intermediate quality, and scores <4 were taken to indicate low quality.

The previous narrative review⁶ acknowledged the heterogeneity of studies and opted to present the findings using a narrative synthesis. We follow that approach but also categorise the effects into a typology of outcome measures. Specifically, we adapted the typology of Black et al.,¹¹ which details the benefits and harms of CPOE, CDSS, ePrescribing and EMRs. For effects that reflect a range over a single dimension (e.g. cost-savings and costs, which reflect a range of financial effects), we used a single outcome category (e.g. costs) and coded effects as positive or negative if more than half the studies in a review reported positive or negative results respectively, in line with previous reviews.¹²

Fig. 2 illustrates the adapted typology of outcome measures.¹¹ According to the typology, EMRs are associated with benefits of accessibility, completeness, legibility, organisational efficiency, secondary use of data and considerations over data security, time, costs, communication with patients and paper persistence. CPOE are associated with improvements in resource utilisation, indicated care and patient outcomes, as well as issues related to workarounds, interruptions, cost and time delays. ePrescribing is associated with improvements in guideline adherence, safer prescribing and clinician communication, as well as considerations related to patient outcomes, time and costs. Finally, CDSS is associated with improvements in indicated care, guideline adherence, surrogate outcomes and considerations into patient outcomes and individual clinician performance.

Fig. 2.  Adapted typology of outcomes. CDSS, clinical decision support systems; CPOE, computerised provider order entry; EMR, electronic medical record.

We first categorised the effects reported in the narrative review⁶ into the typology of outcome measures (Fig. 2) and identified any other emergent categories. Next, the effects reported in the updated studies were classified and compared with the findings reported in Keasberry et al.⁶

Of the seven relevant articles, three (42.9%) examined the effects of more than one eHealth system:^3,5,12 CDSS were studied in six (85.7%) articles,^3,5,12–15 EMRs were studied in three (42.9%),^3–5 ePrescribing was studied in three (42.9%)^3,5,12 and CPOE was studied in two (28.6%).^3,5 Only two studies (28.6%) were high quality;^13,14 three (42.9%) were of intermediate quality^4,12,15 and two (28.6%) were of low quality.^3,5 The characteristics of the studies included are given in Appendix 1.

By way of contrast, the previous review by Keasberry et al.⁶ identified 21 articles, most of which referenced multiple technologies. According to appendices 1 and 2 in Keasberry et al.,⁶ four studies analysed the effects of multiple eHealth systems, five analysed EMRs, two investigated CPOE, four examined ePrescribing and 19 assessed CDSS.

The following sections detail the effects of each eHealth system and compare them with those reported by Keasberry et al.⁶ Table 1 summarises the overall results.

Table 1.  Effects of clinical decision support systems (CDSS), computerised provider order entry (CPOE), electronic medical records (EMRs) and ePrescribing systems reported in the literature
Blank cells indicate that the effect has not been investigated for the specific eHealth system. +, positive effect; –, negative effect; +/–, mixed positive and negative effects; 0, no evidence of effect; +/0, mixed positive and neutral effects

Fourteen EMR effects were investigated in the updated review articles. Although eight of these effects were present in the previous review,⁶ there were differences in valence (i.e. positive, negative, neutral or mixed) in the more recent articles. Overall, the updated results presents a more complex array of mixed and conflicting results when it comes to EMR benefits than reported previously.⁶ For example, in the previous review completeness of information was improved by EMR,⁶ yet the updated review found the opposite findings, with one low-quality study³ and one intermediate-quality study⁴ reporting negative results. Rathert et al.⁴ reported that completeness of information (e.g. documentation of all relevant information of the patient’s journey) in EMRs can be compromised with emotional and psychosocial information being overlooked as patients withhold relevant information due to their negative data security perceptions. Furthermore, Zheng et al.³ identified that EMRs were not capable of recording all required health information. Similarly, data integrity was positive in the previous review,⁶ yet an intermediate-quality study found mixed results reporting better-quality biomedical but poorer-quality psychosocial information in patient records.⁴ The same study⁴ highlighted the presence of unintended data entry errors compromising record quality, with similar findings reported by Zheng et al.³

The positive results in the previous review for costs, organisational efficiency, overall perception and mixed results for data security were all found to be either negative or mixed findings in the updated review, although these mostly arose from two low-quality articles^3,5 that only examined unintended consequences. These same studies also reported some additional negative results not reported previously, including guideline adherence, staff retention, clinician productivity, financial effects, organisational efficiency, changing practices and workarounds.³ However, because these studies were specifically looking for unintended consequences, which are most likely to constitute undesirable effects,^5,16 they run the risk of ascertainment bias in overidentifying and overrating such effects compared with previous studies.

Communication was another effect not reported in Keasberry et al.⁶ One intermediate-quality study reported mixed results for communication between clinicians and patients,⁴ finding EMRs have changed the nature of patient encounters. The study identified that physicians need to actively maintain eye contact and recognise non-verbal cues while interacting with the EMR. On the upside, the same study reported that improved communication can result when physicians actively engage patients by showing them their own data using the EMR.⁴

Findings regarding patient outcomes painted a more complex picture compared with the previous review, which reported insufficient evidence.⁶ Positive patient outcomes were found in an intermediate-quality study,⁴ although negative findings were observed in a low-quality study investigating unintended consequences.³ The former were attributed to EMRs facilitating clinician decision making,⁴ whereas the latter derived from the unsafe use of EMRs with workarounds, continual copy and pasting, and clinician overreliance on the EMR.

Only two studies in our updated review examined the effect of CPOE. Both were of low quality, examined only unintended consequences and only examined CPOE in the presence of other eHealth systems. Both reviews focused only on one effect, communication among clinicians, with one review reporting mixed³ and one reporting negative⁵ findings. Although Zheng et al.³ identified that CPOE use can facilitate proactive communication between clinicians, Kuziemsky et al.⁵ found that errors in CPOE can result in miscommunication between clinicians.

Communication among clinicians was not reported in Keasberry et al.,⁶ although their review identified other CPOE effects not examined in the recent studies, such as cost, individual clinician performance, interruptions, organisational efficiency, patient outcomes, resource utilisation, time and workarounds.

Three studies investigated ePrescribing in the updated review, but all within the context of ePrescribing integrated with other EMR functions. Two studies were of low quality and investigated unintended consequences of eHealth,^3,5 and one study was of intermediate quality.¹² Only three effects were investigated: communication among clinicians, patient outcomes and safer prescribing. All three were investigated in the previous review and were found to be positive.⁶ However, studies in the present review reported mixed, negative and neutral results for these three effects respectively.

With regard to communication among clinicians, Kuziemsky et al.⁵ indicated that communication processes involving clinicians were negatively affected and that protocols needed to be put in place to ensure effective exchange of information. For patient outcomes, Zheng et al.³ found mixed results, with some studies reporting a reduction in adverse drug events whereas others reported an increase. Moreover, Zheng et al.³ found negative results for safer prescribing, whereas Nabovati et al.¹² reported no significant effect of ePrescribing on dangerous drug interactions and safer prescribing. However, when ePrescribing was paired with CDSS, more positive results were reported,¹² which are discussed in the following section.

Overall, six studies investigated CDSS, with two each being of low,^3,5 intermediate^12,15 and high quality.^13,14 In total, 12 effects were observed in the recent studies for CDSS. In intermediate- and high-quality reviews, positive effects were found for guideline adherence,^12,14 indicated care,^12–15 organisational efficiency,^14,15 clinicians’ overall perceptions of the CDSS,^13–15 safer prescribing¹² and patient outcomes.^11,14,15 This is in agreement with findings in the previous review.⁶ Other positive effects not reported previously⁶ include improved data integrity and clinical judgment in one high-quality study.¹⁴ In terms of data integrity, Dunn Lopez et al.¹⁴ found significant improvements in the accuracy of information and the ability of clinicians to accurately interpret CDSS content.

In the previous review,⁶ CDSS either reduced or had no effect on healthcare cost, but in the updated review no evidence was found of this effect. For example, Cook et al.¹³ analysed CPOE integrated with CDSS by means of built-in infobuttons and found no significant difference in order costs compared with stand-alone CPOE. Two new negative effects of CDSS were found, with one low-quality review reporting changes in practice,⁵ which can lead to risky workarounds, and another high-quality review reporting persistent use of paper forms.¹⁴

This paper provides a contemporaneous account of the effects of eHealth technologies in hospital practice and classifies the effects into a typology of outcomes.¹¹ A broad range of effects was reported for CDSS and EMRs, whereas there was limited evidence for ePrescribing and CPOE, which were only investigated in conjunction with other eHealth technologies. The quality of most reviews was limited, with no high-quality reviews relating to CPOE, EMRs and ePrescribing. In contrast, high-quality reviews supported the positive effects of CDSS. The studies analysed were highly heterogeneous, which precluded attempts at meta-analysis, and, accordingly, results are only reported as a narrative synthesis

We found that the effects of EMRs were potentially not as straightforward as reported in the previous review,⁶ with more mixed results apparent in the updated findings. This may be explained by the presence of more studies investigating unintended consequences in the present than previous review. However, when EMR systems were integrated with auxiliary technologies, such as ePrescribing and CDSS, more positive effects on patient outcomes and safer prescribing were obtained. In contrast with the previous review,⁶ new effects were observed in the present review, including changing practice, clinical judgement and staff recruitment.

Because the present study used the approach of Keasberry et al.,⁶ similar limitations exist, including the potential for reviews to be missed, although this is considered unlikely, omission of grey literature, although this has a higher risk of bias, and emphasis on effects devoid of consideration over behaviour and implementation factors. Although decisions over inclusion and exclusion of individual reviews were decided by researcher judgment, which could be potentially biased, the levels of inter-rater agreement for review selection were high. In terms of study strengths, although we followed Keasberry et al.⁶ in limiting our review to systematic reviews, we also included other reviews where their methods approximated systematic reviews to minimise bias and improve the completeness of findings. Like Keasberry et al.,⁶ we used the AMSTAR criteria to judge the quality of reviews, but acknowledge ongoing discussions of the usefulness of AMSTAR and other quality criteria¹⁷ specifically developed for application to evaluation studies of health informatics, such as STARE-HI (STAtement on the Reporting of Evaluation studies in Health Informatics).¹⁸

The effects reported by Keasberry et al.⁶ and here in the updated review were classified according to the particular eHealth system being investigated and, where possible, to the typology of outcomes reported by Black et al.¹¹ This classification is both a strength and limitation. Classifying effects according to each system takes into account their different purposes and functions, thus conferring greater granularity to the findings. However, some of the studies that were analysed in the review looked at eHealth systems as a whole and did not attempt to clearly distinguish between the effects of separate systems. Therefore, subjectivity was involved when decomposing the effects according to the system. Still, classifying the benefits into a typology provided a way to visualise, compare and contrast the effects reported. We should also note that most studies within the reviews analysed examined the effects of eHealth in the US context, which therefore limits the generalisability to other jurisdictions. Given the rapid digital transformations taking place in Australian hospitals, more research on the effects and experiences of eHealth in this country is urgently needed.

Our findings can help hospitals and health authorities know what to expect from the implementation of EMRs and other major eHealth technologies. The authors have been involved in a state-wide roll-out of an integrated eHealth system that includes all technologies of EMR, CPOE, CDSS and ePrescribing. The findings of the present study have been communicated to hospital and state health authorities and were considered helpful in identifying benefits and areas for improvement, as well as guiding future implementations and assessments at other sites.

In summary, it appears that optimisation of completeness and accuracy of information in eHealth systems can still be a challenge due to ineffective workarounds and user resistance. There is limited evidence to suggest that CPOE and EMRs directly improve clinical outcomes, but ePrescribing, when integrated with CDSS, does improve outcomes through safer prescribing behaviours. This updated review also highlights additional effects that hospitals should consider, including changing practice, clinical judgement and staff retention.

The present study can help inform other hospitals of the likely effects resulting from eHealth technologies (e.g. EMRs, CDSS, CPOE and ePrescribing). Overall results are encouraging for the ongoing digital transformation of hospital practice.

The findings in this review indicate that hospitals experience positive clinical and financial outcomes from implementing eHealth technologies, although increased attention is being given to unintended detrimental consequences, such as workarounds and paper persistence. Future research is needed to understand why these positive and negative effects occur. Improving governance structures and optimising the effective use of these eHealth technologies are two potential areas that should be investigated to minimise negative effects. Addressing these issues is of great importance for hospitals and decision makers, because successfully implementing eHealth will enable effective patient care in an increasingly complex environment.

None declared.