From benefits idealisation to value optimisation: application in the digital health context
Natalie Smith
A D , Andrew Burton-Jones A and Clair Sullivan B C
+ Author Affiliations
- Author Affiliations
A UQ Business School, Blair Drive, St Lucia, Qld 4072, Australia. Email: abj@business.uq.edu.au
B Metro North Hospital and Health Service, Royal Brisbane and Women’s Hospital, Herston Road, Qld 4006, Australia. Email: clair.sullivan@health.qld.gov.au
C Present address: School of Medicine, The University of Queensland, 288 Herston Road, Herston, Qld 4006, Australia.
D Corresponding author. Email: natalie.smith@business.uq.edu.au
Australian Health Review 44(5) 706-722 https://doi.org/10.1071/AH19255
Submitted: 22 November 2019 Accepted: 21 February 2020 Published: 28 September 2020
Abstract
Objective This study investigated evidence for the approach known as ‘benefits management’ (BM) used in many digital hospital initiatives.
Methods A qualitative narrative overview was conducted on the BM literature and compared with a qualitative systematic overview of electronic medical record (EMR) implementation literature.
Results Twenty-five articles on BM and 12 literature reviews on EMR implementation were examined. The BM approach does not have strong support in the literature and does not support all the needs of large EMR implementations.
Conclusion The current BM approach provides an inadequate basis for managing and reporting on the outcomes that ensue from a digital hospital initiative. A shift is needed from benefits idealisation to value optimisation.
What is known about the topic? Health services are under increasing pressure to demonstrate that the benefits anticipated from digital health investments have been realised.
What does this paper add? This paper informs the practice of benefits governance in EMR implementations. The results reveal inadequacies in current BM models and practice that are currently enshrined in policy despite a lack of evidence.
What are the implications for practitioners? Health service leaders must be willing to question the governance of benefits from health service transformations using more evidence-based approaches to increase the value obtained from investments in digital transformation.
References
[1] Greenhalgh T, Russell J, Ashcroft R, Parsons W. Why national eHealth programs need dead philosophers: Wittgensteinian reflections on policymakers’ reluctance to learn from history. Milbank Q 2011; 89 533–63.| Why national eHealth programs need dead philosophers: Wittgensteinian reflections on policymakers’ reluctance to learn from history.Crossref | GoogleScholarGoogle Scholar | 22188347PubMed |
[2] Bae J, Rask K, Becker E. The impact of electronic medical records on hospital-acquired adverse safety events: differential effects between single-source and multiple-source systems. Am J Med Qual 2018; 33 72–80.
| The impact of electronic medical records on hospital-acquired adverse safety events: differential effects between single-source and multiple-source systems.Crossref | GoogleScholarGoogle Scholar | 28387525PubMed |
[3] Eden R, Burton-Jones A, Scott I, Staib A, Sullivan C. Effects of eHealth on hospital practice: synthesis of the current literature. Aust Health Rev 2018; 42 568–78.
| Effects of eHealth on hospital practice: synthesis of the current literature.Crossref | GoogleScholarGoogle Scholar | 29986809PubMed |
[4] Greenhalgh T, Russell J. Why do evaluations of eHealth programs fail? An alternative set of guiding principles. PLoS Med 2010; 7 e1000360
| Why do evaluations of eHealth programs fail? An alternative set of guiding principles.Crossref | GoogleScholarGoogle Scholar | 21072245PubMed |
[5] Enam A, Torres-Bonilla J, Eriksson H. Evidence-based evaluation of ehealth interventions: systematic literature review. J Med Internet Res 2018; 20 e10971
| Evidence-based evaluation of ehealth interventions: systematic literature review.Crossref | GoogleScholarGoogle Scholar | 30470678PubMed |
[6] Scott IA, Sullivan C, Staib A. Going digital: a checklist in preparing for hospital-wide electronic medical record implementation and digital transformation. Aust Health Rev 2018; 43 302–13.
| Going digital: a checklist in preparing for hospital-wide electronic medical record implementation and digital transformation.Crossref | GoogleScholarGoogle Scholar |
[7] Ben-Zion R, Pliskin N, Fink L. Critical success factors for adoption of electronic health record systems: literature review and prescriptive analysis. Inf Syst Manage 2014; 31 296–312.
| Critical success factors for adoption of electronic health record systems: literature review and prescriptive analysis.Crossref | GoogleScholarGoogle Scholar |
[8] Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J 2009; 26 91–108.
| A typology of reviews: an analysis of 14 review types and associated methodologies.Crossref | GoogleScholarGoogle Scholar | 19490148PubMed |
[9] Templier M, Paré G. A framework for guiding and evaluating literature reviews. Comm Assoc Inform Syst 2015; 37 112–37.
[10] Peppard J, Ward J, Daniel E. Managing the realization of business benefits from IT investments. MIS Q Exec 2007; 6 1–11.
[11] Department of Finance, Services and Innovation. Benefits realisation management framework: parts 1 to 5. Sydney: State of New South Wales, Department of Finance, Services and Innovation; 2018.
[12] Queensland Government. Queensland Government methodologies: benefits management release 2.0. Brisbane: State of Queensland, Queensland Government Chief Information Office (QGCIO); 2013.
[13] Department of Finance, Services and Innovation. Benefits realisation management framework. 2020. Available at: https://www.finance.nsw.gov.au/publication-and-resources/benefits-realisation-management-framework [verified 1 September 2020].
[14] Queensland Government Chief Information Office (QGCIO). Benefits, business changes and enablers. 2018. Available at: https://www.qgcio.qld.gov.au/information-on/digital-and-ict-strategic-planning/benefits-business-changes-and-enablers [verified 1 September 2020].
[15] Keasberry J, Scott I, Sullivan C, Staib A, Ashby R. Going digital: a narrative overview of the clinical and organisational impacts of eHealth technologies in hospital practice. Aust Health Rev 2017; 41 646–64.
| Going digital: a narrative overview of the clinical and organisational impacts of eHealth technologies in hospital practice.Crossref | GoogleScholarGoogle Scholar | 28063462PubMed |
[16] Shea BJ, Reeves B, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017; 358 j4008
| AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.Crossref | GoogleScholarGoogle Scholar | 28935701PubMed |
[17] Breese R. Benefits realisation management: panacea or false dawn? Int J Proj Manag 2012; 30 341–51.
| Benefits realisation management: panacea or false dawn?Crossref | GoogleScholarGoogle Scholar |
[18] Ward J, Taylor P, Bond P. Evaluation and realisation of IS/IT benefits: an empirical study of current practice. Eur J Inf Syst 1996; 4 214–25.
| Evaluation and realisation of IS/IT benefits: an empirical study of current practice.Crossref | GoogleScholarGoogle Scholar |
[19] Breese R, Jenner S, Serra C, Thorp J. Benefits management: lost or found in translation. Int J Proj Manag 2015; 33 1438–51.
| Benefits management: lost or found in translation.Crossref | GoogleScholarGoogle Scholar |
[20] Laursen M, Svejvig P. Taking stock of project value creation: a structured literature review with future directions for research and practice. Int J Proj Manag 2016; 34 736–47.
| Taking stock of project value creation: a structured literature review with future directions for research and practice.Crossref | GoogleScholarGoogle Scholar |
[21] Coombs C. When planned IS/IT project benefits are not realized: a study of inhibitors and facilitators to benefits realization. Int J Proj Manag 2015; 33 363–79.
| When planned IS/IT project benefits are not realized: a study of inhibitors and facilitators to benefits realization.Crossref | GoogleScholarGoogle Scholar |
[22] Wilson H, Clark M, Smith B. Justifying CRM projects in a business-to-business context: the potential of the benefits dependency network. Ind Mark Manage 2007; 36 770–83.
| Justifying CRM projects in a business-to-business context: the potential of the benefits dependency network.Crossref | GoogleScholarGoogle Scholar |
[23] Remenyi D, Sherwood-Smith M. Business benefits from information systems through an active benefits realisation programme. Int J Proj Manag 1998; 16 81–98.
| Business benefits from information systems through an active benefits realisation programme.Crossref | GoogleScholarGoogle Scholar |
[24] Lin C, Pervan G. The practice of IS/IT benefits management in large Australian organizations. Inf Manage 2003; 41 13–24.
| The practice of IS/IT benefits management in large Australian organizations.Crossref | GoogleScholarGoogle Scholar |
[25] Marnewick C. Benefits of information system projects: the tale of two countries. Int J Proj Manag 2016; 34 748–60.
| Benefits of information system projects: the tale of two countries.Crossref | GoogleScholarGoogle Scholar |
[26] Serra C, Kunc M. Benefits realisation management and its influence on project success and on the execution of business strategies. Int J Proj Manag 2015; 33 53–66.
| Benefits realisation management and its influence on project success and on the execution of business strategies.Crossref | GoogleScholarGoogle Scholar |
[27] Terlizzi M, Albertin A, de Moraes H. IT benefits management in financial institutions: practices and barriers. Int J Proj Manag 2017; 35 763–82.
| IT benefits management in financial institutions: practices and barriers.Crossref | GoogleScholarGoogle Scholar |
[28] Doherty N, Ashurst C, Peppard J. Factors affecting the successful realisation of benefits from systems development projects: findings from three case studies. J Inf Technol 2012; 27 1–16.
| Factors affecting the successful realisation of benefits from systems development projects: findings from three case studies.Crossref | GoogleScholarGoogle Scholar |
[29] Dupont D, Eskerod P. Enhancing project benefit realization through integration of line managers as project benefit managers. Int J Proj Manag 2016; 34 779–88.
| Enhancing project benefit realization through integration of line managers as project benefit managers.Crossref | GoogleScholarGoogle Scholar |
[30] Ashurst C, Doherty N, Peppard J. Improving the impact of IT development projects: the benefits realization capability model. Eur J Inf Syst 2008; 17 352–70.
| Improving the impact of IT development projects: the benefits realization capability model.Crossref | GoogleScholarGoogle Scholar |
[31] Silverman B. Project appraisal methodology: a multidimensional R&D benefit/cost assessment tool. Manage Sci 1981; 27 802–21.
| Project appraisal methodology: a multidimensional R&D benefit/cost assessment tool.Crossref | GoogleScholarGoogle Scholar |
[32] Lefley F. An assessment of various approaches for evaluating project strategic benefits: recommending the strategic index. Manage Decis 2004; 42 850–62.
| An assessment of various approaches for evaluating project strategic benefits: recommending the strategic index.Crossref | GoogleScholarGoogle Scholar |
[33] Zwikael O. International Journal of Project Management special issue on ‘project benefit management’. Int J Proj Manag 2016; 34 734–5.
| International Journal of Project Management special issue on ‘project benefit management’.Crossref | GoogleScholarGoogle Scholar |
[34] Musawir AU, Serra CEM, Zwikael O, Ali I. Project governance, benefit management, and project success: towards a framework for supporting organizational strategy implementation. Int J Proj Manag 2017; 35 1658–72.
| Project governance, benefit management, and project success: towards a framework for supporting organizational strategy implementation.Crossref | GoogleScholarGoogle Scholar |
[35] Badewi A, Shehab E, Zeng J, Mohamad M. ERP benefits capability framework: orchestration theory perspective. Bus Process Manag J 2018; 24 266–94.
| ERP benefits capability framework: orchestration theory perspective.Crossref | GoogleScholarGoogle Scholar |
[36] Badewi A. The impact of project management (PM) and benefits management (BM) practices on project success: towards developing a project benefits governance framework. Int J Proj Manag 2016; 34 761–78.
| The impact of project management (PM) and benefits management (BM) practices on project success: towards developing a project benefits governance framework.Crossref | GoogleScholarGoogle Scholar |
[37] Badewi A, Shehab E. The impact of organizational project benefits management governance on ERP project success: neo-institutional theory perspective. Int J Proj Manag 2016; 34 412–28.
| The impact of organizational project benefits management governance on ERP project success: neo-institutional theory perspective.Crossref | GoogleScholarGoogle Scholar |
[38] Chih Y, Zwikael O. Project benefit management: a conceptual framework of target benefit formulation. Int J Proj Manag 2015; 33 352–62.
| Project benefit management: a conceptual framework of target benefit formulation.Crossref | GoogleScholarGoogle Scholar |
[39] Zwikael O, Chih Y, Meredith J. Project benefit management: setting effective target benefits. Int J Proj Manag 2018; 36 650–8.
| Project benefit management: setting effective target benefits.Crossref | GoogleScholarGoogle Scholar |
[40] Organisation for Economic Cooperation and Development (OECD). G20/OECD principles of corporate governance. Paris: OECD; 2015.
[41] Alshahrani A, Stewart D, MacLure K. A systematic review of the adoption and acceptance of eHealth in Saudi Arabia: views of multiple stakeholders. Int J Med Inform 2019; 128 7–17.
| A systematic review of the adoption and acceptance of eHealth in Saudi Arabia: views of multiple stakeholders.Crossref | GoogleScholarGoogle Scholar | 31160014PubMed |
[42] Greenhalgh T, Potts H, Wong G, Bark P, Swinglehurst D. Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method. Milbank Q 2009; 87 729–88.
| Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method.Crossref | GoogleScholarGoogle Scholar | 20021585PubMed |
[43] Nguyen L, Bellucci E, Nguyen LT. Electronic health records implementation: an evaluation of information system impact and contingency factors. JMIR J Med Inform 2014; 83 779–96.
| Electronic health records implementation: an evaluation of information system impact and contingency factors.Crossref | GoogleScholarGoogle Scholar |
[44] Granja C, Janssen W, Johansen M. Factors determining the success and failure of ehealth interventions: systematic review of the literature. J Med Internet Res 2018; 20 e10235
| Factors determining the success and failure of ehealth interventions: systematic review of the literature.Crossref | GoogleScholarGoogle Scholar | 29716883PubMed |
[45] Dobrow MJ, Bytautas J, Tharmalingam S, Hagens S. Interoperable electronic health records and health information exchanges: systematic review. J Med Internet Res 2019; 7 e12607
| Interoperable electronic health records and health information exchanges: systematic review.Crossref | GoogleScholarGoogle Scholar |
[46] Castillo VH, Martínez-García A, Pulido J. A knowledge-based taxonomy of critical factors for adopting electronic health record systems by physicians: a systematic literature review. BMC Med Inform Decis Mak 2010; 10 60
| A knowledge-based taxonomy of critical factors for adopting electronic health record systems by physicians: a systematic literature review.Crossref | GoogleScholarGoogle Scholar | 20950458PubMed |
[47] Boonstra A, Versluis A, Vos J. Implementing electronic health records in hospitals: a systematic literature review. BMC Health Serv Res 2014; 14 370
| Implementing electronic health records in hospitals: a systematic literature review.Crossref | GoogleScholarGoogle Scholar | 25190184PubMed |
[48] Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res 2010; 10 231
| Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions.Crossref | GoogleScholarGoogle Scholar | 20691097PubMed |
[49] McGinn CA, Grenier S, Duplantie J, Shaw N, Sicotte C, Mathieu L, Leduc Y, Légaré F, Gagnon M-P. Comparison of user groups’ perspectives of barriers and facilitators to implementing electronic health records: a systematic review. BMC Med 2011; 9 46
| Comparison of user groups’ perspectives of barriers and facilitators to implementing electronic health records: a systematic review.Crossref | GoogleScholarGoogle Scholar | 21524315PubMed |
[50] Fritz F, Tilahun B, Dugas M. Success criteria for electronic medical record implementations in low-resource settings: a systematic review. J Am Med Inform Assoc 2015; 22 479–88.
| Success criteria for electronic medical record implementations in low-resource settings: a systematic review.Crossref | GoogleScholarGoogle Scholar | 25769683PubMed |
[51] Wakefield BJ, Turvey CL, Nazi KM, Holman JE, Hogan TP, Shimada SL, Kennedy DR. Psychometric properties of patient-facing eHealth evaluation measures: systematic review and analysis. J Med Internet Res 2017; 19 e346
| Psychometric properties of patient-facing eHealth evaluation measures: systematic review and analysis.Crossref | GoogleScholarGoogle Scholar | 29021128PubMed |
[52] Santhanam R, Hartono E. Issues in linking information technology capability to firm performance. Manage Inf Syst Q 2003; 27 125–53.
| Issues in linking information technology capability to firm performance.Crossref | GoogleScholarGoogle Scholar |
[53] ul Musawir A, Serra C, Zwikael O, Ali I. Project governance, benefit management, and project success: towards a framework for supporting organizational strategy implementation. Int J Proj Manag 2017; 35 1658–72.
| Project governance, benefit management, and project success: towards a framework for supporting organizational strategy implementation.Crossref | GoogleScholarGoogle Scholar |
[54] Bennington P, Baccarini D. Project benefits management in IT projects – an Australian perspective. Proj Manage J 2004; 35 20–30.
| Project benefits management in IT projects – an Australian perspective.Crossref | GoogleScholarGoogle Scholar |
[55] Casey R, Waring T. Towards a critical approach to benefits realisation of information systems in the NHS. Oxford: UK Academy for Information Systems Conference Proceedings; 2014. Paper 41. Available at: https://aisel.aisnet.org/ukais2014/41/ [verified 16 September 2020].
[56] Ward J, Daniel E. Benefits management: how to increase the business value of your IT projects. Chichester: John Wiley & Sons; 2012.
[57] Flyvbjerg B, Holm MS, Buhl S. Underestimating costs in public works projects: error or lie? J Am Plann Assoc 2002; 68 279–95.
| Underestimating costs in public works projects: error or lie?Crossref | GoogleScholarGoogle Scholar |
[58] Changchit C, Joshi K, Lederer A. Process and reality in information systems benefit analysis. Inf Syst J 1998; 8 145–62.
| Process and reality in information systems benefit analysis.Crossref | GoogleScholarGoogle Scholar |
[59] Ward J, Daniel E. Benefits management: delivering value from IS & IT investments. Chichester: John Wiley & Sons; 2005.
[60] Bradley G. Benefit realisation management a practical guide to achieving benefits through change. 2nd edn. Farnham: Ashgate Publishing; 2010.
[61] Ward J, Murray P. Benefits management: best practice guidelines. Cranfield: Information Systems Research Centre; 1997.
[62] Queensland Treasury. Project assessment framework: benefits realisation. 2015. Available at: https://s3.treasury.qld.gov.au/files/paf-benefits-realisation.pdf [verified 31 August 2020].
[63] Building Queensland. Business case development framework: benefits management framework. Release 2, December 2016. 2016. Available at: https://buildingqueensland.qld.gov.au/wp-content/uploads/2016/12/Benefits-Management-Framework-1.pdf [verified 31 August 2020].
[64] Queensland Treasury. Project assessment framework. 2019. Available at: https://www.treasury.qld.gov.au/resource/project-assessment-framework/ [verified 31 August 2020].
[65] Projects Queensland. Benefits realisation: gate 5. 2013. Available at: https://s3.treasury.qld.gov.au/files/gateway-review-process-gate-5-benefits-realisation.pdf [verified 31 August 2020].
[66] eHealth NSW. eMR Connect benefits realisation framework. 2020. Available at: https://mnclhd.health.nsw.gov.au/i/ehealth/wp-content/uploads/sites/6/Benefits-Realisation-Framework.pdf [verified 31 August 2020].
[67] Health Design Authority. Benefits realisation core report: guide for Victorian public health services. Melbourne: State of Victoria, Department of Health; 2013.
[68] Victoria State Government Treasury and Finance. Benefit delivery. 2018. Available at: https://www.dtf.vic.gov.au/applications-investment-management-standard/benefit-delivery [verified 31 August 2020].
[69] Victoria Treasury and Finance. Investment management standard 2017. Melbourne: Victorian Department of Treasury and Finance; 2017.
[70] Department of Premier and Cabinet. ICT, digital and cyber security policies and guidelines South Australia. 2019. Available at: https://www.dpc.sa.gov.au/responsibilities/ict-digital-cyber-security/policies-and-guidelines [verified 31 August 2020].
[71] South Australian Government. Guidelines for evaluation of public sector initiatives. Adelaide: Government of South Australia, Department of Treasury and Finance; 2014.
[72] Government of Western Australia. Gateway review process and gateway reviewer training. 2017. Available at: https://www.wa.gov.au/service/government-financial-management/procurement/gateway-review-process-and-gateway-reviewer-training [verified 31 August 2020].
[73] AXELOS. Managing successful projects with PRINCE2, 2017 edn. Norwich: The Stationery Office Ltd; 2017.
