Register      Login
Australian Health Review Australian Health Review Society
Journal of the Australian Healthcare & Hospitals Association
Shopping Cart: ( empty )
You are here: Home > Journals > AH > AH24326
RESEARCH ARTICLE
Contents Vol 49(3)

Use of artificial intelligence to generate emergency department discharge summaries

Chuting Tang https://orcid.org/0009-0000-2688-798X A B C , Nilupul Mudunna A B D , Ian Turner A , Mohammad Asghari-Jafarabadi E F G , Keith Joe A and Lisa Brichko A H *
+ Author Affiliations
- Author Affiliations

A The Alan, Ada and Eva Selwyn Emergency Department, Cabrini Hospital, 183 Wattletree Road, Malvern, Melbourne, Vic 3144, Australia.

B School of Medicine, Monash University, Melbourne, Vic 3800, Australia.

C Department of Obstertrics and Gynaecology, Monash Medical Centre, Melbourne, Vic 3168, Australia.

D Department of Haematology and Bone Marrow Transplant, Royal Melbourne Hospital, Melbourne, Vic 3050, Australia.

E Cabrini Research, Cabrini Health, Malvern, Melbourne, Vic 3144, Australia.

F School of Public Health and Preventive Medicine, Monash University, Melbourne, Vic 3004, Australia.

G Department of Psychiatry, School of Clinical Sciences, Monash University, Clayton, Melbourne, Vic 3168, Australia.

H The Alfred Emergency and Trauma Centre, Melbourne, Vic 3004, Australia.

* Correspondence to: LisaBrichko@cabrini.com.au

Australian Health Review 49, AH24326 https://doi.org/10.1071/AH24326
Submitted: 29 November 2024  Accepted: 18 March 2025  Published: 28 April 2025

© 2025 The Author(s) (or their employer(s)). Published by CSIRO Publishing on behalf of AHHA.

Abstract

Objective

This study aims to evaluate the effectiveness of utilising an artificial intelligence (AI) model to generate emergency department (ED) discharge summaries in an easily accessible format.

Methods

This single-centre, proof-of-concept trial was conducted at a tertiary metropolitan private hospital. It involved 142 randomly selected patients who attended in 2023 and were able to be discharged home after care by a single ED doctor. A total of 284 documents were randomised, consisting of 142 de-identified ED medical notes and 142 AI-generated discharge summaries created by ChatGPT4 based on the corresponding ED medical notes. Both document types were distributed to six senior ED doctors, each of whom graded them individually and independently using a predetermined tool that assessed 17 items in four domains (expected contents, readability, medical accuracy, and internal consistency). The primary outcome was the graded score for the AI-generated discharge summaries, compared with that of the original ED medical notes.

Results

Across the 17 items and four domains assessed, AI-generated discharge summaries rated comparably to ED medical notes in 12 items (including key information, reason for the ED visit, past medical history, allergies and medications, social history, history of presenting complaint, investigations, differential diagnoses list, grammar, formatting, appropriateness, and consistency) and three domains (expected contents, readability, and internal consistency). AI-generated discharge summaries demonstrated high mean scores in the remaining five items (examination findings, primary diagnosis, detailed plan, language clarity, and reflectiveness of treatment) and one domain (medical accuracy).

Conclusions

AI-generated discharge summaries are potentially comparable to ED medical notes in most key performance domains of a discharge summary.

Keywords: AI, artificial intelligence, ChatGPT, discharge summary, emergency department, health informatics, large language model, LLM, medical communication, medical documentation.

References

1  Shi J, Bendig D, Vollmar HC, Rasche P. Mapping the Bibliometrics Landscape of AI in Medicine: Methodological Study. J Med Internet Res 2023; 25: e45815.
| Crossref | Google Scholar | PubMed |

2  Boonstra A, Laven M. Influence of artificial intelligence on the work design of emergency department clinicians: a systematic literature review. BMC Health Serv Res 2022; 22(1): 669.
| Crossref | Google Scholar | PubMed |

3  Coghlan A, Turner S, Coverdale S. Danger in discharge summaries: abbreviations create confusion for both author and recipient. Intern Med J 2023; 53(4): 550-8.
| Crossref | Google Scholar | PubMed |

4  Morley C, Unwin M, Peterson GM, Stankovich J, Kinsman L. Emergency department crowding: a systematic review of causes, consequences and solutions. PLoS One 2018; 13(8): e0203316.
| Crossref | Google Scholar | PubMed |

5  Weetman K, Spencer R, Dale J, Scott E, Schnurr S. What makes a “successful” or “unsuccessful” discharge letter? Hospital clinician and general practitioner assessments of the quality of discharge letters. BMC Health Serv Res 2021; 21(1): 349.
| Crossref | Google Scholar | PubMed |

6  Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med 2023; 29(8): 1930-40.
| Crossref | Google Scholar | PubMed |

7  Gödde D, Nöhl S, Wolf C, Rupert Y, Rimkus L, Ehlers J, et al. A SWOT (Strengths, Weaknesses, Opportunities, and Threats) Analysis of ChatGPT in the Medical Literature: Concise Review. J Med Internet Res 2023; 25: e49368.
| Crossref | Google Scholar | PubMed |

8  Liu J, Wang C, Liu S. Utility of ChatGPT in Clinical Practice. J Med Internet Res 2023; 25: e48568.
| Crossref | Google Scholar | PubMed |

9  Ali SR, Dobbs TD, Hutchings HA, Whitaker IS. Using ChatGPT to write patient clinic letters. Lancet Digit Health 2023; 5(4): e179-81.
| Crossref | Google Scholar | PubMed |

10  Jeblick K, Schachtner B, Dexl J, Mittermeier A, Stüber AT, Topalis J, et al. ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports. Eur Radiol 2024; 34: 2817-25.
| Crossref | Google Scholar | PubMed |

11  Williams CYK, Bains J, Tang T, Patel K, Lucas AN, Chen F, et al. Evaluating Large Language Models for Drafting Emergency Department Discharge Summaries. medRxiv 2024; 2024.04.03.24305088.
| Google Scholar |

12  Patel SB, Lam K. ChatGPT: the future of discharge summaries? Lancet Digit Health 2023; 5(3): e107-8.
| Crossref | Google Scholar | PubMed |

13  Baker A, Perov Y, Middleton K, Baxter J, Mullarkey D, Sangar D, et al. A Comparison of Artificial Intelligence and Human Doctors for the Purpose of Triage and Diagnosis. Front Artif Intell 2020; 3: 543405.
| Crossref | Google Scholar | PubMed |

14  Gusmeroli M, Perks S, Bates N. Medication reconciliation and discharge communication from hospital to general practice: a quantitative analysis. Aust J Prim Health 2023; 29(6): 679-85.
| Crossref | Google Scholar | PubMed |