Is the laboratory report dead? AI and ChatGPT

Introduction

On 30 November 2022, Open AI launched ChatGPT (Generative Pre-trained Transformer), an artificial-intelligence- (AI) powered chat bot capable of generating human-like responses to text-based queries. This output is generated through statistical models built upon a vast corpus of online text. The language patterns embedded within the training data is what allows ChatGPT to use conversational language to respond to user queries across different genres of communication. ChatGPT generated responses to open-ended exam questions in physics that could not be distinguished from human submissions,¹ and scored a passing grade with performance comparable to a third-year medical student in the standardised United States Medical Licensing Examination.²

Laboratory skills represent a significant component of the microbiology curriculum,³ and hands-on laboratory training cannot be completed by text-based AI chatbots. However, theoretical knowledge, problem-solving and communication skills underpin threshold learning outcomes for microbiology graduates,⁴ and these remain vulnerable to AI disruption. On-campus examinations that do not allow students access to an internet-connected device may resolve part of the problem, but invigilation is not readily scalable towards other forms of university assessment. Scientific reports that summarise, interpret and analyse experimental findings generated by students after a laboratory module represent the culmination of microbiology threshold learning outcomes. The viability of this assessment task in the face of ChatGPT will be the focus of this article.

ChatGPT and laboratory assessment

A hypothetical laboratory assignment on antibiotic resistance was designed to evaluate the extent to which ChatGPT can be used by students to complete laboratory reports on their behalf. The Antibiotic Resistance Report (ARR) covers an overarching concept within ASM’s introductory microbiology curriculum,³ and involves student completion of a laboratory module that tests the antibiotic sensitivity of specific strains of microbes against a range of antibiotics. The data collected from these experiments then form the basis of the ARR, where students need to interpret the sensitivity profiles and discuss the experimental findings compared to scientific literature. The report has five interconnected Learning Objectives (LOs) that span across different cognitive domains of learning complexity (Table 1), and when ordered from least to most complex using Bloom’s Taxonomy these are: Knowledge, Comprehension, Application, Analysis, Synthesis and Evaluation.⁵

Apart from LO2, which necessitates hands-on application of laboratory skills, the remainder of the assignment could theoretically be completed by generative AI. What teachers should interrogate more deeply are the series of prompts that students can pose to the most freely accessible version of AI at any point in time. To ask ChatGPT to complete the ARR on behalf of a student, five prompts were entered into the publicly accessible free version of ChatGPT (GPT-3.5) in June 2023. Individual instructors are encouraged to adapt these prompts towards assessment items in their own teaching contexts to observe how AI is improving over time.⁶

Prompt 1. Can you write a scientific report on testing antibiotic resistance across different bacteria?

This initial prompt is the ‘can you do my assignment for me’ umbrella query, which, given the lack of specificity, is unlikely to yield meaningful results. A snapshot of ChatGPT’s attempt at responding to this query is shown in Fig. 1. This response does not meet the passing threshold, but it does provide students with a sample report structure to build upon. The section that requires the least amount of human intervention appears to be the Materials and Methods section. Pending additional detail on bacterial strains, culture conditions and concentrations of antibiotics used, ChatGPT’s response can be readily edited to resemble a conventional methods section. The other sections require significant rewriting to align with scientific reporting standards.

Fig. 1. 

ChatGPT (GPT-3.5, accessed 1 June 2023) generated scientific report on antibiotic resistance. No parameters other than report topic was provided to ChatGPT.

Prompt 3. Can you format these data into an appropriate type of graph?

In a normal learning sequence, students would generate their own experimental data for the ARR, but for this prompt a simulated zone of inhibition dataset was entered into ChatGPT (Table 2). ChatGPT misinterpreted the numerical values as percentage antibiotic resistance rates rather than zones of inhibition and suggests a clustered bar graph to visualise the data. It even attempted to draw this graph using dashes (Fig. 2). This misinterpretation can be corrected through additional prompts, but the text-based user interface is the main bottleneck that limits the use of AI to generate graphs and figures. This limitation may be short-lived, as Microsoft’s Copilot promises to integrate AI into the suite of Office 365 programs including Word, Powerpoint and Excel. The paid subscription version of ChatGPT Plus (GPT-4) also allows users to upload images as prompts; however, this functionality does not extend to any of the free AI chatbots (GPT-3.5, Bard, Bing) that all students have access to online.

Table 2. Simulated zone of inhibition (mm) antibiotic resistance data for Staphylococcus aureus (S. aureus), Escherichia coli (E. coli) and Klebsiella pneumoniae (K. pneumoniae) against four antibiotics.

Microbe	Replicate	Benzylpenicilin	Tetracycline	Erythromycin	Chloramphenicol
S. aureus	Replicate 1	38	34	34	32
	Replicate 2	14	27	15	27
	Replicate 3	20	20	18	26
E. coli	Replicate 1	8	30	19	32
	Replicate 2	10	30	14	32
	Replicate 3	10	29	15	31
K. pneumoniae	Replicate 1	0	32	13	31
	Replicate 2	0	32	14	31
	Replicate 3	0	28	15	33

Three replicates were simulated for each experimental condition.

Fig. 2. 

ChatGPT (GPT-3.5, accessed 1 June 2023) generated graph when prompted with simulated antibiotic resistance data from Table 2.

Conclusion

Scientific projects are open-ended and require significant human intervention to specify laboratory conditions and the relevant scientific context. Although the free and most accessible version of ChatGPT (GPT-3.5) can provide structure to a scientific report, formulating a hypothesis, visualising data and comparisons to peer-reviewed literature are all higher-order cognitive domains in Bloom’s taxonomy that are not readily approximated by AI. Students may be able to use ChatGPT to generate their first draft but will need a series of sophisticated and iterative prompts to refine the AI-generated text. Moreover, students should be strongly cautioned against entering primary research data into ChatGPT as this may transfer intellectual property to the AI platform. In research, journal editorial boards now need to contend with large volumes of AI-generated text.⁸ Although these AI-generated responses can be prone to hallucinations,⁷ their frequency has markedly decreased in Open AI’s next iteration, GPT-4.⁹

In addition to new software and AI-detection mechanisms,¹⁰ educators may also find value in pedagogical design. The five prompts outlined in this article can serve as a tool to establish the baseline AI response to a scientific report. Although students may utilise an unlimited number of prompts to refine the AI output, the weighting of assessment criterion should be considered relative to the quality of AI generated output to act as a deterrent for academic misconduct. For instance, ChatGPT’s attempts at the Introduction and Materials and Methods sections are closer to conventional report formatting than the Results and Discussion sections. Its attempts at data visualisation and identifying relevant scientific literature remain flawed for the time being, so assessment weighting across these sections should be adjusted accordingly. It is incumbent upon teachers to maintain currency with these technological innovations to ensure the ongoing evolution of science education.

Data availability

The data that support this study are available in the article.

Conflicts of interest

The author declares that he have no conflicts of interest.

Declaration of funding

This research did not receive any specific funding.