Evaluating environment cleanliness using two approaches: a multi-centred Australian study

• Visual cleanliness assessment may overestimate the level of environmental cleanliness.

• A structured approach, supported by resources, is required to evaluate environmental cleanliness.

Evidence demonstrates that the environment plays an important part in the transmission of healthcare-associated infection,^1–4 thus environmental hygiene plays a critical role in an infection prevention and control program.^5–8 An environmental cleaning program and subsequent assessment (or surveillance) of cleanliness is an integral part of an infection control program, with the goal of ensuring a healthcare environment that is both aesthetically acceptable and has a reduced bioburden.^5,9,10

The Tasmanian Infection Prevention and Control Unit (TIPCU) published a report in 2012 on the methodologies used locally, nationally and internationally for assessing environmental cleanliness within healthcare.¹¹ There are two major methods of assessing the cleanliness of the healthcare environment: (1) process evaluation which evaluates the cleaning process itself and includes visual inspection and fluorescent gel and light assessment; and (2) outcome evaluation which evaluates microbial burden post-cleaning and involves the use of adenosine triphosphate (ATP) or microbial cultures. In Australia, visual assessment is commonly recommended and used as the primary method to assess environmental cleanliness. Visual assessment is also used overseas, for example in the United Kingdom.¹¹

The findings of this report were presented to a meeting of Tasmanian stakeholders where there was a decision made that TIPCU would devise and assist in the implementation of an environmental assessment program for use within Tasmanian healthcare facilities. The consensus was that the program would be used across Tasmania in a variety of healthcare settings and would be performed in a standardised manner, by trained assessors using the two process measures identified – visual assessment and fluorescent gel assessments. These methods were chosen due to cost and ease of use. A standardised approach was important to ensure consistency of assessor training, allow benchmarking of results between facilities, ensure consistency of the actual assessment of environmental hygiene, and assist in meeting national accreditation standards, specifically Standard 3 (Preventing and Controlling Healthcare-Associated Infection) of the National Safety and Quality Health Service Standards.¹²

This paper describes the development process and the findings of the first 12 months of data following the introduction of a standardised program for evaluating environmental cleanliness within Tasmanian healthcare facilities using two different evaluation methods.

The standardised method to evaluate environmental cleanliness included the development of a highly structured protocol outlining the methodology for two types of assessment. The protocol included which environmental sites should be assessed, when to assess, who can assess and data entry requirements. The program also included a standardised online education training program for assessors, an online data entry tool, online resources and a ‘Frequently asked questions’ brochure.¹¹

The four larger Tasmanian public hospitals were invited to participate in a 4-week pilot study to assess the usability and acceptability of the program. Both quantitative and qualitative feedback were sought with the majority of feedback received being positive about the protocol, education and processes. Amendments based on the pilot feedback were made to both the protocol and data collection tool.

All Tasmanian public and private hospitals were invited to participate in the program. Invitations were disseminated via email and in person when the opportunity arose. Although voluntary, participation required a hospital executive board member, the manager of environmental services and the manager of the infection prevention and control unit to all formally agree to the hospital’s participation in the program. Twelve Tasmanian hospitals participated in the program. The hospitals ranged from rural hospitals to large public and privately funded hospitals.

The evaluation of environmental cleanliness involved two elements: the use of an ultraviolet (UV) solution with fluorescent light assessments conducted quarterly in patient care areas that had undergone discharge cleans, and a visual assessment conducted at least quarterly in both patient care and general ward areas. Cleaning in participating hospitals was undertaken by cleaning staff employed by the hospital.

The UV solution and light method (Ecolab^® DAZO^®) involved the application of the UV solution to up to eight high-touch surfaces in patient care areas by an auditor. The solution was allowed to dry before cleaning was undertaken. As the gel is easily removed with light abrasion, an evaluation was conducted post-discharge cleaning using a UV light to determine whether the surface had been cleaned correctly. Numerous studies have shown that this procedure improves cleaning practices.^13–17 The UV solution was only applied to rooms or patient care areas that were undergoing discharge cleans. The rationale for this was twofold. First, evidence suggests that prior room occupancy is a risk factor for acquisition of infectious agents.^18–20 Second, the assessment is easier to implement in rooms where patients are no longer present. The objects in patient care areas to which the UV gel can be applied vary in the literature. In our study, the gel was applied to one of the sites detailed in Table 1 in each patient care area. These sites were determined following a review of the literature, the suitability of previously documented sites in the Australian hospital context and consultation with infection control professionals in Tasmania.^{13–15,17,21} Once a clean had been completed, the auditor returned to determine which sites had been cleaned. If any level of fluorescence was present, then it was determined that the object had not been cleaned.

Table 1.  Locations of the fluorescent gel applications

Visual assessments were conducted to determine cleanliness in all areas of the hospital. These assessments were developed following a review of approaches taken in two Australian states and current practices in Tasmania.^11,22,23 To allow flexibility and clarity regarding specific items, two different visual assessment tools were used in this study. The two visual areas were defined by the location in which the assessments were undertaken – the patient care areas and the general ward areas. These two areas were clearly detailed for the auditors and were specifically chosen to ensure consistency with the approaches taken in New South Wales,²² Victoria²³and existing practices in Tasmania.^21,24 Additionally, some sites were more specific to allow comparisons between the visual assessments and fluorescent gel method. The assessed areas were deemed ‘clean’ or ‘not clean’ based on the descriptors provided to the auditors.

To improve inter-rater reliability, only auditors who had received training and successfully completed an online exam were able to undertake assessments. The exam was developed by two experienced, credentialed Clinical Nurse Consultants. Questions in the exam related to key points of the program, including determining whether a site was clean and when to conduct assessments. To assist, the TIPCU developed training tools, including a PowerPoint presentation, videos, a quiz and a protocol.¹¹ Training was also provided by the TIPCU to hospital hotel service managers and supervisors, as well as infection control staff. The TIPCU maintained the list of auditors and was thus able to readily communicate any changes in the processes. Figure 1 summarises the data collection methods and project.

Fig. 1.  Summary of data collection and project overview.

Using an iPad or smartphone, the results of each auditor assessment were entered directly into an online database specifically designed for this process (Fig. 2). The assessments were undertaken quarterly, and the required number of assessed patient care areas was dependent on the number of beds in the hospital, equating to ~10% of hospital beds.

Fig. 2.  Examples of online data entry methods. Note: The image on the left is the data entry portal for computers. The image on the right is the data entry portal for smart phones.

Patient care area: the space temporarily dedicated to an individual patient for that patient’s stay. In this study, these areas comprised inpatient bed areas, including isolation rooms, patient bays, paediatric cots and neonatal incubators and/or cots, emergency departments (where assessment or treatment is undertaken), theatres and outpatient clinics.

General ward area: an area where the assessment or treatment of patients does not occur directly. In this study, these areas comprised ward corridors, nurses’ stations, sterile stockrooms, equipment rooms and toilets, showers and bathrooms that were located off ward corridors.

The data were imported into IBM SPSS version 21.0 (IBM) for analysis. The aggregated data from the participating sites were analysed – the individual hospital results are not presented, as agreed by the participating organisations. A comparison of the variables was undertaken using independent t- tests, and the nonparametric independent data were compared using the Mann–Whitney U test. The mean percentage and 95% confidence intervals (CIs) of items deemed to be cleaned were calculated using Poisson distribution. Analysis of variance was performed to compare the differences between the variable mean scores.

Approval for this study was granted by the Tasmanian Human Research Ethics Committee.

Two Tasmanian health organisations responsible for public hospitals in two geographic regions in Tasmania and one private hospital participated in this study, equating to 12 individual hospitals. There are 25 hospitals in Tasmania. The number of overnight beds in the participating hospitals ranged from 20 to 280 beds.

A total of 290 fluorescent light assessments and 232 visual inspections were undertaken in the first 12 months of this study. Using the fluorescent light method, 1668 individual objects were assessed. The percentage of correctly cleaned items increased from 82.3% (95% CI, 78.7–85.5%) to 85.4% (95% CI, 82.4–88.0%) over the study period, with an overall average of 82.8% (95% CI, 78.8–86.2%). Figure 3 illustrates the trend data for the proportion of items deemed correctly cleaned over the 12-month period. Using the visual assessment method, 92.5% of items were determined to be acceptable during the course of the study with no overall trend during the study period. Table 2 lists the individual items that were assessed using both methods.

Fig. 3.  Proportion of items cleaned correctly, assessed for fluorescent gel. Note: Errors bars indicated 95% confidence intervals.

Table 2.  Items evaluated using the fluorescent light and visual inspection assessments

For the eight most frequently touched objects, the fluorescent light assessments indicated 82.8% (95% CI, 78.9–86.9%) were cleaned to an acceptable level compared with 95.9% (95% CI, 89.3–95.8%) for the visual inspection audits (P < 0.01).

Of the 290 fluorescent light assessments, 62 (21.3%) were done in rooms where the occupant was under transmission-based precautions. There was no significant difference in the proportion of individual items cleaned correctly when comparing rooms where the occupants were under transmission-based precautions with those where the occupants were not under transmission-based precautions.

Our study differed from other published works in this area as it used and compared a combination of UV gel assessments and visual assessments to assess environmental hygiene. Furthermore, we found a higher baseline level of cleanliness using the fluorescent light method than previously documented in the literature. We also assessed several high-touch sites using both visual inspections and fluorescent light assessments. The project was supported by training tools, the formal assessment of auditors and a range of online tools for data submission.

In an Australian study undertaken by Murphy et al. the authors evaluated fluorescent markings, education and feedback to assess and improve cleaning in an Australian inpatient hospital setting.¹⁷ The baseline data in this study were 34%, increasing to 53.5% before declining to 41%.¹⁷ The baseline data in our study were considerably higher than the data of this and other studies.^13,15 Possible explanations for this include some minor differences in the physical locations of where the fluorescent gel was applied, the educational awareness campaign that commenced at the start of our study and our focus on discharge cleans. The locations of the gel applications were decided after reviewing the literature and their applicability to the Australian healthcare context, including public and private hospitals.^13–15,21 In addition to site inspections, a survey of infection control professionals and cleaning staff in Tasmania was conducted to ensure these locations were sufficiently similar between different institutions.

The education awareness campaign we undertook was in part to ‘sell’ the study to environmental health services staff and managers. Both the importance of cleaning and the findings of existing research were explained in various settings, including at staff meetings, during staff training and informally with staff in causal settings. This may have had the desired effect of increasing awareness about cleaning before the study commenced, with follow-on positive effects on cleaning performance. It is also worth noting the high proportion of objects cleaned correctly, as assessed by the fluorescent light method, remained high and improved further over the 12-month study period.

Unlike other studies, we focused on discharge cleans only. The rationale for this decision included the availability of resources at the time, namely the fluorescent gel and lights, the availability of staff with competing interests and evidence suggesting the important role that prior room occupancy may have on future infection acquisition.^25–28 From a risk assessment perspective, we deemed discharge cleans an area of priority. We did not covertly assess cleaning. The decision on which rooms would be assessed was determined by the environmental cleaning supervisors, often in conjunction with infection control professionals. Cleaners were not informed that the rooms would be assessed, but as the assessments were not undertaken in a covert manner, it is possible that cleaners were aware that assessments were taking place. This is a potential limitation from a methodological perspective; however, the end result was to improve cleaning standards and, if this limitation contributed to that, it could be argued that it was a successful approach.

In this study, we were able to compare the cleanliness of objects with a fluorescent light against what would have been deemed visually acceptable. Objects were frequently considered to be visually acceptable yet may not have been cleaned. Although obvious, this provides evidence that because something is visually clean, it does not necessarily mean it was actually cleaned. A formal study evaluating correlations between visual inspections, the fluorescent light method and other measures of environmental cleanliness, such as ATP detection, is required.^29,30

Our study had several strengths. We employed methods to improve inter-rater reliability through a formal auditor assessment process. Our study was also conducted over a 12-month period at 12 different hospitals. It was supported by bedside online data entry methods using iPads and smartphones, and real-time reporting, which enabled immediate feedback to staff and the option for hospitals to access their own data. The participating hospitals were each given an iPad to assist with data entry. A dedicated webpage was developed to provide access to resources, including a manual and videos. Although difficult to quantify, we observed a tangible sense of enthusiasm from environmental health services staff. They were interested in this project and improving patient safety and we believe it could lead to greater collaboration between infection prevention and control and environmental services. In some hospitals, this project appeared to improve collegiality and communication between infection control professionals and environmental health services. The use of a collaborative or ‘bundled’ approach to environmental cleanliness is one area that could be explored in future research.

Our multi-centre study identified a higher baseline level of cleanliness using the fluorescent light method than previously documented in the literature. We also assessed several high-touch sites using both visual inspection and fluorescent light assessments. Objects were frequently deemed to be visually acceptable yet may not have been cleaned. The auditors in our study were required to complete a formal assessment process and were supported by a range of resources.

One of the authors of this paper has an editorial affiliation with the journal. The author played no role in the peer review or decision rendering process.

No funding was received in relation to this study.