lable at ScienceDirect

American Journal of Infection Control 40 (2012) e233-9

Contents lists avai

American Journal of Infection Control

journal homepage: www.aj ic journal .org

American Journal of Infection Control

Major article

Validation and comparison of three adenosine triphosphate luminometersfor monitoring hospital surface sanitization: A Rosetta Stone for adenosinetriphosphate testing

Carmen V. Sciortino PhD a,b,*, R. Allen Giles BS b

aRobley Rex Veterans Affairs Medical Center, Pathology and Laboratory Medicine Service, Louisville, KYbUniversity of Louisville, Louisville, KY

Key Words:LinearityPrecisionLimit of detection

* Address correspondence to Carmen V. Sciortino, PAssociate Director of Molecular Pathology, Robley RMedical Center, 800 Zorn Avenue, Louisville, KY 4020

E-mail address: Carmen.Sciortino@va.gov (C.V. SciThis work was performed at the Robley Rex Depar

Administration, Louisville, KY, with the support of Infdifficileproject, EnvironmentalManagement Services, t

CharmSciences, Inc, and 3MCorporationprovidedat no charge in support of this study. Charm Sciencmanufacturers of the Hygiena System Sure II instrumfinancially from the results of this study.

Conflicts of interest: C.V.S. is an employee of theMedical Center andperformed these studies as a part ofno financial gain from his employer or from any outformed. R.A.G. is an official volunteer at the RobleyCenter. He received no financial compensation for his w

0196-6553/$36.00 - Copyright � 2012 by the Associadoi:10.1016/j.ajic.2012.04.318

Background: The application of adenosine triphosphate (ATP) luminometry for monitoring sanitizedhospital surfaces is an evolving infection control practice. The goal of this study is to validate andcompare 3 commercially available ATP luminometry, swab, rapid-test systems for use in hospital surfacesanitization testing. Regulatory agencies may consider this a point-of-care laboratory test; it is thereforeimperative that validation criteria are established.Methods: The ability of instruments to measure long-term stability of ATP dried on surfaces wasdetermined. Dilutions of 3 species of microorganisms and a blood sample were dried onto a surface andtested. The performance characteristics of instruments were compared side-by-side for their ability torecover microorganism-derived ATP from surfaces.Results: Timed studies showed that surface biologic-ATP remained detectable for 10 days. Instrumentclinical sensitivity, precision, detection range, limit of detection, and linearity were determined. Swabrecovery of microorganisms from surfaces varied by instrument and organism. All 3 systems detectedmicroorganisms in the presence of disinfectants.Conclusion: All instruments were validated, but only 1 was verified, because of variations in performancefor each system. These studies indicate that careful consideration of the technologic application andinstrument performance are important criteria for the selection of an ATP monitoring system.

Copyright � 2012 by the Association for Professionals in Infection Control and Epidemiology, Inc.Published by Elsevier Inc. All rights reserved.

Appropriate disinfection of environmentally contaminatedhospital surfaces can eliminate or reduce communicable diseasetransmission. Hospital surfaces contaminated with methicillin-resistant Staphylococcus aureus, vancomycin-resistant enterococci,

hD, Chief of Microbiology andex Veterans Administration6.ortino).tment of Veterans Healthcareection Control, the VA VISN CheEmployee’sWorkersUnion.loaner instruments and swabses, Inc, 3M Corporation, andent may either gain or lose

Robley Rex Veterans Affairshis official duties. He receivedside source for the work per-Rex Veterans Affairs Medicalork.

tion for Professionals in Infection C

Clostridium difficile, Candida sp, multidrug-resistant Acinetobacterbaumannii, and Pseudomonas aeruginosa are some of the mostthreatening microorganisms to hospitalized patients and may poseour greatest infection control challenge. It has been difficult toassess standard housekeeping practices and develop protocols fordecontamination because of the lack of a standardized assessmenttool.1 Bacteriologic cultures using standard techniques are an inef-ficient means to assess cleanliness because of limited recovery ofmicroorganisms.2 Adenosine triphosphate (ATP) bioluminescenceassays offer a novel approach for the detection and quantification ofbioload occurring on hospital high-touch surfaces. Quantitativeresults are available in less than 5 minutes with these assays. Thismakes it possible for infection preventionists or housekeeping staffto monitor the adequacy of cleaned surfaces. Several publishedstudies address the effective use of ATP methodology for hospitalsurfacemonitoring.1,3-10 AlthoughATP technology is nonspecific, thecaveat that a reduction of ATP relative light units (RLU) correlateswith a reduction of infectious microorganismsmay be an acceptableoutcome.7,8 Some investigators have discussed benchmarking and

ontrol and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved.

C.V. Sciortino, R.A. Giles / American Journal of Infection Control 40 (2012) e233-9e234

the establishment of ranges for acceptability.3,9,10 We realize thatATP luminometer systems vary in capacity to measure bioload andthat ATP benchmarking may require some instrument-specificstandardization. To make valid decisions regarding the results ofATP luminometry testing, instrument verification and validation arenecessary. We found only 1 validation study that compared ATPresults instrument to instrument.11

The specific aim of this study was to validate, compare, andaddress specifics of instrument functionality for 3 commerciallyavailable ATP luminometers. It was not our intention to addressinfection prevention and control issues, monitoring methodology,microbiologic differences, or benchmarking issues. Although weused pathogenic microorganisms, we do not suggest or imply thatnormal biofilms found on hospital surfaces contain any or all ofthese microorganisms. The microorganisms chosen are a relativebut diverse, quantifiable source of biologic ATP used in thesestudies to compare instruments.

METHODS

Instruments and materials

The 3 ATP-luminometry systems used for this studywere system1 (SY1): NovaLUM (Charm Sciences, Inc, Lawrence, MA) with thePocketSwab plus swab; system 2 (SY2): Clean-Trace (3M HealthCare, Hammfelddamm, Germany) with the UXC swab; and system 3(SY3): Hygiena SystemSure II (Hygiena, Camarillo, CA) with Ultra-snap swab. The material surface used for swabbing was stainlesssteel cut into plenums (15.3 cm � 15.3 cm � 1 mm). Used,contaminated plenums were soaked overnight in 1:200 Lysol(National Labs,Montrale, NJ) before recycling. Recycling consisted ofscrubbing plenums with Comet cleanser (Procter and Gamble, Cin-cinnati, OH) and Scotch-Brite (3M, St Paul, MN) abrasive pads.Abrasive scrubbing was necessary for inoculums to spread evenlyover the stainless steel surface, or the inoculumswould bead up andnot dry evenly. Cleaned plenumswere rinsed 6 timeswith tapwaterand 2 times with type 1 water. Autoclaving plenums created a non-wetting surface that caused inoculums to bead. However, wedetermined that autoclaving was not necessary because negativecontrols gave zero RLU values on cleaned, nonsterile test surfaces.Air-dried plenums were ruled into 12 smaller surface areasmeasuring 2 cm � 5.6 cm (area ¼ 11.2 cm2); wrapped in Kimwipes(Kimberly-Clark, Atlanta, GA) and used within 1 day. All pipettesused for these experiments were precision pipettes, standard error<5%, with sterile, nonwetting tips.

Inoculum preparation

The microorganisms Staphylococcus aureus (ATCC 29213), Aci-netobacter baumannii (BAA747), and Candida albicans (ATCC 14053)were grown for 24 hours on tryptic soy blood agar at 36�C.Microorganisms (all 3 species) were individually suspended in 2mLof 0.45% sterile saline, each at approximately 108 organisms permilliliter as determined by McFarland optical density with anelectronic Vitek (bioMérieux, Inc, Durham, NC) colorimeter. Allinoculums were verified by quantitative culture with plate-countcolony formation on tryptic soy blood agar. For quantitativeculture, inoculum dilutions consisted of 7 serial 1:10 dilutions insterile saline. In most experiments, aliquots (0.05 mL) of bacterialdilutions were spread onto stainless surface plenums using sterilepipette tips. Negative controls consisted of sterile saline only.Inoculated plenums were dried on heat blocks (40�C for 10minutes). Instrument-specific ATP swabs were used to swab the fullruled area of plenums (11.2 cm2). Test swabs were immediatelyinserted into the reaction swab units, and luminometer readings

were performed as indicated by themanufacturers’ instructions. Allexperiments were performed in duplicate unless otherwise statedin the Methods section.

Swab sampling

Swabbingwasperformedonan11.2-cm2areausingaside-to-side,up-and-down motion to encompass the entire ruled area, whilerotating the swab during the swabbing motion. This was in accor-dance with the manufacturers’ instructions. The SY1 sponge swabswere oval shaped with a flat surface, and wet (saturated), allowingmostof the swab to contact the surface. The SY1 swabwas introducedinto a reactant tube containing reagents that completely cover theswab, allowing all organisms on the swab to react with reagents. TheSY2andSY3swabswerepear shaped, slightlymoist,withonlya smallsurface area coming in direct contact with the sampling surface. TheSY2 swab is similar to the SY1, allowing reagents to react with allorganisms on the swab within a reactant tube. The SY3 swab hasa bulb containing reagents located opposite the swab end.When thebulb is squeezed, reagents are perfused through the hollow swabshaft, through the swab into the lower end of the tube system. Someswabs contain sodium azide, and removal of residues was necessarypostswabbing.

Environmental microbiologic culture

Environmental surfaces were cultured for total colony counts toinclude aerobic bacteria/yeast using surface touch plates (RODAC;Becton, Dickinson and Company, Sparks, MD). RODAC platescontain lecithin and P80 that detoxify disinfectants. Plates wereincubated at 36�C for 24 hours followed by incubation at 22�C foran additional 48 hours.

Reproducibility over time

We evaluated each ATP system to determine whether eachcould detect biologic samples dried on steel plenums for up to 10days. Inoculums (50 mL) contained either 3.25 � 106 A baumannii orS aureus, 1.5 � 107 C albicans, or sterile diluted blood. Human bloodwas diluted 1:3 in BacT Alert blood culture medium and incubatedat 37�C for 5 days on the BacT Alert instrument (BacT alert; bio-Mérieux, Inc, Durham, NC). The bacterial growth-negative bloodfrom the blood culture bottle was inoculated in 50-mL aliquots ontosteel plenum surfaces. All inoculums were dried, wrapped in cleanaluminum foil, and stored at 22�C for timed sampling. ATP swab-bing and testing were performed at 0.25, 6, 72, 120, and 240 hours.The experiment was performed in triplicate.

Swabbing efficiency

The efficiency of swabs to remove microbial ATP from steelplenums was determined for each instrument by dividing the RLUobtained from swabbed, inoculated plenum surfaces by the RLUobtained from pipetting organisms directly onto ATP swabs. Toestablish a numerical equality of organism inoculums, 0.005 mL of5 � 108 colony-forming units (cfu)/mL for each organism werepipetted directly onto system swabs, compared with 0.050 mL of5 � 107 cfu/mL of each organism spread over the surface of steelplenums, dried as described above, and swabbed. Ten replicatesamples were tested for each organism and ATP instrument system.

Reportable range (linearity)

The reportable range is the highest and lowest detectable analytewithin the linear range for which valid measurements can be

Table 1Swabbing efficiency of various organisms recovered from steel plenums

Recovery of organisms from swabbedsurfaces

% Swabbing efficiency �% standard deviationInstrument Organism

1 A baumannii 7.3 � 1.5S aureus 55.3 � 12.3C albicans Indeterminate

2 A baumannii 6 � 1.8S aureus 13.3 � 4.4C albicans 34 � 15

3 A baumannii 3.8 � 1.6S aureus 7.2 � 7.2C albicans 75.3 � 12

NOTE. Efficiency calculated by dividing ATP RLU from the swabbed plenum by RLUfrom the direct addition of inoculums to an ATP swab system (n ¼ 10).

C.V. Sciortino, R.A. Giles / American Journal of Infection Control 40 (2012) e233-9 e235

obtained.12 The limit of detection (LOD) refers to the lowest value ofanalyte that the instrument can detect and distinguish from back-ground or negative control.12,13 Linearity affirms the ability of thesystem to return values that are directly proportional to theconcentration of the analyte.12,14 The reportable range, LOD, andinstrument linearity were determined as follows: Testing included 6concentrations of 3 organisms (S aureus, A baumannii, C albicans) anda negative saline control. Inoculums were standardized and inocu-lated onto plenums (50 mL), dried, and tested as described above.Duplicate testing was performed, and all experiments repeateda second time.

Clinical sensitivity and precision

Gage repeatability and reproducibility (R&R) analysiswasused todetermine clinical sensitivity and precision. The clinical sensitivityof an instrument is determined by testing samples at the high andlow end of linearity, and below the LOD, while determining therepeatability and reproducibility.12 The precision of an instrument isthe closeness of agreement between measurements that are biasedby variations in operators or day-to-day changes in reagents,lighting, or atmospheric conditions.12,13 The clinical sensitivity andprecision of the 3 instruments were determined as follows: Steelplenums were loaded with 3 concentration ranges (high ¼ 106-107

cfu, low ¼ 103-104 cfu, and below the LOD ¼ 101-102 cfu) of each:S aureus, A baumannii, C albicans, and negative saline control. Inoc-ulums were standardized and inoculated onto plenums (50 mL),dried, and tested as described above. Day-to-day variation wasdetermined by repeated testing for 10 days. Operator swabbingvariation was determined with 2 different operators.

Interfering substances

An interfering substance is one that systematically causes anerror in the measured results. Timed studies tested the effect ofdisinfectants/detergents on ATP RLU systematic errors. Chemicaltest substances were as follows: 10% chlorine (Clorox ProfessionalProducts Co, Oakland, CA), Lysol spray (Reckitt Benckiser Inc, Par-sippany, NJ), Terg-A-Zyme (Alconox Inc, New York, NY), and sterilebovine serum albumin (BSA) (Intergen Co, Purchase, NY), 5% inphosphate-buffered saline. Stainless steel plenums were wiped orsprayed with the test substance and allowed to air-dry at 22�C. ForTerg-A-Zyme, the rulings on plenums were outlined in vacuumgrease to avoid suspensions of test microorganisms from runningtogether. Inoculums were standardized and inoculated ontoplenums (50 mL of 108 cfu/mL) and tested as described above. Inoc-ulated plenumswere placed on heat blocks for drying. However, the1-minute sample was taken before samples dried, and, therefore, itwaswet. Sample swabbingwas performed at timed intervals of 1,10,and 30 minutes. A control/baseline series was tested in parallel byplacing inoculums of each organism on plenum surfaces withoutpretreatment of the plenumwith a test substance.

Accuracy

Accuracy determines howwell the assay performs in actual use,comparing a test value to a standardized measurement.12,14 Wecompared system ATP RLU with RODAC culture cfu. Commonhospital surfaces were tested in duplicate, both pre- and post-cleaning. Three hospital patient room surfaces (table top, bed rail,and toilet seat) were tested from 2 hospital isolation rooms, dis-infected with 1:10 chlorine bleach. Surfaces tested from themicrobiology laboratory, cleaned with Lysol were culture-benchtop, sink top, and the exit door handle. A clinician’s office desk,sink, and telephonewere cleanedwith Scrubbing Bubbles (Johnsons

and Son, Inc, Racine, WI) and tested. A public hospital restroomcleanedwithWex-cide 128 (Wexford Labs, Kirkwood,MO) includedthe exit door handle, sink top, and toilet seat. Prior to the study,authority to test surfaces was obtained through Medical Adminis-tration, Infection Control, Environmental Management Service, andthe EmployeeWorkers’Union. ATP swabbing consisted of testing anarea of 10 to 12 cm2 and was performed by us. RODAC plates werealso used to sample surface areas by applying the plate directly tothe surface for imprint cfu analysis.

Statistical analysis

Statistical software used in this study were QI Macros (Know-Ware International Inc dba Lifestar, Denver, CO) and MedCalcSoftware (Mariakerke, Belgium). Linearity and R2 were determinedby scatter plot analysis. Statistical significance of measurementswas determined by repeatedmeasures analysis of variance, paired ttest, comparison of means for range differences, and Gage R&Ranalysis for instrument operator variation. In all cases, there wereno outliers excluded from the data regardless of how extreme thevalue because the aim of the study was to evaluate actual raw dataprovided by each instrument and use that data in the validationprocess to compare technologies.

RESULTS

The long-term stability of source specific ATP on stainless steelwas determined over a period of 10 days. For SY1 and SY2, ATPstability was conserved, although some statistically significantdifferenceswere seen (SY1 forAbaumanniiat240hours,P�.001;SY2for C albicans at 240 hours, P� .002). For SY3, ATP stability at 6 hourswas significantly reduced for S aureus, A baumannii, and C albicans(P � .005) but not for blood. ATP RLU for SY3 were significantlyreduced at later times for A baumannii (P � .005) and for C albicans(120 hours, P � .004, 240 hours, P � .003). Overall, ATP RLU wererelatively stablewith theexceptionofmeasurements takenwithSY3.

The percentage of swabbing efficiency is shown in Table 1. Therewas great variability in the recovery of microorganisms usingdifferent ATP luminometery systems. With SY1, we were unable toobtain relevant values for C albicans. We repeated the experiment 5times, and, each time, the plenum swab RLU were approximately 3times higher than the RLU of directly inoculated swabs. Alterna-tively, because our procedure required heating plenums at 40�C fordrying samples, we heated the direct sample to 40�C for 10minutes, but results were unchanged. Wewere unable to overcomethis artifact, so we listed it as indeterminate.

Fig 1. Line graph for 1:10 serial dilutions of A baumannii dried onto steel plenums then swabbed and tested for ATP RLU with 3 separate ATP test systems. Instrument linearityplotted on logarithmic axes showing actual RLU values versus colony-forming units. Shown are the extrapolated exponential linearity, R2 value, and standard deviation error bars(n ¼ 2) for each instrument.

C.V. Sciortino, R.A. Giles / American Journal of Infection Control 40 (2012) e233-9e236

The instrument linearity, LOD, and linear range were deter-mined for each instrument for 3 test microorganisms (Figs 1-3). Theinstrument linear ranges and LOD are shown in Table 2.

As determined by Gage R&R analysis, the SY1 instrument hadthe least equipment variation (EV), 4.2%; operator variation (OV),3.2%; and R&R variance, 5.3%. Gage analysis rated the SY1 process“okay overall.” Instrument SY2 had the greatest EV (17.4%), OV(19.9%), and variance in R&R (26.4%). Gage analysis rated the SY2process as “gage okay, but operator variance needed improvement.”Instrument SY3 had a moderate EV (12.1%), low OV (6.8%), andmoderate R&R (13.9%). Gage analysis rated the SY3 process as “gageneeds modification or adjustment, and operator okay.” Gage anal-ysis also provides analysis of variance, which indicated thefollowing values: SY1, OV P � .061, R&R ¼ 6.4%; SY2, OV P � .009,R&R ¼ 35.1%; SY3, OV P � .109, R&R ¼ 26.9%.

Interference was determined by means t test comparison forRLUs from chemically treated surfaces versus RLUs from untreatedsurfaces, with statistical significance at P � .05. For comparison ofinstruments, the total percent interference was calculated bysumming the significant observations for all organisms within thetest substance group and dividing by the total observations withinthat group. The total percent interference of chemical substanceson ATP RLUs was as follows: Detergent > quaternary ammoniumcompound > chlorine > BSA. For all instruments, there was lessthan 1-logarithm difference in RLU at any timed period for organ-isms or chemical test substances, with 1 exception. Instrument SY3with detergent showed a 2-logarithm decrease in RLU for A bau-mannii and S aureus at all time periods, but this was likely due toinstrument low-scale sensitivity. BSA-associated increase or loss ofATP RLUs up to 67% for SY1 was significant but random andorganism dependent. Although chemical substances significantlyand randomly affected (0%-78%) ATP recovery, the effect was notbeyond biologic explanation. No single test substance systemati-cally altered ATP RLUs for any instrument, all organisms.

The clinical accuracy of ATP luminometer measurements fromvarious hospital surfaces was compared by measuring the percentreduction of ATP post- versus precleaned surfaces. Results werecompared instrument to instrument and by percent reduction ofRodac culture cfu. The data were evaluated by determining thePearson’s correlation coefficients. For all instruments, the correla-tion coefficient (r) comparison with culture was poor (SY1:r ¼ 0.104; SY2: r ¼ 0.218; SY3: r ¼ �0.036). Instrument-to-instrument comparison showed the closest correlation betweenSY1 and SY2 (r ¼ 0.733). The correlation coefficients of SY3compared poorly with SY1 (r ¼ 0.358) and very poorly with SY2(r ¼ 0.014). The mean swab data RLUs (n ¼ 28) and data range forpreclean versus postclean RLU values for each instrument were asfollows: SY1 (preclean: mean, 77,827 � 112,919; range, 5,322-695,409; postclean: mean, 16,513 � 16,366; range, 629-72,482);SY2 (preclean: mean, 1,368 � 4,797; range, 24-26,621; postclean:mean, 146 � 133; range, 14-562); SY3 (preclean: mean, 26 � 29;range, 1-141; postclean: mean, 10 � 10; range, 0-44). The t testmeans comparisons between preclean versus postclean for eachinstrument were as follows: SY1 (P � .006), SY2 (P � .183), and SY3(P � .007).

DISCUSSION

We validated 3 commercially available ATP luminometerysystems in accordance with standardized clinical laboratory prac-tice.12-14 The parallel design of the study allowed us to compareresults among instruments.

We examined the stability of ATP on surfaces over time toestablish whether ATP systems would accurately measure latentATP as well as newly deposited ATP. The SY3 system showed somevariability, whereas other systems were relatively stable. As welearned later in the study, this is most likely due to swab design andinstrument component sensitivity.

Fig 2. Line graph for 1:10 serial dilutions of S aureus dried onto steel plenums then swabbed and tested for ATP RLU with 3 separate ATP test systems. Instrument linearity plottedon logarithmic axes showing actual RLU values versus colony-forming units. Shown are the extrapolated exponential linearity, R2 value, and standard deviation error bars (n ¼ 2) foreach instrument.

Fig 3. Line graph for 1:10 serial dilutions of C albicans dried onto steel plenums then swabbed and tested for ATP RLU with 3 separate ATP test systems. Instrument linearity plottedon logarithmic axes showing actual RLU values versus colony-forming units. Shown are the extrapolated exponential linearity, R2 value, and standard deviation error bars (n ¼ 2) foreach instrument.

C.V. Sciortino, R.A. Giles / American Journal of Infection Control 40 (2012) e233-9 e237

We addressed the questions, “what percent of ATP is actuallyrecovered from surfaces coated with microorganisms and do differ-ences occur between ATP systems?” Our steel plenum results arerelative to the study design, not to be compared with the recovery ofATP from hospital surfaces. The variation in the percent efficiency ofATP recoverywas organism and systemdependent. Swabdesignmayexplain our higher recovery efficiency with the SY1 swab because it

waswet and hadmore contact surface area. The unexpected increasein ATP RLUs for SY1with C albicansmay be explained as follows: It ispossible that the mechanical swabbing of dried organisms with thelysis reagent swab caused cell damage and release of ATP, whereasdirect swab inoculation did not. Nevertheless, swab design should bean important consideration because hospital surfaces have a widevariation in composition, design, moisture, and biofilm burden.

Table 2Linear range and lower limit of detection by instrument and organism

Instrument Organism Linear range in cfu Lower LOD in cfu

1 A baumannii 102-107 102

S aureus 104-108 104

C albicans 10-107 102 A baumannii 103-107 102

S aureus 102-108 102

C albicans 102-107 102

3 A baumannii 104-107 103

S aureus 104-108 102

C albicans 103-107 102

C.V. Sciortino, R.A. Giles / American Journal of Infection Control 40 (2012) e233-9e238

The greatest challenge for the use of ATP technology to monitorhospital surfaces has been interpretation of results. In these studieswhen all surface RLUs from single rooms were combined, SY1and SY3 achieved a statistical difference between pre- versuspostcleaning samples, but SY2 did not. Brown et al15 recentlyargued that, because of interfering substances, ATP monitoring ofpatient room hygiene is not an effective tool. Our studies show that,even in the presence of detergents and disinfectants, 2 systemsgave discriminate data, whereas SY3 was minimally successful.These findings may account for the argument presented by Brownet al15 regarding earlier studies that were unable to differentiateprecleaned from postcleaned surfaces.

This study also examined the clinical specificity of the systems.Green et al16 reported that chlorine sanitizer quenched ATP reagentsfor certain systems. They added dilutions of chlorine sanitizerdirectly to purified ATP or chicken exudate ATP to test theirhypothesis. They reported that chlorine sanitizer decreased purifiedATP RLUs and only slightly reduced chicken exudate ATP RLUs.Brown et al15 also examined interfering substances on ATP recoverybut, again, used purified ATP for their assay.We dried test substancesonto stainless steel surfaces and then challenged treated surfaceswithmicroorganisms.We observed that ATP RLUswere significantlyaffected by some substances, but this was a random effect. No onesubstance systematically altered all measurable RLUs for all organ-isms; therefore, the definition of interference was not met. SomeRLUs even increased as previously reported.7 These random changesmay be explained as a killing effect when microorganisms die,releasing ATP, sometimes increasing RLU readings at first, followedby destruction of the ATP over a 10- to 30-minute period.

We chose not to show our ATP measurements from hospitalsurfaces because the goal of this study was to compare instrumentaccuracy and not to compare our hospital surface data with morecomprehensive trials.1,3,4,6-10,17 We observed statistically significantdifferences in pre- versus postclean swab data, but no differencesalso occurred. Whether postcleaned surfaces yielding high ATP RLUvalues is due to disruption of biofilms, dirty rags, or poor cleaning,that question is still unanswered.

We found that Gage analysis best represented our overallexperience with each instrument. Results for SY1 were highlyreproducible between operators, although there were occasionaloutliers (high values) with all systems. It had the highest clinicalsensitivity and could detect and discriminate minor differenceswithin the linear range. A high LOD for S aureus but low LOD forother bacteria was seen with SY1, indicating that the lysis reagentmay not have lysed S aureus as efficiently as other microorganismsor other system swabs (Table 2). For SY2, Gage analysis indicatedthat the greatest variation was between operators. This suggeststhat user training is important in establishing swabbing techniqueand that a single swab is more likely to give a false reading,requiring multiple sampling protocols. Review of all data indicatesthat SY3 was least sensitive because of the low instrument readingscale. This instrument design factor was not corrected by repeat

swabbing. The observation that the Gage analysis rated SY2 as gageokay (EV ¼ 17.4%) yet SY3 was rated as gage needs improvement(EV ¼ 12.1%) is further explained. Gage analysis compares repeat-ability (defined as: using the same gage, can the same operator getthe same result consistently?) with reproducibility (defined as: can2 operators measure the same item and get the same result?). Ifrepeatability > reproducibility then Gage analysis rates the gage asflawed. If reproducibility > repeatability Gage analysis rates thegage as okay and attributes the overall variation in R&R to theoperator.

It is difficult to correlate culture results with ATP readings.2,18,19

We found instrument-to-instrument data comparisons moreuseful. A weakness of this study is the lack of a “gold standard”eliminating the determination of the analytical sensitivity andspecificity. One could argue that purified ATP standards meet thispurpose,16 but purified ATP is different from ATP found on hospitalsurfaces and may introduce biochemical artifacts.

CONCLUSION

We demonstrated that ATP systems differ in respect to linearity,LOD, and clinical sensitivity. These studies validate that all 3 ATPsystemsreachedaLOD, clinical sensitivity, and linear rangeneeded forhospital surface monitoring. Two instruments failed verification,meaning that both systems require modifications to the manufac-turers’protocols to improvedataacquisitionandanalysis.Thisstudy isintended to aid users as a Rosetta Stone in the selection of an instru-ment that meets their intended purpose. Each user should verify andvalidate their system in-house, prior to setting interpretive criteria.

ATP system monitoring may uncover the need for new disin-fectant designs that remove hospital surface biofilms, renderingused hospital equipment to its native state whereby a zero readingby ATP monitoring can be achieved.

Acknowledgment

The authors thank our statistician, Timothy Wiemken, PhD,University of Louisville, Louisville, KY, for review of statisticalmethods, and Charm Sciences and the 3M Corporation for the loanof instruments and their contribution of swabs.

References

1. Dumigan DG, Boyce JM, Havill NL, Golebiewski M, Bologun O, Rizvani R. Who isreally caring for your environment of care? Developing standardized cleaningprocedures and effective monitoring techniques. Am J Infect Control 2010;38:387-92.

2. Dolan A, Bartlett M, McEntee B, Creamer E, Humphreys H. Evaluation ofdifferent methods to recover methicillin-resistant Staphylococcus aureus fromhospital environmental surfaces. J Hosp Infect 2011;79:227-30.

3. Lewis T, Griffith C, Gallo M, Weinbren M. A modified ATP benchmark forevaluating the cleaning of some hospital environmental surfaces. J Hosp Infect2008;69:156-63.

4. Huslage K, Rutala WA, Sickbert-Bennett E, Weber DJ. A quantitative approachto defining “high-touch” surfaces in hospitals. Infect Control Hosp Epidemiol2010;31:850-3.

5. Weber DJ, Rutala WA, Miller MB, Huslage K, Sickbert-Bennett E. Role of hospitalsurfaces in the transmission of emerging health care-associated pathogens:Norovirus, Clostridium difficile, and Acinetobacter species. Am J Infect Control2010;38:S25-33.

6. Carling PC, Bartley JM. Evaluating hygienic cleaning in health care settings: whatyou do not know can harm your patients. Am J Infect Control 2010;38:S41-50.

7. Boyce JM. Monitoring the effectiveness of hospital cleaning practices by use ofan adenosine triphosphate bioluminescence assay. Infect Control Hosp Epi-demiol 2009;30:678-84.

8. Willis C, Morley J, Westbury J, Greenwood M, Pallett A. Evaluation of ATPbioiluminescence swabbing as a monitoring and training tool for effectivehospital cleaning. Br J Infect Control 2007;8:17-21.

9. Moore G, Smyth D, Singleton J, Wilson P. The use of adenosine triphosphatebioluminescence to assess the efficacy of a modified cleaning program imple-mented within an intensive care setting. Am J Infect Cont 2010;38:617-22.

C.V. Sciortino, R.A. Giles / American Journal of Infection Control 40 (2012) e233-9 e239

10. Boyce JM, Havill NL, Lipka A, Havill H, Rizvani R. Variations in hospital dailycleaning practices. Infect Control Hosp Epidemiol 2010;31:99-101.

11. Simpson WJ, Giles CJ, Flockhart HA. Repeatability of hygiene test systems inmeasurement of low levels of ATP. Report 30606. Surrey [UK]: Cara TechnologyLimited, Leatherhead Enterprise Center; 2006.

12. Jennings L, Van Deerlin VM, Gulley ML. Recommended principles and practicesfor validating clinical molecular pathology tests. Arch Path Lab Med 2009;133:743-55.

13. Clinical and Laboratory Standards Institute (CLSI). User verification ofperformance for precision and trueness; approved guideline. Second Edition.2005; CLSI document EP15-A2 (ISBN 1-56238-574-7). Wayne [PA]: CLSI;2005.

14. Clinical and Laboratory Standards Institute (CLSI). Evaluation of the linearity ofquantitative measurement procedures: a statistical approach; approvedguideline. 2003. CLSI document EP6-A (ISBN 1-56238-498-8). Wayne [PA]:CLSI; 2003.

15. Brown E, Eder AR, Thompson KM. Do surface and cleaning chemistries interferewith ATP measurement systems for monitoring patient room hygiene? J HospInfect 2010;74:193-5.

16. Green TA, Russell SM, Fletcher DL. Effect of chemical cleaning agents andcommercial sanitizers on ATP bioluminescence measurements. J Food Protect1999;62:86-90.

17. Heathcote R, Stadelmann B. Measuring of ATP bioluminescence as a means ofassessing washer disinfector performance and potentially as a means ofvalidating the decontamination process. Healthc Infect 2009;14:147-51.

18. Lappalainen J, Loikkanen S, Havana M, Karp M, Sjöberg AM, Wirtanen G.Microbial testing methods for detection of residual cleaning agents anddisinfectants: prevention of ATP bioluminescence measurement errors in thefood industry. J Food Protect 2000;63:210-5.

19. Chen FC, Godwin SL. Comparison of a rapid ATP bioluminescence assay andstandard plate count methods for assessing microbial contamination ofconsumers’ refrigerators. J Food Protection 2006;69:2534-8.