Research

Binary Cumulative Sums and Moving Averages in Nosocomial Infection Cluster Detection1

Samuel M. Brown,* James C. Benneyan,† Daniel A. Theobald,‡ Kenneth Sands,§ Matthew T. Hahn,‡ Gail A. Potter-Bynoe,¶ John M. Stelling,#** Thomas F. O'Brien,#** and Donald A. Goldmann¶
*Massachusetts General Hospital, Boston, Massachusetts, USA; †Northeastern University, Boston, Massachusetts, USA; ‡Vecna Technologies, Inc., Hyattsville, Maryland, USA; §Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA; ¶Children's Hospital, Boston, Massachusetts, USA; #WHO Collaborating Center for Antimicrobial Resistance Surveillance, Boston, Massachusetts, USA; and **Brigham and Women's Hospital, Boston, Massachusetts, USA

Suggested citation for this article: Brown SM, Benneyan JC, Theobald DA, Sands K, Hahn MT, Potter-Bynoe GA et al. Use of binary cumulative sums and moving averages in nosocomial infection cluster detection. Emerg Infect Dis [serial online] 2002 Dec [date cited];8. Available from: URL: http://www.cdc.gov/ncidod/EID/vol8no12/01-0514.htm

Clusters of nosocomial infection often occur undetected, at substantial cost to the medical system and individual patients. We evaluated binary cumulative sum (CUSUM) and moving average (MA) control charts for automated detection of nosocomial clusters. We selected two outbreaks with genotyped strains and used resistance as inputs to the control charts. We identified design parameters for the CUSUM and MA (window size, k, , ß, p₀, p₁) that detected both outbreaks, then calculated an associated positive predictive value (PPV) and time until detection (TUD) for sensitive charts. For CUSUM, optimal performance (high PPV, low TUD, fully sensitive) was for 0.1 < <0.25 and 0.2 <ß <0.25, with p₀ = 0.05, with a mean TUD of 20 (range 8–43) isolates. Mean PPV was 96.5% (relaxed criteria) to 82.6% (strict criteria). MAs had a mean PPV of 88.5% (relaxed criteria) to 46.1% (strict criteria). CUSUM and MA may be useful techniques for automated surveillance of resistant infections.

Nosocomial infections afflict 2 to 5 million patients in the United States annually and contribute to approximately 88,000 deaths (1,2). These infections are the second most frequent adverse effect of hospitalization (3,4).  In most instances such infections are isolated, though studies have reported that from 2% (5,6)  to 20% (7) to 60% (8) occur in clusters. A minimal estimate of the epidemic nosocomial infection burden is thus 40,000 cases annually (2% of 2,000,000), while a maximal estimate is conceivably five times that figure or more.

Most hospitals in the United States will have at least one outbreak per year, and large referral hospitals may have several (9). Nosocomial infection clusters can be difficult to diagnose and detect (5), which can have serious ramifications (10). Although options for computerized surveillance are increasing (11–15),  many current methods for outbreak detection are effective only when substantial time has elapsed from the actual events. Techniques are often poorly automated (16–18),  and few sophisticated cluster detection techniques have been employed in nosocomial infection surveillance (19–21).

Cumulative sums (CUSUMs) are statistical tools, based on a type of sequential hypothesis test, that were originally used in manufacturing processes to monitor production defect rates (22–24).  Increments are added or decrements are subtracted from a running total over time, according to measurements of quality of serial items. The behavior of this cumulative sum is tracked until one of two conditions is met, with CUSUM values beyond these thresholds signaling either 1) a statistically significant change in quality to some prespecified level or 2) acceptance of the hypothesis of no change. CUSUMs have been used for several decades in health care settings, including for tracking operator improvements in performing procedure (25–27),  monitoring fever curves in neutropenic patients (28), and detecting community Salmonella outbreaks (15). Several forms exist, including a so-called binary or Bernoulli CUSUM in which failure is rated as 1 and success as 0, a coefficient is subtracted, and the resulting values are added to the CUSUM. This binary form has not to our knowledge been applied to outbreak detection.

Moving averages (MAs) are in wide use in several fields, such as economics, where methods sensitive to sudden changes and filtering out background noise are required. Thus, for instance, economic indicators may be analyzed, with a MA calculated for the most recent values and compared with the historical mean for that indicator. An MA much higher than the historical mean indicates a statistical increase. MAs also are used in manufacturing quality control for the same reason (28). Although various MA techniques have been applied to disease rates in public health surveillance (29), they have not previously been applied to monitor changes in strain characteristics, such as antimicrobial resistance.

We hypothesized that by treating antimicrobial resistance as the quality indicator of individual isolates, these techniques could be used to detect nosocomial clusters. Both techniques have been demonstrated in the quality control literature to be more sensitive to small rate changes than conventional p-type charts (22–24,30) . We evaluated the performance of these techniques in simulated real-time detection of two genotypically characterized outbreaks of nosocomial infection caused by antimicrobial-resistant bacteria.

Methods

Outbreaks Investigated

The study hospital is a 330-bed tertiary-care pediatric facility in the northeastern United States. We selected all investigated nosocomial outbreaks of antibiotic-resistant bacteria in the study hospital for which genotyping data were available for the period 1995–2000, inclusive. An outbreak with genotyped organisms from 1997 was excluded because the causative agent, Pseudomonas aeruginosa, was sensitive to all standard therapeutic agents. This cluster was thus not a candidate for detection with our techniques. A line listing of all patients, with isolates, from both outbreaks is presented in the Table.

The Institutional Review Board of the study hospital authorized us to perform this study without obtaining informed consent. All patient identifiers were either deleted or irreversibly encrypted to ensure confidentiality.

Outbreak 1

An outbreak of surgical site infections caused by methicillin-resistant Staphylococcus aureus (MRSA) occurred in August through September 1999 in patients after cardiac surgery. Approximately 800 such surgeries are performed annually in the study hospital. Immediately after surgery, patients are cared for in the cardiovascular intensive care unit (CICU), which has 23 beds, 1,550 admissions per year, and an average length of stay of 4.4 days. After they are stabilized, the patients are transferred to the cardiac surgery ward (28 beds, >2,300 admissions per year; and average length of stay, 3 days). A single genotype of MRSA was isolated from four patients with evidence of deep/organ-space surgical infection after cardiac surgery. One of the genotypically identical isolates (O3-2) was detected by admission screening culture at another hospital to which the patient had been transferred. Another isolate (O3-7) was detected in a blood culture obtained at the hospital to which the patient had been transferred. Two surgical patients without clinical infection were colonized with isolates of a second genotype. Methicillin resistance was defined as a MIC of oxacillin of >0.5 µg/ml. All isolates of Staphylococcus aureus from any body site from the CICU and cardiac surgical ward were included in the analyses.

Outbreak 2

An outbreak of vancomycin-resistant enterococcus (VRE) occurred in May through June 2000 involving two units: the bone marrow transplant unit and the general pediatric intensive care unit PICU. The bone marrow transplant unit is a 13-bed unit providing hematopoietic stem-cell transplantation. It has approximately 260 admissions per year, with an average length of stay of 12.9 days. When patients require ICU care, they are transferred to specially ventilated rooms in the PICU. The PICU is an 18-bed multidisciplinary unit, with approximately 1,650 admissions per year, and an average length of stay of 3.2 days. In May 2000, a patient colonized with VRE in the bone marrow transplant unit was transferred to the PICU. Other cases of VRE colonization or infection were detected in both the bone marrow transplant unit (4 cases) and the PICU (3 cases). Isolates of Enteroccocus faecium  from five patients were demonstrated to be genotypically identical. Vancomycin resistance was defined as a MIC of vancomycin of >16 µg/mL. All isolates of E.faecium or unspeciated Enterococcus from any body site on the affected units were included in the analyses. Genotyping was performed by ARUP Laboratories (Salt Lake City, UT). Genotypic identity was defined according to a published procedure (31).

Data Acquisition:

Records for all inpatient cultures were downloaded from the study hospital’s information system for January 1995–September 2000 into WHONET 5.0 (WHO Collaborating Center, Boston, MA). Species identification had been performed per standard laboratory procedures. Antibiotic sensitivities had been performed by measurement MIC with a MicroScan Walkaway-96 (Dade Behring, Inc., Deerfield, IL). Standard Kirby-Bauer technique was used when an organism failed to grow sufficiently to perform MIC analysis. Only final susceptibility readings were included. Susceptibility cutoffs were defined according to National Committee for Clinical Laboratory Standards (32). Indication for culture was specified as either clinical (C), routine surveillance (R), or outbreak investigation (O). Clinical cultures were ordered by treating physicians for care of the individual patient. Routine surveillance cultures included weekly stool screens for VRE and sentinel event screens. Infection control policy at the study hospital was to screen a high-risk unit (ICU or bone marrow transplant unit) if a patient was found to have new MRSA or VRE colonization or infection. Outbreak investigation cultures were those taken as part of a formal or informal outbreak investigation. Culture indications were determined from infection control records.

Data Analysis

Isolates of the same species from a given patient within 60 days of the previous isolate were excluded as duplicate isolates. All isolates of E. faecium, enterococcus, and S. aureus from the affected units were parsed by the BugCruncher program (Vecna Technologies, Hyattsville, MD) in the manner depicted in Figures 1 and 2. The resistance value (for binary tests 0 = susceptible or 1 = nonsusceptible; for quantitative tests, the actual MIC) for each isolate was then passed to CUSUM (binary only) or MA (binary and quantitative) modules, where alerts were generated on the basis of control limits. Test statistics and control limits were recalculated with the addition of each new isolate and processed in chronological order.

Each type of chart is calculated based on several design parameters (w and k for MA; a, β, p₀, p₁ for CUSUM). To explore performance robustness under various conditions, we selected a reasonable range of values for the control parameters for CUSUM (0.01 < <0.25; 0.01 < ß < 0.25; 0.01 < p₀ < 0.25; 0.01 <p₁ <0.25) and MA (5 < w <90 and 1 < k < 4) charts. Positive predictive value (PPV) was calculated for those design parameter values that detected both outbreaks. Further detail on these statistical methods and the formulae used for calculating their test statistics and detection thresholds are presented in the Appendix.

To validate the empirically derived design parameters in terms of theoretic performance, we then calculated the out-of-control (an actual change in incidence) and in-control (no change in incidence) time until detection (TUD) for the sets of design parameters that detected both outbreaks. We used standard methods for calculating TUDs, employing a Monte Carlo simulation program we wrote for that purpose. Simulations were run over 10,000 iterations.

Two definitions of cluster detection were used: generation of an alert at the second outbreak isolate (isolate-level detection) or during the first month of the outbreak (month-level detection). Positive predictive value (percent of detected events considered relevant) was calculated in the following manner (33) all detected events previously unnoted by infection control personnel were evaluated independently by two hospital epidemiologists (KS, DG). The epidemiologists classified each event as A) initiate investigation, B) monitor situation, or C) ignore. A “C” rating from both epidemiologists or a “B” from one and a “C” from the other was considered a false-positive result. True positives were divided into positives by strict criteria (receiving an “A” rating) and by relaxed criteria (receiving at least “B” ratings from both epidemiologists). PPVs were calculated by strict and relaxed criteria separately.

Results

Cluster Descriptions

The dataset contained a total of 6,382 positive cultures of any organism (from 3,346 different patients) from the units affected by the outbreak of oxacillin-resistant S. aureus. Of those, 728 (from 323 patients) were S. aureus. Of the 323 unique isolates of S. aureus in the affected units, 14 (4.3%) were oxacillin resistant, whereas for the hospital as a whole 84 (4.2%) of 1,983 S. aureus isolates were oxacillin resistant.

The dataset contained a total of 9,012 positive cultures of any organism (from 4,315 patients) from the units affected by the outbreak of vancomycin-resistant enterococcus. In the affected units, 21 (14.1%) of 149 enterococcal isolates were vancomycin resistant, whereas for the entire hospital 41 (5.3%) of 768 enterococcal isolates were vancomycin resistant.

For all implicated units, the 15 most common bacterial species represented 4,948 unique isolates, an average of 18 per unit per month. Overall 165 different organisms were isolated, 74 of them representing only three or fewer isolates over the 69 months included in the dataset.

CUSUMs

Several CUSUM charts proved capable of detecting both outbreaks by the second isolate. Figure 3 displays a representative CUSUM chart, which detected the VRE outbreak early in its course. Maximal performance robustness was obtained when 0.1< <2 and 0.2 < ß <0.25, with p₀ = 0.05. Values of ß< 0.2 were associated with poor performance.

Monte Carlo simulations, run with p₁ = 0.2 over the sets of design parameters that performed most robustly, yielded an out-of-control TUD ranging from 8 to 45 isolates (average 20.4), and an in-control TUD, ranging from 55 to 2,390 isolates (average 427). Both the out-of-control TUD and in-control TUD decreased with higher values of ; for = 0.2 or 0.25, the in-control TUD ranged from 55 to 88; whereas at = 0.1, it ranged from 184 to 306 isolates.

The mean PPV of CUSUM techniques ranged from 96.5% (relaxed criteria) to 82.6% (strict criteria). Lower values for were associated with higher PPV. On average, the sensitive control charts generated 9.5 novel alerts over the 69 months of the study period, or 1.6 events per year for all involved units and organisms (enterococcus, S. aureus). 

Moving Averages

For MA control charts, only those which used quantitative MICs (vancomycin: 2–16 µg/mL; oxacillin: 0.25–4 µg/mL) were capable of detecting both outbreaks; no binary (susceptible = 0; nonsusceptible = 1) MA charts detected both outbreaks. Sensitive window sizes (w, the number of isolates considered in calculating the MA) varied from 5 to 30 isolates. Parameter sets with larger window sizes failed to detect both outbreaks.

Monte Carlo simulations for the design parameters that detected both outbreaks, assuming a change in MICs of one standard deviation, yielded an out-of-control TUD ranging from 4 to 10,796 isolates (mean 1,568; median 14), and an in-control TUD ranging from 11 to 25,488 (mean 4,006; median 180). For k < 4, the mean out-of-control TUD was 14, while the mean in-control TUD was 350 isolates.

Figure 4 displays a representative MA test combination that detected the MRSA outbreak by the second isolate. The mean PPV ranged from 88.5% (relaxed criteria) to 46.1% (strict criteria). On average, sensitive MA charts generated 10.9 novel alerts over the entire study period, or 1.9 per year for all units and organisms studied.

	Figure 1
	Click to view enlarged image Figure 1. Data processing methodology for cumulative sums...
	Figure 2
	Click to view enlarged image Figure 2. Data-processing methodology for moving averages...

Discussion

We illustrated the performance of a system designed for real-time monitoring of clinical microbiology data from the hospital laboratory information system. Two techniques borrowed from other domains were capable of detecting two carefully characterized outbreaks in simulated real time. The binary CUSUM proved more robust than MAs.

Many metrics for outbreak detection are based on month of outbreak (11,14,15,17,18,34),  whereas in nosocomial outbreaks greater attention to individual cases is probably warranted given the smaller numbers of patients involved, the possibility of early definitive intervention, and the comorbidities of infected patients. The techniques used in this study proved capable of detecting an outbreak before the end of a monthly surveillance period.

The reproducibility of these findings is of key importance. We used an a priori reasonable set of possible design parameter values, then combined empirical evaluation of their performance with theoretical evaluation via Monte Carlo simulations.

We used only two outbreaks for evaluation, given the difficulty of generating and validating such datasets. A study that investigates larger numbers of similar outbreaks would improve generalizability. The theoretical simulations tend to support the generalizability of the test statistics used, as the empirically robust design parameters were associated with low out-of-control and high in-control TUD values.

The techniques appear most useful when the baseline incidence is relatively low, and it is unclear whether these methods would be applicable in settings where antibiotic-resistant bacteria are more common, as the study hospital had relatively low rates of MRSA and VRE.

The surveillance methods evaluated here are primarily useful for detecting outbreaks caused by resistant organisms. In their current implementation, they would not be useful for settings where outbreaks are caused by organisms whose antibiotic susceptibilities are indistinguishable from those of endemic flora, as in the cluster of Pseudomonas excluded from the present study. Additional research would be required to make these methods applicable in those settings.

From a practical perspective, the CUSUM charts detected the outbreaks by the second isolate, a finding corroborated by results of the Monte Carlo simulations. An increased incidence from 0.05 to 0.20 would be detected on average within 1.5 actual outbreak isolates for an out-of-control TUD of 10 (best-performing CUSUM), or at the third outbreak isolate for an out-of-control TUD of 20 (mean CUSUM performance). These results, supported empirically and theoretically, are consistent with the goals of nosocomial outbreak detection.

	Figure 3
	Click to view enlarged image Figure 3. Cumulative sum test iteration detecting an outbreak of vancomycin-resistant enterococcus...
	Figure 4
	Click to view enlarged image Figure 4. Moving average test iteration detecting an outbreak of methicillin-resistant Staphylococcus aureus...

In terms of resources potentially wasted on false-positive results, the CUSUM charts that detected both outbreaks were remarkably accurate, with an average PPV of >80%, even by strict criteria, whereas the MIC MA parameter sets had lower PPVs. According to our calculated PPV for CUSUM, only 1 in 20 alerts would be deemed retrospectively as unworthy of any further evaluation, while 1 in 5 would not be deemed worthy of actual investigation. Assuming an annual rate of 1 alert per organism and unit, 4 units under surveillance, and 15 organisms under surveillance, 60 alerts would be generated annually, of which 12 would not be deemed worthy of attention, approximately one false alarm per month. Slightly more than twice as many would be considered spurious in retrospect on the basis of the MA results.

Using the in-control TUD values to estimate the frequency of spurious results yields a better estimate. With 18 isolates of the 15 most commonly isolated bacteria per unit per month, 4 units under surveillance, we would anticipate 72 isolates per month. The mean in-control TUD value for CUSUM charts is 427, suggesting a false-positive alert once every 5 months, though false-positive alerts are associated with a higher out-of-control TUD. Taking the chart with the lowest out-of-control TUD, the in-control TUD is 55, suggesting a false-positive result slightly more than once per month, similar to our observed rate.

Strengths of this study include the availability of genotyping data for outbreak characterization and the availability of quantitative MICs, the use of practical outcome measures, and combination of empirical and theoretical methods for evaluating test statistics.

An additional problem in validating detection techniques is the lack of a gold standard for determining the relevance of a computer-detected cluster. We chose a practical approach, given the ultimate clinical application of such a system. We may have overestimated the positive predictive value, although we evaluated by both strict and relaxed criteria. At the time of evaluation, reviewers were unaware of events that followed, decreasing the probability of outcome-based bias. A prospective trial of these techniques, with collection of genotyping information, should help to resolve this problem.

Areas for additional research include methods for analyzing duplicate isolates from a single patient, more sophisticated techniques for modeling patient location, accounting robustly for changes in sampling intensity, methods for using quantitative CUSUMs, and the potential need for corrections for interdependence.

CUSUM and MA analyses of antimicrobial resistance proved capable of detecting two important nosocomial outbreaks early in their course in simulated real time. Both methods had relatively high positive predictive values; CUSUM performed better than MA. These analytical techniques may be of value in automated detection of nosocomial outbreaks and should be evaluated in real-time clinical practice.

Acknowledgments

We are grateful to Tim Martin, who designed the graphical output; Edward O'Rourke, who provided early domain guidance and analysis; and Sophie Yuckienuz, Karla O'Byrne, and Daniel Dicicco, who extracted data from hospital information systems.

Mr. Hahn has future equity in Vecna Technologies, Inc, and Mr. Theobald is part-owner of Vecna Technologies, Inc. All other authors were paid consultants for the purposes of this study; they have no other financial association with Vecna Technologies, Inc. This work was supported by a Small Business Innovation Research Grant (1 R43 AI48332-01) from the National Institute of Allergy and Infectious Disease. Dr. Benneyan was partially supported by National Science Foundation Grant DMI-0085262.

Dr. Brown is a resident in internal medicine at Massachusetts General Hospital. His research interests include nosocomial infections, antibiotic resistance, quality of medical care, outbreak detection, and infectious disease control in areas with limited resources.

Appendix: Calculating Test Statistics

Binary cumulative sum charts, based on the theory of sequential probability ratio tests, monitor a cumulative term that is incremented or decremented by certain amounts for each positive or negative result, respectively, in order to sequentially test between user-specified acceptable and unacceptable rates (35,36)  (Equation 1). In our application, the CUSUM statistic S_i is reduced at the time of each isolate by an amount D, a calculated value that depends on the shift we wish to detect, and then increased by 1 for those isolates that are antibiotic resistant. The plotted statistic for the ith isolate, S_i, and the control limit factors h₀ and h₁ are calculated as

S_i               =    =     S_i-1 + X_i - D ,                                    (1)

h₀               =    , and                                                                          (2)

h₁               =    ,                                                                                (3)

where X_i = 1 if the ith isolate is resistant and 0 if it is not, the decrement D is computed as

D               =    ,                                                                               (4)

is the desired type I error rate, is the desired type II error rate, p₀ is the acceptable occurrence rate, p₁ is the unacceptable occurrence rate that is desired to be detected, and S₀ = 0 as a starting value.

The cumulative sum then is compared to nonconstant control limits that periodically are recalculated by subtracting h₀ from and adding h₁ to any S_i value that falls outside either limit, resulting in new limits until the next such violation and starting with lower control limit (LCL) = S₀ - h₀ = h₀ and upper control limit (UCL) = S₀ + h₁ = h₁ . Values above the UCL indicate an outbreak, i.e., rejection of the hypothesis of p₀ in favor of the hypothesis of p₁, although contrary to traditional control charts values beneath the LCL here do not indicate a rate decrease but rather acceptance of p₀ over p₁.

For the moving average (MA) charts, the moving average for the ith isolate with a “window” of size w (varied in different test conditions), Y_w,i, is calculated as

Y_w,i             =                           (4)

This result then is compared to estimated upper (UCL) and lower k-sigma control limits for the ith isolate, LCL_i and UCL_i, with the standard deviation of the ith moving average, Y,w,i, estimated by using the conventional moving range (MR) control chart method for individual data that occur over time,

 ,                                                                 (5)

 ,                                                                                (6)

 , and                                                 (7)

 ,                                                       (8)

all for i >2, where i is the current total number of data points, X_i is the ith data value, w is the size of the moving average, and  is the average of all data up to and including the ith data value. An MA value that exceeds its corresponding UCL will trigger an outbreak alert.

References

1. Haley RW, Culver DH, White JW, Morgan WM, Emori TG, Munn VP, et al. The efficacy of infection surveillance and control programs in preventing nosocomial infections in US hospitals. Am J Epidemiol 1985;121:182–205.
2. Committee on Quality of Health Care in America. To err is human: building a safer health system. Kohn LT, Corrigan JM, Donaldson MS, editors. Washington, D.C.:Institute of Medicine; National Academy Press; 2000
3. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I. N Engl J Med 1991;324:370–6.
4. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, et al. The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. N Engl J Med 1991;324:377–84.
5. Stamm WE, Weinstein RA, Dixon RE. Comparison of endemic and epidemic nosocomial infections. Am J Med 1981;70:393–7.
6. Scheckler WE. Nosocomial infections in a community hospital: 1972 through 1976. Arch Intern Med 1978;138:1792–4.
7. Wenzel RP, Thompson RL, Landry SM, Russell BS, Miller PJ, Ponce de Leon S, et al. Hospital-acquired infections in intensive care unit patients: an overview with emphasis on epidemics. Infect Control 1983;4:371–5.
8. Gastmeier P, Sohr D, Geffers C,.Nassauer A, Dettenkofer D, Ruden H. Occurrence of methicillin-resistant Staphylococcus aureus in German intensive care units. Infection 2002;30:198–202.
9. Haley RW, Tenney JH, Lindsey JO, Garner JS, Bennett JV. How frequent are outbreaks of nosocomial infection in community hospitals? Infect Control 1985;6:233–6.
10. Goldmann DA, Dixon RE, Fulkerson CC, Maki DG, Martin SM, Bennett JV. The role of nationwide nosocomial infection surveillance in detecting epidemic bacteremia due to contaminated intravenous fluids. Am J Epidemiol 1978;108:207–13.
11. Brossette SE, Sprague AP, Jones WT, Moser SA. A data mining system for infection control surveillance. Methods Inf Med 2000;39:303–10.
12. Ngo L, Tager IB, Hadley D. Application of exponential smoothing for nosocomial infection surveillance. Am J Epidemiol 1996;143:637–47.
13. Sahm DF, O'Brien TF. Detection and surveillance of antimicrobial resistance. Trends Microbiol 1994;2:366–71.
14. Stern L, Lightfoot D. Automated outbreak detection: a quantitative retrospective analysis. Epidemiol Infect 1999;122:103–10.
15. Hutwagner LC, Maloney EK, Bean NH, Slutsker L, Martin SM. Using laboratory-based surveillance data for prevention: an algorithm for detecting Salmonella outbreaks. Emerg Infect Dis 1997;3:395–400.
16. Birnbaum D. Analysis of hospital infection surveillance data. Infect Control 1984;5:332–8.
17. Childress JA, Childress JD. Statistical test for possible infection outbreaks. Infect Control 1981;2:247–9.
18. McGuckin MB, Abrutyn E. A surveillance method for early detection of nosocomial outbreaks. APIC 1979;7:18–21.
19. Koontz FP. A review of traditional resistance surveillance methodologies and infection control. Diagn Microbiol Infect Dis 1992;15(2 Suppl):43S–7.
20. Jacquez GM, Waller LA, Grimson R, Wartenberg D. The analysis of disease clusters, Part I: state of the art. Infect Control Hosp Epidemiol 1996;17:319–27.
21. Jacquez GM, Grimson R, Waller LA, Wartenberg D. The analysis of disease clusters, Part II: introduction to techniques. Infect Control Hosp Epidemiol 1996;17:385–97.
22. Kenett RS, Zachs S. Modern industrial statistics. Belmont (CA): Duxbury Press; 1998.
23. Lucas JM. Counted data cusum. Technometrics 1985;27:129–44.
24. Reynolds MR, Stoumbos ZG. A cusum chart for monitoring a proportion when inspecting continuously. Journal of Quality Technology 1999;31:87–108.
25. Parry BR, Williams SM. Competency and the colonoscopist: a learning curve. Aust N Z J Surg 1991;61:419–22.
26. Williams SM, Parry BR, Schlup MM. Quality control: an application of the cusum. BMJ 1992;304:1359–61.
27. Bolsin S, Colson M. The use of the Cusum technique in the assessment of trainee competence in new procedures. Int J Qual Health Care 2000;12:433–8.
28. 28.  Kinsey SE, Giles FJ, Holton J. Cusum plotting of temperature charts for assessing antimicrobial treatment in neutropenic patients. BMJ 1989;299:775–6.
29. Nobre FF, Monteiro AB, Telles PR, Williamson GD. Dynamic linear model and SARIMA: a comparison of their forecasting performance in epidemiology. Stat Med 2001;20:3051–69.
30. Montgomery DC. Introduction to statistical quality control. 4th ed. New York: Wiley; 2001
31. Tenover FC, Arbeit RD, Goering RV. How to select and interpret molecular strain typing methods for epidemiological studies of bacterial infections: a review for healthcare epidemiologists. Molecular Typing Working Group of the Society for Healthcare Epidemiology of America. Infect Control Hosp Epidemiol 1997;18:426–39.
32. National Committee for Clinical Laboratory Standards (NCCLS). Methods for dilution antimicrobial susceptibility tests for bacteria that grow aerobically—Fourth Edition; approved Standard. 1997. Wayne (PA): NCCLS; NCCLS document M7-A4.
33. Klaucke DN, Buehler JW, Thacker SB, Gibson RG, Trowbridge FL, Berkelman RL. Guidelines for evaluating surveillance systems. MMWR Morb Mortal Wkly Rep 1988;37:1–17.
34. Moser SA, Jones WT, Brossette SE. Application of data mining to intensive care unit microbiologic data. Emerg Infect Dis 1999;5:454–7.
35. Lucas JM. Counted data cusum. Technometrics 1985; 27:129–44.
36. Reynolds MR, Stoumbos ZG. A cusum chart for monitoring a proportion when inspecting continuously. Journal of Quality Technology 1999; 31:87–108.

[1]Portions of this research were presented at the 39th Annual Meeting of the Infectious Diseases Society of America (IDSA), San Francisco, California, USA, October 25–28, 2001.

EID Home | Top of Page | Ahead-of-Print | Past Issues | Suggested Citation | EID Search | Contact Us | Accessibility | Privacy Policy Notice | CDC Home | CDC Search | Health Topics A-Z

This page posted November 4, 2002
This page last reviewed January 7, 2004

Emerging Infectious Diseases Journal
National Center for Infectious Diseases
Centers for Disease Control and Prevention