Using Meta Analysis Techniques to Assess the

Safety Effect of Red Light Running Cameras

Prepared by

Aimee Flannery, Ph.D. and Robert Maccubbin

Mitretek Systems

February 2002

PDF Version 415KB

ABSTRACT

Automated enforcement programs, including automated systems that are used to enforce red light running violations, have recently come under scrutiny regarding their value in terms of improving safety, their primary purpose. One of the major hurdles to overcome in assessing the value of automated enforcement programs is the limited amount of before/after crash and violation data available. The authors of this paper were able to conduct a wide reaching literature review and, in addition, were able to gather together crash data from a few sites in the United States. This study draws on several techniques, termed meta-analysis techniques, to evaluate the effect of automated enforcement cameras on reducing crashes and violations at signalized intersections.

Meta-analysis techniques are a family of statistical techniques developed to allow researchers to quantitatively synthesize the findings of a set of evaluation studies that were conducted over a span of years, conducted in several countries, and published in several formats (Elvik, 1999). Meta-analysis techniques summarize the results of several studies and provide an estimate of the average effect of a measure, in this case the effect of automated enforcement systems in reducing crashes at signalized intersections.

The small number of identified studies that reported useable crash frequency data limited the number of meta-analysis techniques performed in this study. Even so, the tests that could be conducted demonstrated that the available data appear to be consistent. The mean, mode, and median effect of automated enforcement cameras on reducing combined right-angle and rear-end crashes were each found to be very close to 26%. This means that data from the two studies included indicate the probability of these types of accidents occurring in the after period was reduced by approximately 26%. Given the data limitations, these results should not be emphasized. Rather, the value of this research effort is the demonstration of meta-analysis methods applied to assess the impact of an ITS application. Additional data are needed to fully determine the effect of automated red light enforcement systems on safety at signalized intersections.

1.0 Introduction

Technological advancements have enabled the introduction of automated enforcement programs for several areas including traffic signal enforcement, speed limit enforcement, and toll collection enforcement. One of the more prominent of these applications is the use of automated enforcement to deter red light violations at signalized intersections. The use of automated enforcement systems at signalized intersections is increasing in the United States (U.S.). In late 1999, the Institute of Transportation Engineers (ITE) identified thirty U.S. communities with operational automated red light enforcement programs, (ITE, 1999). As of November 2001, the Insurance Institute for Highway Safety reports that over sixty U.S. communities currently operate automated red light violation reduction programs (IIHS, 2001). Recent interest from the U.S. Congress has called into question the effectiveness of automated traffic signal enforcement systems in improving traffic safety. While much has been written about the implementation and operation of these systems, few studies have investigated the safety impact of these programs. In addition, the few studies that have been published primarily report violation reductions in lieu of crash performance data. The work presented in this paper demonstrates how the techniques of meta-analysis can be applied to draw some early conclusions regarding the effectiveness of automated enforcement programs in improving intersection safety in the U.S.

Recent synthesis studies of the automated red light enforcement programs in the U.S. describe the operation of the programs and some early results regarding their effectiveness. Three reports prepared between 1997 and 2000 describe the implementation and operation of automated enforcement programs in San Francisco, CA, New York, NY, Polk County, FL, and Howard County, MD. These reports also contain available information on the impact of the systems, typically before/after comparisons of area-wide crash statistics or violation reductions between the initial operation of the systems and a point in time after implementation (Passetti, 1997; FHWA, 1999; Smith, et al., 2000). The authors of each of these syntheses call attention to the need for additional, more detailed data to better establish the impact of the systems.

The 1999 ITE report, Automated Enforcement in Transportation, documents the techniques for implementing automated enforcement programs, legal requirements regarding their implementation, operating characteristics of the programs, as well as the results achieved by several of the programs. In addition to the communities discussed in the three syntheses described previously, the report describes experiences in Oxnard, CA, Lincoln, NE, and Victoria, Australia (ITE, 1999). An earlier Australian literature review (Zaal, 1994) documented the experience in Australia and cited a violation reduction through the operation of an automated red light enforcement program in the United Kingdom. Rocchi and Hemsing (1999) also documented reported reductions in violation and collision rates in the U.S., Australia, and the United Kingdom. While previous efforts have documented reported impacts of automated red light enforcement programs in individual localities, this research effort seeks to integrate similar data from multiple locations to develop a more complete assessment of the programs’ impacts. In addition to reviewing existing studies, the authors were able to collect additional crash data from a few programs in the U.S. Because automated enforcement programs have not existed for very long, this application of meta-analysis techniques is somewhat experimental or exploratory in nature.

2.0 Background on Meta-Analysis Techniques

Evaluating the performance of improvements, operational or geometric, is common practice to determine if the changes have had an effect on either operational or safety performance. Meta-analysis techniques refer to a set of statistical techniques that provide the means for synthesizing the results of a set of evaluation studies (Elvik, 1999). Meta-analysis techniques allow a researcher to summarize the findings of a series of evaluation studies in order to estimate the overall effect of a particular improvement or characteristic. For instance, assume an agency would like to investigate the safety effects of a particular intersection design before making a decision to invest in this type of design. The agency could perform a literature review to gather together evaluation studies that investigated the safety performance of this particular intersection design. The studies found would likely come from different states, countries, time periods, etc (variations that can be accounted for in a meta-analysis study). Meta analysis techniques could then be used to quantitatively synthesize the findings of the various studies that investigated the safety performance of this particular intersection design.

2.1 Meta-Analysis Tests and Methods

There are several simple statistical and graphical methods of analysis that can be used to determine the explanatory value of the mean effect of a treatment estimated from a set of evaluative studies. A brief overview of the various methods are described here; however, the interested reader is referred to a more comprehensive review of the methods as described in several journal articles by Rune Elvik (Elvik, 1999, Elvik, 2001).

The c² test for homogeneity can detect the presence of significant heterogeneity in the sample results. Significant heterogeneity can be interpreted to mean that there is more than one significant factor influencing the findings of the studies. The occurrence of an accident and the severity of an accident are of course influenced by a number of factors. Many researchers believe that if all of these factors are attempted to be accounted for in traffic safety studies, it will be impossible to conclude the effects of improvements or changes to the roadway (Elvik, 1999). The c² test can be used to partition a set of results into smaller subsets in an attempt to account for factors, other than the test factor, that are influencing the performance of study sites. If a significant number of factors are found to influence the results, then a random effects model should be implemented which will attempt to identify the various factors and assign appropriate weights for these factors.

Results from several evaluation studies can be compared and tested to ensure that one or more studies are not highly skewed in their measure of the mean effect of a treatment. A highly skewed data set will have a large number of points lying to the right or left of the mean value. A test for skewness can be performed using plots that depict the weighted mean, median, and mode on the same graph. Visual inspection of the plots can help the researcher rule out the presence of highly skewed findings. It is desirable to have very little skewing of the data. If the data are not skewed, or skewed very little, then the mean result has more meaning and provides a better sense of the overall mean effect of a treatment. The opposite is true also, in that a highly skewed data set will result in a mean effect measure that can be misleading.

A test for modality among the studies can also be performed. Modality, meaning the number of peaks in the data when plotted, can help the researcher determine if estimating a weighted mean safety effect will be informative. If the data, when plotted, reveal more than one peak, then it is usually more informative to determine the mean effect for each mode of the data, not the overall mean effect. When more than one peak exist in the data, then it can be argued that one or more studies differed significantly from the remaining data sets and the data should not be grouped together to determine a mean safety effect. For example, the findings from one country may differ significantly from another country where the same treatment was tested. In this case, the two data sets should be analyzed independently.

The sensitivity of the overall mean effect of the treatment to the presence of outliers can also be tested. Each result is removed from the calculation of the mean effect and compared to the mean effect that was calculated using all of the study findings. This test helps to identify any studies that have findings significantly greater or smaller than the bulk of the findings.

There is often a concern in evaluation studies that only favorable findings are published, or those studies which resulted in statistically significant improvements in operations or safety performance. The test for publication bias can also be performed on the studies chosen for the meta-analysis study by evaluating the shape of the funnel graph. If only favorable results have been published, one of the tails of the funnel graph will be missing. If, on the other hand, only statistically significant improvements have been reported, then the funnel graph will have a hollow core.

When multiple results from one study are included in the mean effect calculations, the jack-knife technique can be used to assess their impact on the mean effect of all studies included in the meta-analysis. The jack-knife technique helps to refine the standard error associated with the weighted mean, accounting for the correlation between multiple results from one study. Applying the jack-knife technique to meta-analysis involves basing the analysis on the weighted mean results within each study instead of using the original, multiple results of each study. While the value of the weighted mean does not change, a larger standard error term results from application of the jack-knife technique.

Several of the techniques described above have been applied to the automated red-light enforcement programs identified in the literature review conducted for this study. They are further described and applied in section 4 of this paper. Before launching into the review of automated red-light enforcement programs, however, a short review of other meta-analysis studies is provided to demonstrate the usefulness of this technique to transportation evaluation studies.

2.2 Example Meta-Analysis Studies

Rune Elvik has published several studies that apply the previously described tests and methods to evaluation studies within the transportation field in an attempt to determine mean safety effects for various treatments (Elvik, 1999). In a study evaluating the value of guardrails and crash cushions on safety performance, thirty-two studies were compiled that resulted in 232 estimates of safety effects. The studies were conducted in several countries including the United States, Australia, and Great Britain, and the publication dates of the studies spanned over thirty-five years. Using meta-analysis techniques, Elvik was able to demonstrate that median barriers were found to increase accident rates, by approximately 30%, but actually reduced accident severity (a 20% reduction in fatal injuries given an accident occurred, and a 10% reduction in sustaining personal injury given an accident occurred). The presence of guardrails were found to reduce the chance of sustaining a fatal injury by about 45% given that an accident had occurred, and the chance of sustaining a personal injury was reduced about 50% given an accident had occurred.

Another study investigated the effect of public lighting on safety performance of transportation facilities (Elvik, 1999). Thirty-seven studies were identified that evaluated the effect of public lighting on roadway safety performance. The thirty-seven studies included 142 results from eleven countries. The publication dates of the studies spanned forty years beginning in 1948. The meta-analysis study found that roadway lighting reduces nighttime fatal accidents by 65%, nighttime injury accidents by 30%, and nighttime property-damage-only accidents by 15%.

These studies, and others performed outside of the transportation field, demonstrate how the findings from several studies conducted in various locations and during various time periods, can be quantitatively summarized to estimate a mean effect. The goal of this study was to use the meta-analysis techniques and methods on red light camera enforcement studies currently published or deemed to be acceptable (according to criteria discussed in the next section), in order to determine a mean overall safety effect.

3.0 Literature for Automated Red Light Enforcement Safety Studies

The literature review conducted for this study focused on documented safety impact data for automated enforcement systems at signalized intersections. The first sources were gleaned from the ITS Benefits database sponsored by the U.S. DOT (www.benefitcost.its.dot.gov). The online research databases sponsored by the Transportation Research Board also provided numerous references in this search, as did conversations with traffic engineers in localities operating the programs and the authors of some of the previous reports on automated enforcement. This initial search resulted in approximately 75 documents that describe the impact of various automated traffic signal enforcement programs. One of the major hurdles to overcome in assessing the safety impact of automated red light enforcement programs is the limited availability of before/after crash and violation data. Many systems have only recently been installed, limiting the duration of crash experience after implementation. In many cases, violation data are only available for the after period of performance.

The initial set of documents was reviewed with regard to three criteria. Each of the criteria needed to be met in order for the data source to be included in the meta-analysis. The criteria applied to the data included:

· Safety performance data needed to be reported at the intersection level

· A significant period of performance needed to be reported

· Crash data needed to be reported in terms of frequency of crashes

Unfortunately, as discussed below, only two of the studies identified in the literature had data that met all three criteria. Seven contained detailed safety data. Conversations with transportation engineers in Howard County, MD, and Charlotte, NC yielded more detailed data from those two locations. The authors considered data from each of the six localities covered by the identified reports for inclusion in the meta-analysis: Oxnard, CA (Retting, et al., 1999; Retting and Kyrychenko, 2001), Fairfax, VA (Retting et al., August 1999), Glasgow, Scotland (The Scottish Office Central Research Unit, 1995), Mesa, AZ (Vinzant and Tatro, 1999), Howard County, MD (“Maryland House of Delegates…”, 2001), and Charlotte, NC (Charlotte Department of Transportation, 2000).

Two of these seven studies were later dropped from the analysis because they did not contain intersection level data, including the studies by Retting and Kyrychenko and a study from Glasgow, Scotland. The study by Retting and Kyrychenko in Oxnard, California (Retting and Kyrychenko, 2001) reviewed changes in citywide crash statistics coinciding with the implementation of a camera program. The lack of intersection level detail in the Oxnard study precluded the inclusion of it in the meta-analysis. The study conducted in Glasgow included a detailed discussion of impacts at several locations throughout the city (The Scottish Office Central Research Unit, 1995). Unfortunately, impact data was presented in a summarized format for all sites and discussions with the authoring company revealed that the intersection level data had been destroyed at some time during the six years since the completion of the study.

The length of the before and after periods was the second criteria used to determine if a study could be included in the meta-analysis. The shortest observation period for the selected studies was one year and four months of before and after crash data, while the longest observation period was three years before and after installation of the automated enforcement program. The average length of the observation periods for included data was approximately two years seven months. Periods shorter than twelve months were not considered in the analysis.

Observation periods for the identified violation studies ranged in value from approximately 22 hours in the before and after periods to over 200 hours in the before and after periods. Due to the inconsistent length of observation periods and the large variation in study periods between the two studies, the violation studies were dropped from the meta-analysis.

A third criterion for inclusion in the study was the availability of data from consistent measures of effectiveness for safety. For the crash analysis, crash frequency data was necessary for each observation period. The Mesa, AZ study (Vinzant and Tatro, 1999) cited changes in crash rates during each period. Communication with the organizations involved in the study did not yield the corresponding volume data necessary to convert these rates to frequencies so the data could not be included in the meta-analysis. The most common measure of effectiveness for the impact on violations was the rate of violations per 10,000 vehicles measured in the Retting studies in Oxnard, California, and Fairfax, Virginia (Retting, et al., 1999; Retting, et al., August 1999). The Glasgow study (The Scottish Office Central Research Unit, 1995) measured the change in the rate of vehicles violating among those that had the opportunity to infringe, by arriving when they had a clear path to the signal.

Only two studies met the criteria previously described. Crash data from Howard County, MD and Charlotte, NC were included in the meta-analysis study. The crash data from Howard County was published in a review for the Maryland House of Delegates in early 2001. Conversations with a transportation engineer in Howard County revealed that the data presented were police reported crashes at the enforced intersections in Howard County (“Maryland House of Delegates…”, 2001). An annual report on the program in Charlotte, NC contained data on crash reductions for several of twenty enforced intersections in that city (Charlotte Department of Transportation, 2000). Upon request, the city provided crash frequencies based on police reports for the seventeen intersections that still have operational cameras. Both Charlotte and Howard County provided police-reported right-angle and rear-end crashes at the enforced locations. The meta-analysis conducted in this study investigated the impact of the camera programs on the total of both of these crash types.

Tables 1 and 2 present the data included in both the violation and crash analyses collected as part of the literature review. The tables include the location, before and after violation rates or crash frequencies, the percent change in the measure before and after camera implementation, and the length of the observation periods. It should be noted that only fourteen data points were identified for before/after violations studies.

Table 1. Crash Frequency Data included in Meta-Analysis

Location	Crashes Before*	Crashes After*	Percent Change	Observation Period
Howard County, Maryland ("Maryland House of Delegates…", 2001)
Little Patuxent Parkway @ Columbia Rd.	45	30	-33	2 yr 10 mo B/A
NB Broken Land Parkway @ Stevens Forest Rd.	60	43	-28	2 yr 10 mo B/A
NB Broken Land Parkway @ Snowden River Pkwy.	50	38	-24	2 yr 9 mo B/A
SB Broken Land Parkway @ Snowden River Pkwy.	41	27	-34	2 yr 9 mo B/A
SB Broken Land Parkway @ Cradlerock North	34	23	-32	2 yr 9 mo B/A
SB Broken Land Parkway @ Stevens Forest Rd.	36	20	-44	2 yr 9 mo B/A
NB Cedar Lane @ Hickory Ridge Rd.	22	12	-36	2 yr 8 mo B/A
EB Governor Warfield @ Little Patuxent Pkwy.	39	30	-23	2 yr 8 mo B/A
NB Little Patuxent Pkwy @Governor Warfield	33	26	-21	2 yr 7 mo B/A
SB Little Patuxent Pkwy @Governor Warfield	31	22	-29	2 yr 5 mo B/A
SB Route 1 @ Guilford Rd	37	33	-40	2 yr 5 mo B/A
NB Route 1 @ Guilford Rd	31	23	-26	2 yr 5 mo B/A
SB Route 29 @ Rivers Edge	25	18	-28	2 yr 5 mo B/A
Cedar Lane @ Freetown Rd	20	14	-30	2 yr 5 mo B/A
Route 32 @ Route 144	26	16	-38	2 yr B/A
WB Route 40 @ Chatham Rd	23	15	-35	2 yr B/A
WB Route 40 @ Rogers Ave	43	32	-26	2 yr B/A
SB Route 29 @ Route 216	26	19	-27	2 yr B/A
SB Brokenland Pkwy @ Hickory Ridge	29	21	-28	2 yr B/A
EB Snowden River @ Oakland Mills	36	23	-36	1 yr 11 mo B/A
WB Snowden River Pkwy @ Brokenland Pkwy	32	21	-34	1 yr 10 mo B/A
EB Route 40 @ Rogers Ave	30	20	-33	1 yr 8 mo B/A
WB Snowden River Pkwy @ Oakland Mills Rd	19	14	-26	1 yr 6 mo B/A
WB Little Patuxent Pkwy @ Columbia Rd	14	9	-36	1 yr 6 mo B/A
EB Route 40 @ Marriottsville Rd	14	10	-28	1 yr 4 mo B/A
Charlotte, North Carolina (obtained from Charlotte DOT, 2001)
Beatties Ford Rd/Hoskins Rd	4	2	-50.00%	3 years B/A
Morehead St/College St	29	10	-65.52%	3 years B/A
Tyvola Rd/Wedgewood Dr	27	12	-55.56%	3 years B/A
Morehead St/McDowell St	18	10	-44.44%	3 years B/A
Brookshire Freeway/Hovis Rd	44	28	-36.36%	3 years B/A
11th St/Brevard St	26	16	-38.46%	3 years B/A
Arrowood Rd/Nations Ford Rd	9	14	55.56%	3 years B/A
N. Tryon St/Harris Blvd	43	46	6.98%	3 years B/A
South Blvd/Archdale Dr	25	29	16.00%	3 years B/A
Westinghouse Blvd/S. Tryon	23	11	-52.17%	3 years B/A
Poplar St/ 4th St	24	20	-16.67%	3 years B/A
Albemarle Rd @ Harris Bv	61	34	-44.26%	3 years B/A
Sharon Amity Rd @ Central Av	32	43	34.38%	3 years B/A
Eastway Dr @ Kilborne Dr	25	27	8.00%	3 years B/A
Fairview Rd @ Sharon Rd	27	28	3.70%	3 years B/A
Idlewild Rd @ Independence Bv	33	25	-24.24%	3 years B/A
Randolph Rd @ Sharon Amity Rd	18	12	-33.33%	3 years B/A

*Crash frequencies are total of rear end and angle accidents on camera approaches.

Table 2. Violation data gathered during study.

Location	Before Violations	After Violations	Violations per 10,000 Before	Violations per 10,000 After	Percent Change	Observation Period (B/A)
Oxnard, California (Retting, et al., 1999)
Channel Isl/C St	69	27	6.5	2.5	-62	216h/241h
Channel Isl/Rose	81	34	17.5	7.7	-56	174h/174h
Harbor/Wooley	53	36	19.8	13.9	-30	120h/122h
Rice/Sturgis	106	49	19	9.8	-48	142h/142h
Rose/Wooley	117	87	17.4	13.6	-22	176h/176h
Saviers/Pleasant Vly	32	16	17.6	8.9	-49	158h/164h
Ventura/Bay	66	29	7.3	3.1	-58	166h/166h
Ventura/Doris	153	152	14.7	10.3	-30	167h/236h
Ventura/Vineyard	42	25	9.6	5.6	-42	169h/169h
Fairfax, Virginia (Retting, et al., August 1999)
Fairfax Circle	99	78	51.8	37.7	-27	*22.5/23.4h
Main/University	44	21	24.2	10.2	-58	*22.5/23.4h
123/Eaton	78	37	33	17.7	-46	*22.5/23.4h
123/North	94	38	56.1	22.3	-60	*22.5/23.4h
Lee Hwy/123	21	18	14.1	12	-15	*22.5/23.4h

*Average hours per site

4.0 Meta-Analysis of Automated Red-Light Enforcement Safety Performance Data

As described in section 3.0, a limited number of safety performance studies met the criteria for inclusion in this meta-analysis study. Because of this limitation, several techniques and methods described in section 2.1 were not conducted. The tests that were performed include:

· Testing skewness of the sample of results

· Testing the modality of the distribution of results

· Testing the sensitivity of the mean to outlying data points

Tests that were not performed include:

· Testing for heterogeneity in a sample of results

· Testing for the possible presence of publication bias

· Assessing the uncertainty of a weighted mean result by means of the Jack-Knife Technique

In order to determine if the set of identified crash data were highly skewed, meaning a large number of points lie to the left or right of the mean value, a funnel graph was developed. The vertical axis of the graph represents the statistical weight of each value taken from the acceptable studies. The statistical weight was calculated using the following equation:

W_i = 1/(1/B_i + 1/A_i)

Where,

B_i = the number of accidents at the subject site before installation of red light cameras

A_i = the number of accidents at the subject site after installation of red light cameras

Figure 1 contains a scatter plot of the crash data. The horizontal axis of the graph represents the effect of automated enforcement programs on the probability of a rear end or right angle accident occurring. If no change occurred between the before and after periods, the effect would be equal to 1.00. If there was a positive change, meaning a reduction in the probability of a right-angle or rear-end accident occurring, the effect would be <1.00. However, if the automated enforcement program was found to have a negative effect on the number of right-angle and rear-end crashes, the effect would be >1.00.

Figure 1. Funnel graph of crash data at intersections with camera enforcement

Three points of interest for testing the crash data for skewness are highlighted on Figure 1: the mean, mode and median of the data set. If the results are not highly skewed, these three values should not differ significantly from each other. If however, the data are skewed to the left, meaning an abnormally large reduction in crashes was found in one or two data sets, then the mode will be smaller than the median, and the median will be smaller than the mean. If instead, the data were found to be skewed to the right, the converse would be true. In Figure 1, the mean, mode and median are 0.735, 0.72, and 0.68, respectively. This means that the probability of a right-angle or rear-end crash occurring at the study intersection, was decreased by approximately 26%. The small variation between the three measures also gives additional confidence in the data, by demonstrating only a small skew of the data.

Also from Figure 1, the test for modality can be performed. This test investigates the ability of the data to provide a meaningful estimate of the mean effect of automated enforcement on reducing rear-end and right-angle accidents. If the data form a single funnel shape, then the estimated mean safety effect will be informative. The funnel shape also demonstrates that the observed impacts from the two studies converge around the mean as the sample size increases (Elvik, 1999). From Figure 1, one can visualize the upside down funnel shape, with the narrow part of the funnel encompassing the data points near the mean, mode and median measures. This test, like the first test, gives additional merit to the ability of the data to provide a meaningful estimate of the effectiveness of enforcement cameras in reducing rear-end and right-angle crashes.

The last test performed on the identified set of accident data was the test for outliers. This test seeks to identify any bias that has been introduced in the estimate of the mean effect by the presence of a single data point that is significantly affecting the mean. For this test, each data point is removed from the calculation of the mean one point at a time. If the mean is significantly changed with the removal of any of the individual data points, then the point can be identified as an outlier. Figure 2 shows the upper and lower 95% confidence limits for the mean effect of automated enforcement on reducing rear-end and right angle crashes for the forty-two data points included in the study. As is demonstrated in Figure 2, all of the estimates of the mean effect of automated enforcement on reducing rear-end and right-angle crashes based on forty-one data points were within the 95% confidence limits of all the estimates. This demonstrates that the mean is very stable and hardly affected by the removal of any of the individual data points.

Figure 2. Test for impact of outliers on weighted mean of crash data

4.1 Observations from Meta-Analysis Study

The tests performed in this study were limited due to the small number of identified studies that reported useable crash frequency data in the before and after periods. Even so, the tests that were conducted demonstrated that the available data appear to be consistent. The mean, mode, and median effect of automated enforcement cameras on reducing right-angle and rear-end crashes were found to be very similar (around 26%), meaning the probability of this type of accident occurring in the after period was reduced by approximately 26% (for the studies included in the analysis). The funnel graph shows that the data are able to provide a meaningful estimate of the effect of automated enforcement on reducing right-angle and rear-end crashes. And finally, the test for outliers demonstrated the stableness of the mean, in that it was not significantly changed by one individual data point.

The weakness of this study lies in the limited number of studies that were identified to be included in the study. Due to the limits of the available data, several of the meta-analysis techniques could not be applied. Given this limitation, the results should not be emphasized and caution should be exercised when reviewing this study and applying the results. Rather, the value of this research effort is the demonstration of meta-analysis methods applied to an ITS application area to assess the impact of an application. Additional data are needed to fully determine the effect of automated red light enforcement cameras on safety at signalized intersections in the U.S. It should also be noted that this study did not look at possible spill-over effects at neighboring intersections, as only the intersections with the enforcement equipment were included in this study.

There were several lessons learned through this effort. One of the challenges to conducting meta-analysis techniques is working with the various formats in which data and information are presented in studies. For instance, the Scotland study had collected three years of before and after crash data at six intersections installed with automated enforcement equipment, but when the findings were reported, the authors chose to aggregate the findings into a summary statement regarding the safety improvements. This type of presentation made the data unusable for this study.

Another indicator of the impact of automated enforcement systems on safety at signalized intersections is the change in the number of red light violations. Unfortunately, many studies reported the number of violations recorded in the after period but fail to report or study the number of violations in the before period. There is also an unresolved issue surrounding the number of hours or days of observation of driver behavior required in the before and after periods to be considered significant. Also, what is the best way to collect “before” violation data? These questions need to be resolved before an in-depth study to determine the effect of automated enforcement programs on reducing red light running violations at signalized intersections can succeed.

5.0 Directions for Future Study

As described previously, the results of this meta-analysis are of uncertain validity due to the small number of studies incorporated in the assessment of each measure of effectiveness. The studies by Elvik on guardrails (Elvik, 1999) and public lighting (Elvik, 1999) made use of 232 and 142 estimates of safety impact, respectively. In both cases the estimates were drawn from over thirty separate studies. In contrast, this meta-analysis only had 42 estimates of safety impact from two studies. As experience with the operation of red light enforcement programs grows in the U.S., continued study and gathering of the impacts of individual programs will be needed in order to increase the validity of any future meta-analysis.

Another research question with broad implications is the validity of applying meta-analysis techniques to informally gathered data sets in lieu of a strict reliance on published reports. In particular, the crash data used in this analysis, while closely related to published studies, was not directly incorporated in the published documents but rather gathered through additional communication with the agencies involved. A useful study would investigate how dependence on this type of data affected the applicability of several of the techniques used. For example, the test used for publication bias may retain value in assessing any impact of reporting inaccuracies from selected sites. Investigation in these areas would help further the understanding of both the impact of red light camera enforcement programs and the applicability of meta-analysis statistical techniques in transportation safety research.

It should be noted that a new study investigating the impact of automated enforcement systems used to reduce red light running violations at signalized intersections is currently underway. The Federal Highway Administration is sponsoring a study that will seek to gather crash and violation data from several locations in the U.S. This new study will not be limited to data published in journal articles and should enable the authors to identify additional data sets that can be studied more thoroughly. The meta-analysis techniques described in this paper should be revisited once additional data are available for incorporation. The same techniques can also be applied to assess the impact of camera enforcement programs on individual categories of crashes (for example rear-end, right-angle, and pedestrian crashes). In addition, with sufficient data, meta-analysis could be applied to intersections without the enforcement cameras within the boundaries of jurisdictions with automated enforcement programs, to determine if reductions in accidents or violations apply beyond the individual intersections that are equipped.

References

Charlotte Department of Transportation. “SafeLight Charlotte: Annual Report: August 1999 – July 2000.” 2000. http://www.charmeck.nc.us/citransportation/programs/safelight/report2.htm

Elvik, Rune. “Assessing the Validity of Evaluation Research by Means of Meta-Analysis”. Institute of Transport Economics, Norwegian Centre for Transport Research, Oslo, May 1999.

Elvik, Rune, Amundsen Finn, Hofset, Frode. “Road Safety Effects of Bypasses”. 80^th Annual Transportation Research Board Meeting, January 2001 Proceedings.

Federal Highway Administration (FHWA). Synthesis and Evaluation of Red Light Running Automated Enforcement Programs in the United States. FHWA Report [FHWA-IF-00-004]. September 1999.

Insurance Institute for Highway Safety (IIHS). “Red Light Cameras In Action.” November 2001. <http://www.hwysafety.org/safety%5Ffacts/rlc%5Fcities.htm>

Institute of Transportation Engineers. Automated Enforcement in Transportation. ITE Informational Report. December 1999.

“Maryland House of Delegates Commerce and Government Matters Committee Automated Enforcement Review: Red-Light Running Detection Camera Systems.” Howard County, MD. January 18, 2001.

Passetti, K. “Use of Automated Enforcement for Red Light Violations.” August 1997, CVEN-677 Advanced Surface Transportation Systems, Department of Civil Engineering, Texas A&M University, College Station, Texas.

Retting, Richard A. and Sergey Y. Kyrychenko. Crash Reductions Associated with Red Light Camera Enforcement in Oxnard, California. Insurance Institute for Highway Safety Report. Arlington, VA: April 2001.

Retting, Richard A., et al. “Evaluation of Red Light Camera Enforcement in Fairfax, Va., USA.” ITE Journal. August 1999. pp. 30-34

Retting, Richard A., et al. “Evaluation of red light camera enforcement in Oxnard, California.” Accident Analysis and Prevention 31. 1999. pp. 169-174.

Rocchi, Sarah, and Suzanne Hemsing. “A Review of the Road Safety Benefits of Red Light Cameras.” ITE International Conference. Kissimmee, Florida, 1999.

The Scottish Office, Central Research Unit. “Running the Red: An Evaluation of Strathclyde Police’s Red Light Camera Initiative.” 1995.

Smith, David M. et al. “Automated Enforcement of Red Light Running Technology and Programs: A Review.” Transportation Research Record 1734. 2000. pp. 29-36

Vinzant, Janet C. and B.J. Tatro. “Evaluation of the Effects of Photo Radar Speed and Red Light Camera Technologies on Motor Vehicle Crash Rates.” Prepared for the City of Mesa Police Department. March 1, 1999.

Zaal, Dominic. Traffic Law Enforcement: A Review of the Literature. Monash University Report (No. 53). Canberra, Australia: April 1994.