FMCSA Safety Program Effectiveness Measurement: Intervention Model

Roadside Inspection and Traffic Enforcement Effectiveness Annual Report

Prepared For:

Federal Motor Carrier Safety Administration

Office of Information Management,

Analysis Division

400 Seventh Street, S.W.

Washington, DC 20590

Prepared By:

John A. Volpe National Transportation Systems Center

Office of System and Economic Assessment

Motor Carrier Safety Assessment Division, DTS-47

Kendall Square

55 Broadway

Cambridge, MA 02142

Preface

This report documents the methodology and results from an improved model to measure the effectiveness of two of the key safety programs of the Federal Motor Carrier Safety Administration (FMCSA). The research was conducted by the Research and Special Programs Administration's (RSPA) John A. Volpe National Transportation Systems Center (the Volpe Center) in Cambridge, MA under a project plan agreement with the FMCSA. The work on FMCSA Program Performance Measures addresses the requirements of the Government Performance and Results Act (GPRA) of 1993, which obligates federal agencies to measure the effectiveness of their programs as part of the budget cycle process.

Work on FMCSA Program Performance Measures was initiated during FY 93. In December 1994, a report titled "Office of Motor Carriers Safety Program - Performance Measurement" was prepared. That report provided a comprehensive breakdown of Office of Motor Carriers (OMC) safety programs and activities and described about a dozen potential evaluation models. (Note: The OMC later became the FMCSA.) Based on the OMC's review, the Volpe Center revised the report and recommended four evaluation models to assess the key OMC programs: roadside inspections conducted by participating states under the Motor Carrier Safety Assistance Program (MCSAP), on-site compliance reviews conducted by the OMC field offices and the states, commercial vehicle traffic enforcement also performed by the states under the MCSAP, and a comprehensive assessment of combined effects. Two initial evaluation models covering the roadside inspection program and the compliance review program were described in detail in a December 1998 report titled "OMC Safety Program Performance Measures." A review panel was convened to evaluate these models and made recommendations for improvement. The Volpe Center incorporated these recommendations together with other Volpe Center defined improvements into two "second-generation" models that measure the effectiveness of these two programs. This report describes the implementation of the Intervention Model, which covers not only the roadside inspection program, but also the traffic enforcement program.

At the FMCSA, the project is managed by Dale Sienicki, Division Chief of the Analysis Division in the Office of Information Management. The Volpe Center project manager is Donald Wright, Chief of the Motor Carrier Safety Assessment Division in the Office of System and Economic Assessment. The analysis was performed at the Volpe Center by Kevin Gay, Nancy Kennedy, and Julie Nixon of the Volpe Center, and Dennis Piccolo and Kha Nguyen of EG&G Services, under contract to the Volpe Center.

Acronym List


Acronym	Full Name
CDL	Commercial Driver's License
CMV	Commercial Motor Vehicle
CVSA	Commercial Vehicle Safety Alliance
FARS	Fatality Analysis Reporting System
FMCSA	Federal Motor Carrier Safety Administration
GPRA	Government Performance and Results Act
HM	Hazardous Materials
MCMIS	Motor Carrier Management Information System
MCSAP	Motor Carrier Safety Assistance Program
NAS	North American Standard
NHTSA	National Highway Traffic Safety Administration
OMC	Office of Motor Carriers
OOS	Out of Service
RI	Roadside Inspection
RSPA	Research and Special Programs Administration
TE	Traffic Enforcement
U.S. DOT	United States Department of Transportation

List of Tables

Table ES-1. Roadside Inspection Benefits 1998 - 2003

Table ES-2. Traffic Enforcement Benefits 1998 - 2003

Table ES-3. Total Benefits 1998 - 2003

Table ES-4. Intervention Breakdown

Table 1. Relative Weights for Driver and Vehicle Categories

Table 2. Roadside Inspection Benefits 1998 - 2003

Table 3. Traffic Enforcement Benefits 1998 - 2003

Table 4. Total Benefits 1998 - 2003

Table 5. Intervention Breakdown

Table 6. Lower Bound of Number of Violations to Avoid One Crash

Table 7. Higher Bound of Number of Violations to Avoid One Crash

Table 8. Lower Bound Crash Reduction Probabilities

Table 9. Higher Bound Crash Reduction Probabilities

Table 10. Violations for Intervention A

Table 11. Violations for Intervention B

Table 12. Violation Occurrences per Risk Category

Table 13. Roadside Inspection Program Benefits 1998 - 2000

Table 14. Traffic Enforcement Program Benefits 1998 - 2000

Table 15. Indirect Benefits as Percentage of Total

Table 16. Crash Severity Shares

Table 17. Two Year Average of Crash Severity Shares

Table 18. Average Numbers of Fatalities and Injuries by Year

Table 19. Two Year Average of Fatalities and Injuries

Table 20. Example Program Benefits

Table 21. Roadside Inspection Category 1 Crash Reduction Probabilities

Table 22. Roadside Inspection Category 1 Violations

Table 23. Roadside Inspection Category 2 Crash Reduction Probabilities Table 24. Roadside Inspection Category 2 Violations 44

Table 25. Roadside Inspection Category 3 Crash Reduction Probabilities

Table 26. Roadside Inspection Category 3 Violations

Table 27. Roadside Inspection Category 4 Crash Reduction Probabilities

Table 28. Roadside Inspection Category 4 Violations

Table 29. Roadside Inspection Category 5 Crash Reduction Probabilities

Table 30. Roadside Inspection Category 5 Violations

Table 31. Traffic Enforcement Category 1 Crash Reduction Probabilities

Table 32. Traffic Enforcement Category 1 Violations

Table 33. Traffic Enforcement Category 2 Crash Reduction Probabilities

Table 34. Traffic Enforcement Category 2 Violations

Table 35. Traffic Enforcement Category 3 Crash Reduction Probabilities

Table 36. Traffic Enforcement Category 3 Violations

Table 37. Traffic Enforcement Category 4 Crash Reduction Probabilities

Table 38. Traffic Enforcement Category 4 Violations

Table 39. Traffic Enforcement Category 5 Crash Reduction Probabilities

Table 40. Traffic Enforcement Category 5 Violations

List of Figures

Figure ES-1. Analysis of Interventions by Type

Figure 1. Intervention Model Overview

Figure 2. Violation Crash Risk Profile

Figure 3. Direct Effect Approach

Figure 4. Indirect Effects Approach

Figure 5. Program Benefits Determination

Figure 6. Analysis of Interventions by Type

Figure 7. 2001 State Level TE & RI Program Benefits

Figure 8. 2001 State Level Roadside Inspection Program Benefits

Figure 9. 2001 State Level Traffic Enforcement Program Benefits

Figure 10. 2002 State Level TE & RI Program Benefits

Figure 11. 2002 State Level Roadside Inspection Program Benefits

Figure 12. 2002 State Level Traffic Enforcement Program Benefits

Figure 13. 2003 State Level TE & RI Program Benefits

Figure 14. 2003 State Level Roadside Inspection Program Benefits

Figure 15. 2003 State Level Traffic Enforcement Program Benefits

Figure 16. Direct-Effect Approach with Roadside Allowance

Figure 17. Indirect Effect Approach with Roadside Allocation

Figure 18. Indirect Benefits as a Percentage of the Total Benefits

Figure 19. Program Benefits Determination

Executive Summary

Brief Description

The Intervention Model is designed to provide the Federal Motor Carrier Safety Administration (FMCSA) with a means to gauge the effectiveness of two of its safety programs - roadside inspections and traffic enforcements - in preventing crashes involving interstate motor carriers and in reducing related fatalities and injuries. The model is also intended to be a tool that the FMCSA can use periodically to measure the relative performance of its programs, and to analyze the effects of implementing program changes.

The model measures program effectiveness in terms of safety, commercial vehicle crashes prevented, lives saved, and injuries avoided. Although the methodology is believed to be sound and roadside inspection results are judged to be complete and accurate, the model has known limitations. It lacks empirical data regarding driver behavior and the contribution that vehicle defects and driver faults have on crash causation. In lieu of empirical data, the model defaults to other means (including expert judgment) and establishes a benchmark to measure roadside inspection and traffic enforcement program effectiveness.

The model is based on the premise that the two programs - roadside inspection and traffic enforcement - directly and indirectly contribute to a reduction in crashes. As a result, the model includes two submodels that are used for measuring these different effects. Direct effects are based on the assumption that vehicle and/or driver defects discovered and then corrected as the results of interventions reduce the probability that these vehicles/drivers will be involved in subsequent crashes. The model calculates direct-effect-prevented crashes according to the number and type of violations detected and corrected during an intervention.

Indirect effects are considered to be the by-products of the carriers' increased awareness of FMCSA programs and the potential consequences that these programs pose if steps are not taken to ensure and/or maintain higher levels of safety. This change in behavior will result in higher levels of compliance, fewer future violations, and therefore, a reduction in the number of crashes.

Critical to the model is its ability to link vehicle and driver defects detected during roadside inspections and/or traffic enforcements to crash probabilities. Currently available research and expert judgments provided the basis for establishing these linkages and assigning probabilities. Since there is little in the way of empirical data to support these probabilities, the values developed are intentionally conservative so as not to overstate the safety benefits of the programs.

Major investigations focusing on special large truck crash data collections and crash reconstruction analysis are currently being sponsored by the FMCSA will assist in improving crash probabilities. The model's methodology will enable the incorporation of the results of these efforts once they become available.

This model, which measures the effectiveness of the roadside inspection and traffic enforcement programs, when combined with the Compliance Review Effectiveness Model, forms a powerful performance measurement capability that will facilitate a combined-effects assessment of the three FMCSA safety programs. The expectation is that the combined-effects assessment results will further guide FMCSA decision-making when directing resources to achieve optimal program effectiveness.

Enhancements

With each implementation of the model, it is important to identify any modifications to the methodology of the Intervention Model that might impact any comparisons between historical results (1998 - 2000) and the new results (2001 - 2003). There has been one modification to the model since the last implementation, and the belief is that it will not adversely affect any comparisons or analysis with historical data.

Indirect Effects

The model enhancement is the ability to compute indirect effects for a particular year without having to wait for the subsequent year of data. Prior to the enhancement, intervention data from 2003 and 2004 was required to compute the indirect effects for 2003, which implies that results could not be computed until the middle of 2005. After a detailed analysis of the 1998 - 2000 results of the indirect component of the model, an enhancement was recommended and approved that allows results to be published in the year following the intervention data (i.e. in 2004 for 2003 data). The details of this analysis and subsequent enhancement are covered in See Indirect-Effect Approach.

Program Benefits

The model was implemented to estimate three years of safety benefits (2001, 2002, and 2003). The 2001 - 2002 safety benefits are based on data current through March of 2004, while the 2003 safety benefits are based on data current through June of 2004.

National Level

The national level program safety benefits, which are crashes avoided, lives saved, and injuries avoided, for 2001 through 2003 are displayed in See Roadside Inspection Benefits 1998 - 2003, See Traffic Enforcement Benefits 1998 - 2003, and See Total Benefits 1998 - 2003. These tables also present the historical results (from previous implementations of the model 1998 - 2000) in order to provide additional data for comparison. These results were taken directly from the September 2002 report "Intervention Model: Roadside Inspection and Traffic Enforcement Effectiveness Assessment."

Roadside Inspection Benefits 1998 - 2003
	1998	1999	2000	2001	2002	2003
Crashes Avoided	8,612	9,119	9,362	11,294	12,235	12,667
Lives Saved	369	391	402	550	568	534
Injuries Avoided	5,902	6,250	6,416	8,689	9,240	9,647

Traffic Enforcement Benefits 1998 - 2003
	1998	1999	2000	2001	2002	2003
Crashes Avoided	2,800	3,021	3,306	3,844	4,602	4,484
Lives Saved	120	130	142	187	214	189
Injuries Avoided	1,919	2,071	2,265	2,957	3,476	3,415

Total Benefits 1998 - 20031
	1998	1999	2000	2001	2002	2003
Crashes Avoided	11,412	12,140	12,688	15,138	16,837	17,151
Lives Saved	489	521	544	738	781	722
Injuries Avoided	7,821	8,321	8,681	11,646	12,716	13,062

It should be noted that the program benefits have increased every year since 1998 with the largest jump in benefits occurring from 2000 to 2001. In 2003, the number of lives saved is actually lower than the 2002 number even though more crashes were and injuries were avoided in 2003. This is a result of a change in the crash severity statistics from 2002 to 2003. For the 2002 results, 4.0% was used for the share of crashes that were fatal, but in 2003 this number dropped to 3.6%, which leads to a smaller number of lives saved. These values were calculated from the Motor Carrier Management Information System (MCMIS) and the Fatality Analysis Reporting System (FARS) data, and a full discussion of the methodology can be found in See Program Benefits.

Additionally, it is useful to analyze the number of interventions in each of the years. See Intervention Breakdown and See Analysis of Interventions by Type provide tabular and graphical breakdowns of the number of interventions per year by type of intervention.

Intervention Breakdown
	1998	1999	2000	2001	2002	2003
Roadside Inspections with No Violations	571,731	621,962	651,949	758,297	849,422	828,195
Roadside Inspections with Violation(s)	1,128,791	1,161,786	1,181,039	1,292,489	1,406,499	1,387,567
Traffic Enforcements with Violation(s)	516,048	579,219	620,226	695,619	762,561	791,157
Total Interventions	2,216,570	2,362,967	2,453,214	2,746,405	3,018,482	3,006,919

By analyzing the intervention breakdown it is clear that the increase in program benefits is at least partly due to the increase in the number of interventions performed. The number of interventions performed per year increased continually until 2003, with a significant jump from the year 2000 to 2001. Almost 300,000 more interventions were performed in 2001 than in 2000. It appears the trend has leveled off with the 2003 data being virtually identical to the 2002 data.

Analysis of Interventions by Type

Intervention Model

Introduction

Project Background

During the 1980s, Congress passed several acts intended to strengthen motor carrier safety regulations. This led to the implementation of safety-oriented programs both at the federal and state levels. The Surface Transportation Assistance Act of 1982 established the Motor Carrier Safety Assistance Program (MCSAP), a grants-in-aid program to states, to conduct roadside inspection and traffic enforcement programs aimed at commercial motor vehicles. The 1984 Motor Carrier Safety Act directed the U.S. Department of Transportation (U.S. DOT) to establish safety fitness standards for carriers. The U.S. DOT, along with the states, responded by implementing the MCSAP to fund roadside inspection and traffic enforcement programs, and the safety fitness determination process and rating system (based on on-site safety audits called compliance reviews).

It is expected that a major benefit of these programs has been and will continue to be an improved level of safety in the operation of commercial motor vehicles. Previously, however, there was no means to measure the benefits and effectiveness of these programs. The Safety Program Effectiveness Measurement Project was established to identify major functions and operations (programs) associated with the FMCSA mission and to develop results-oriented performance measures for those functions and operations as called for in the Government Performance and Results Act (GPRA) of 1993.

Project Objective

Program evaluation should be viewed as a continuous management process that encourages the organization to reflect periodically upon how it is implementing its programs. Program effectiveness should be reassessed in light of the mission, available resources, changing requirements, political climate, technological change, public demands, and costs. Periodic review of the results of the evaluations will ensure that the activities are working, i.e., that they are delivering what was promised. This report is intended to satisfy the desire of the FMCSA to verify the effectiveness of two of its motor carrier safety programs, the roadside inspection and traffic enforcement programs. The immediate objective of this effort is to measure how much of an impact the safety program activities have on avoiding crashes involving motor carriers and reducing resulting injuries and fatalities.

One of the main objectives of the Safety Program Effectiveness Measurement Project is to provide a baseline of the effectiveness of the selected programs through the use of standard safety performance measures. This baseline allows the FMCSA to judge the relative performance of its programs on a periodic basis by reflecting the benefits resulting from each program. The results of these analyses are intended to provide a basis for FMCSA resource allocation and budgeting decisions that will more closely optimize the effectiveness and efficiency of its motor carrier safety programs.

Project Scope

The scope of this overall effort is limited to the major identifiable FMCSA programs and their effectiveness in reducing crashes and avoiding injuries and fatalities. Currently the Safety Program Effectiveness Measurement Project includes the compliance review, roadside inspection, and traffic enforcement activities and programs performed and supported by the FMCSA. Two models have been developed to estimate the benefits of these programs: the Compliance Review Effectiveness Model and the Intervention Model (for roadside inspections and traffic enforcement). The benefits of these programs are calculated in terms of crashes avoided, lives saved, and injuries avoided.

An objective of the project is to continue to improve these models and run them on a recurring basis. The models will serve the program specific requirement to measure program effectiveness as well as the broader function of supporting annual budget requirements and helping to determine the best resource allocation among program elements.

This report describes the methodology of the Intervention Model and presents the final results from the implementation of the model for carriers receiving a roadside inspection or traffic enforcement in 2001, 2002, and 2003.

Report Structure

This report includes descriptions of the evolution of the Intervention Model, the effects that it measures, and how the model is to be applied. The report also explains concepts driving the development process and affecting the model structure. Report sections include:

Background on an earlier model, known as Safe-Miles, with an explanation of its limitations (See Background: Safe Miles),
An overview of the model with descriptions of the calculation of direct and indirect effects (See Methodology),
A discussion of future model enhancements (See Future Enhancements),
A mathematical description of the model (See Intervention Model Technical Documentation), and
Detailed information on the types and classification of violations critical to running the model (See Violations).

Model Description

Background: Safe Miles

Overview

The Safe-Miles Model was also developed to measure the effectiveness of the roadside inspection program and preceded the Intervention Model. It is discussed here by way of background, since the Intervention Model borrows substantially from the experience with the Safe-Miles Model. Included is a discussion of the direct and indirect effects approach first used in that model as well as the model's limitations leading to the development of the "second-generation" Intervention Model.

The Safe-Miles Model employed a two-step analysis process to perform the evaluation. Instances were recorded in which vehicles and/or drivers were taken out of service during roadside inspections. Next, subsequent travel by the out-of-service (OOS) vehicles and drivers, once conditions were corrected, was converted into "safe miles" and estimates were made concerning crashes avoided during the "safe-miles" period.

Direct-effect benefits were accumulated from the point at which vehicles or drivers with OOS conditions were detected and removed from service. A three-month "safe" post-inspection period for vehicles was incorporated into the model. This time frame was considered appropriate since the Commercial Vehicle Safety Alliance (CVSA) has a three-month period after a vehicle receives a satisfactory inspection that it is exempt from additional inspections.2 Lacking an empirical basis with which to govern the duration of the direct effect findings for drivers, the post-inspection safe period for corrected driver OOS defects was shortened to a more conservative period of two months.

Indirect effects are an equally important element of the roadside inspection program. The very existence of the program (as well as its magnitude) is believed to act as a deterrent. Knowledge of the program results in motor carrier managers making procedural changes that result in improvements in vehicle maintenance and inspection and in driver qualifications and behavior. These indirect effects, although assumed substantial, are much more difficult to quantify. The indirect effects are estimated in the Safe-Miles Model by assuming that carriers with a high frequency of (that is, greater exposure to) either vehicle or driver inspections, as a result of the enforcement of the roadside inspection program, change their behavior and voluntarily improve their safety, resulting in lower vehicle or driver OOS rates.

Direct effects (crashes avoided) were added to indirect effects to derive total roadside inspection program benefits. These benefits were also expressed as estimates in dollar terms by using crash cost factors. There was no traffic enforcement component in the Safe-Miles Model.

Limitations

The 1998 Volpe Center report - "OMC Safety Program Performance Measures" - identified the following limitations associated with the Safe-Miles Model:

No observed evidence existed for the establishment of a driver safe-miles period. In future empirical studies of driver behavior, post-OOS violation detection would be required to establish the reliability of the two-month interval that was used.
Each violation was considered in isolation. This precluded any heightening of the safety risk as a result of the presence of multiple violations found during an inspection.
The lack of crash causation statistics hindered the ability to estimate the contribution of specified vehicle and driver defects to crash likelihood.

The deterrence component of the model (indirect effects) relied on measured changes in OOS rates of carriers that had multiple inspections as a foundation for calculating indirect effects from roadside inspections. However, overall improved preparation and compliance of drivers and vehicles motivated by the presence of a roadside inspection program were thought to be greater than improvements that could be measured by the model.

The research team defined the Intervention Model as a means to remedy these limitations. As with the Safe-Miles Model, the Intervention Model includes direct and indirect effect components; however, it:

Eliminates the empirically weak "safe-miles" concept,
Makes allowances for inspections with multiple violations, and
Uses recent crash causation research to estimate the contribution of vehicle and driver faults to crash causation.

The model also considers total inspection results. This means that it includes non-OOS violations, although with a lesser-assigned weight, in its calculations. Finally, the Intervention Model remedies a Safe-Miles omission by including traffic enforcements in its analysis. The benefits of the Intervention Model are expressed as crashes, fatalities, and injuries avoided.

Methodology

The Intervention Model was developed to determine the effectiveness of the roadside inspection and traffic enforcement programs in reducing motor carrier crashes. The roadside inspection program consists of roadside inspections performed by qualified safety inspectors following the guidelines of the North American Standard, which was developed by the Commercial Vehicle Safety Alliance in cooperation with the FMCSA. Most roadside inspections are conducted by state personnel under a grant program (MCSAP) administered by the FMCSA. There are five levels of inspections including a vehicle component, a driver component or both. The traffic enforcement program is based on the enforcement of twenty-one moving violations noted in conjunction with a roadside inspection. Violations are included in the driver violation portion of the roadside inspection checklist.3 See Intervention Model Overview provides an overview of the Intervention Model.

Intervention Model Overview

As with the Safe-Miles Model, this model is based on the premise that the two programs - roadside inspection and traffic enforcement - directly and indirectly contribute to the reduction of crashes. As a result, the model includes two submodels that are used for measuring these different effects. Direct effects are based on the assumption that vehicle and/or driver defects discovered and then corrected as the results of interventions reduce the probability that these vehicles/drivers will be involved in subsequent crashes. Indirect effects are considered to be the by-products of the carriers' increased awareness of FMCSA programs and the potential consequences that these programs pose if steps are not taken to ensure and/or maintain high levels of safety.

Crash Risk Probabilities

In the model, the assumption is made that observed deficiencies (OOS and non-OOS violations) discovered at the time of roadside inspections and/or traffic enforcements can be converted into crash risk probabilities. This assumption is based on the premise that detected defects represent varying degrees of mechanical or judgmental faults, and, further, that some are more likely than others to play a contributory role in motor vehicle crashes. The assumption is that these deficiencies can be noted and ranked into discrete risk categories, each of which possesses a probability that reflects the crash risk that it poses. The process by which the resulting Violation Crash Risk Probability Profile (VCRPP) is formed appears in See Violation Crash Risk Profile.

Violation Crash Risk Profile

The development of risk categories for violations relied upon a recent study conducted by Cycla Corporation.4 Each violation was classified according to the risk caused by the conditions of the violation. Cycla's report defined risk as "the likelihood of a violation leading to a crash" and, subsequently, divided the violations into five categories based on the level of risk. The risk categories and their descriptions are as follows:

Risk Category 1 - The violation is the potential single, immediate factor leading to a crash or fatalities/injuries from a given crash.
Risk Category 2 - The violation is the potential single, eventual factor leading to a crash or fatalities/injuries from a given crash.
Risk Category 3 - The violation is a potential contributing factor leading to a crash or fatalities/injuries from a given crash.
Risk Category 4 - The violation is an unlikely potential contributing factor leading to a crash or fatalities/injuries from a given crash.
Risk Category 5 - The violation has little or no connection to crashes or the prevention of fatalities/injuries.

While covering most inspection violations, Cycla's assignment of violations to risk categories was incomplete. This required Volpe Center analysts to make violation assignments for those driver or vehicle violations not included in the Cycla risk assessment. These assignments were made based on comparability with the Cycla list.

In the Cycla study, recommended weights were given to each of the risk categories, as shown in See Relative Weights for Driver and Vehicle Categories. The heaviest weight (1,000) was assigned to Risk Category 1 since these violations are considered to represent a significant safety hazard. Risk Categories 2 through 5 were given lesser weights (100, 10, 1, and 0.1, respectively). Cycla justifies this by stating that since "each relative numerical weight represents a different order of magnitude of likelihood, the weights decrease by a factor of ten." The Cycla study cautions, however, that the values do not refer to any "absolute" risk level. (The detailed list of roadside inspection violations and traffic enforcement violations are separated into tables by risk categories in the section entitled See Violations. Each table indicates the source of the categorization - either Cycla or Volpe Center.)

To execute the model, Volpe Center analysts converted Cycla's relative numerical weights into crash reduction probabilities.5 Each probability is an estimate of the portion of a crash avoided when an inspection uncovers a particular violation. For example, if a violation carried a probability of 0.001, inspectors would have to discover that violation 1,000 times in order for the model to "take credit" for avoiding a crash. Since driver-related errors are thought to be more of a factor in crash causation relative to mechanical defects, traffic enforcement violations were assigned higher probabilities. Based on expert judgments formed from the results of previous studies and available data, traffic enforcement violations are considered 4 times more likely to result in a crash than roadside inspection violations.6

Relative Weights for Driver and Vehicle Categories7
Risk Category	Category Description	Relative Weight
1	Violation is the potential single, immediate factor leading to a crash or fatalities/injuries from a given crash.	1,000
2	Violation is the potential single, eventual factor leading to a crash or fatalities/injuries from a given crash.	100
3	Violation is a potential contributing factor leading to a crash or fatalities/injuries from a given crash.	10
4	Violation is an unlikely potential contributing factor leading to a crash or fatalities/injuries from a given crash.	1
5	Violation has little or no connection to crashes or the prevention of fatalities/injuries.	0.1

Direct Effects

This section describes the methodology employed to estimate the number of direct-effect crashes avoided.

Conceptually, the approach at the heart of the Direct Effects Submodel is straightforward. Since the occurrence of a single violation implies a certain degree of crash risk, each inspection that uncovers at least one violation can be interpreted as having reduced the risk linked with its noted violation(s). The model expresses this risk reduction in terms of the likelihood of a crash being avoided by each inspection violation that was noted and corrected. For an individual intervention, the avoided crash probability will be dependent upon the number and type of violations. Multiple violations, of course, will have a compounding effect, thereby increasing the likelihood of a prevented crash. By accounting separately for the two types of violations (roadside and traffic enforcement) and summing the portions of crashes avoided for all inspections within each group, it is possible to estimate direct-effect crashes that have been avoided due to the programs. See Direct Effect Approach depicts the process used to determine program direct effects.

Direct Effect Approach

Four steps make-up the direct-effect approach.

One year of inspection data is extracted from the Motor Carrier Management Information System (MCMIS) database. The MCMIS contains information compiled from federal and state safety agencies. Each intervention has its own set of associated driver and/or vehicle violations.
An inspection's violations are matched to the Violation Crash Risk Probability Profile, whereby a list of crash reduction probabilities becomes attached to that inspection. This list becomes the basis for calculating the inspection's effect on avoiding a crash.
The likelihood of an avoided crash for each inspection is calculated by using the crash reduction probabilities of the inspection. An inspection with multiple violations will have a greater likelihood of an avoided crash than will an inspection with a single violation, assuming all the violations are in the same risk category. This result reflects the belief that multiple violations compound the safety hazard posed from driver deficiencies and/or vehicle defects.
Once each inspection has been assigned its probability of avoiding a crash, the inspections are grouped by their initiating intervention. An inspection with a traffic enforcement driver violation is classified as traffic enforcement with a driver and/or vehicle roadside inspection component(s). All other inspections are classified as entirely driver and/or vehicle roadside inspections. Direct-effect crashes-avoided totals are simply the summation of 1) the portions of crashes avoided for all traffic enforcement violations and 2) the summation of the portions of crashes avoided for all roadside inspection violations.