Section 1

Introduction and Background

Utah Transit Authority (UTA) implemented a Connection Protection system (CP) to improve the reliability of transfers from the higher frequency light rail trains, TRAX, to the lower frequency bus services. The CP system examines the status of TRAX trains and issues a “HOLD” message to buses waiting at the connecting rail stations via the onboard Mobile Data Terminal (MDT), if the lateness of train is within a pre-determined threshold (e.g., 3 minutes). The system was completed and tested in January 2002 prior to the Winter Olympic Games in Salt Lake City.

The successful implementation and operation of the CP system has received attention from the Federal Transit Administration (FTA) (the funding agency) and USDOT Intelligent Transportation System (ITS) Joint Program Office (JPO) to warrant a national evaluation study. The objectives of the evaluation are to assess and document the success of CP system and share the experience and lessons learned with other agencies that may be considering a similar system.

Battelle Memorial Institute was selected in September 2002 to conduct the evaluation. Brigham Young University (BYU) in Provo, Utah, was contracted by Battelle to provide various field data collection support to the evaluation. The evaluation is expected to last from October 2002 to October 2003. A kickoff meeting, attended by the evaluation team and the UTA CP project team, was held in the UTA office at Salt Lake City, Utah, in November 2002. In order to understand how the CP system works and to provide guidance to the overall evaluation design, an exploratory analysis using operation data provided by the UTA was performed in February 2003.

This evaluation plan is the first in a series of deliverables to be developed by the evaluation team. The evaluation plan includes the background and the objectives of the evaluation, a description of the CP system, the evaluation approach, results of an exploratory analysis, discussion of the proposed evaluation tests, and an evaluation management plan, including estimated level of effort, management structure, schedule and deliverables. Following the acceptance of this evaluation plan, detailed plans for the individual evaluation tests will be developed. This evaluation plan is organized as follows:

Section 1 – Introduction and Background
Section 2 – Objectives of the Evaluation
Section 3 – System Description
Section 4 – Evaluation Approach
Section 5 – Exploratory Analysis
Section 6 – Proposed Evaluation Tests
Section 7 – Evaluation Management

Section 2

Objectives of the Evaluation

A list of objectives for the evaluation of the CP system was initially identified by the ITS Joint Program Office (as part of the Statement of Work). This list was later reviewed and enhanced by the evaluation team and the UTA CP project team in the November 2002 kickoff meeting and in subsequent discussions. The objectives of the evaluation are as follows:

The evaluation will compare the number of successful rail to bus connections with and without the CP system at transfer locations that have similar characteristics;
The evaluation will compare the number of successful rail to bus connections before and after the implementation of the CP system at selected transfer location(s) where new bus services will be added;
The evaluation will assess the number of connections that would be missed without the CP system;
The evaluation will provide an assessment of the operational performance of the CP system in terms of the number of successful connection due to the Connection Protection. Note that the evaluation does not intend to assess the hardware, the specific technology or algorithms that comprise the CP system;
The evaluation will provide an assessment of ancillary impacts associated with the CP operations, including the effect of CP on bus schedule adherence;
The evaluation will provide an estimate of annual time savings resulting from improved connections with the CP system;
The evaluation will assess passengers’ perceptions of improved connections where CP is implemented;
The evaluation will assess the effectiveness of the CP system based on feedback from bus operators, dispatchers, and supervisors;
The evaluation will analyze and compare customer comments from the surveys and the UTA complaint logs with regard to connection reliability with and without CP.

In general, the above objectives support two types of evaluation activities. A quantitative analysis using UTA system operation data including transfer data (e.g., CP message logs, bus arrival and departure times, train arrival times, etc.) will address objectives 1 through 6. A qualitative analysis will address objectives 7 through 9, using surveys and interviews with passengers, operators, supervisors, and analysis of archived customer comments.

These evaluation objectives are explored in greater detail in Section 6: Proposed Evaluation Tests.

Section 3

System Description

UTA’s Connection Protection system is designed to improve the reliability of transfers from the light rail train, TRAX, to the connecting buses on selected bus routes. The CP system constantly monitors the train schedule adherence status by examining the estimated arrival times for the next 3 stations. If the lateness of a train is within a pre-determined threshold (e.g., 3 minutes), a “HOLD” instruction is sent to the connecting buses via the onboard Mobile Data Terminal (MDT). The relatively short 3 minutes threshold was used to avoid possible adverse effects on the bus on-time performance.

The merit of UTA’s CP system is the intelligent integration of a number of existing systems and data, which greatly reduced the project’s capital cost. For example, the estimated train arrival time data are produced for real-time train status display at the light rail stations by a subsystem developed by Geo Focus. The transmission of “HOLD” instruction messages is achieved using the existing Mobile Data Terminal (MDT) as part of the voice/data radio system equipped on all UTA buses. The CP messages are sent the same way as other data messages from the dispatcher to the bus operators but without human intervention. The major capital cost of the CP system is the cost of the computer server and the interface peripherals with other systems. Figure 3-1 depicts related components of the CP system.

Figure 3-1. Pictures of CP System in Operation

Figure 3-1. Pictures of CP System in Operation

The CP system is essentially a computer program that takes relevant data (e.g., estimated train arrival time, train schedule, bus schedule) from other systems and determines if a late train would jeopardize the bus connections at downstream stations. When a potentially missed connection is identified and the delay of the train is within a threshold (e.g., 3 minutes), the CP system looks up the information of the particular bus (using the bus schedule) and sends a “HOLD” message to the onboard Mobile Data Terminal (MDT) using UTA’s radio dispatch communication system.

The CP system is flexible in terms of which transfer location and time to protect. For example, a bus route can be protected only for a certain period of the day. In addition, internal parameters such as the threshold of train lateness (typically 3 minutes) can also be adjusted by individual transfer location and time. For example, to protect the connection of a last bus, the threshold might be increased in order to hold the bus longer until the train’s arrival.

Section 4

Evaluation Approach

This section describes the overall approach taken toward the evaluation of the UTA CP system.

As suggested in the Statement of Work, the quantitative assessment of the effectiveness of the CP system is meant to be a “with” and “without” comparison using transfer locations that are currently under CP system protection and similar locations without CP system protection. The reason for using a “with” and “without” instead of “before” and “after” evaluation design was the lack of useful system data (e.g., train arrival times and bus arrival and departure times) before the implementation of the CP system in January 2002.

In order to better understand how the CP system works and to identify the possible “with” and “without” evaluation scenarios, the Battelle evaluation team conducted an exploratory analysis using system data provided by UTA. The results of the exploratory analysis were used to guide the development of this evaluation plan and the detailed test plans that will follow. Figure 4-1 shows the overall process employed by this evaluation.

Figure 4-1. Overall Evaluatioin Process

Figure 4-1. Overall Evaluation Process

The general evaluation approach of the quantitative study of the effectiveness of the CP system lies in the comparison of a number of key system operation data, including:

CP message logs that indicate the content, time and recipient of the CP messages (i.e., “HOLD” instruction);
Train arrival times;
Bus arrival and departure times.

The CP message logs and train data are being systematically archived by the UTA. Bus data are not available system-wide, as UTA does not have an Automatic Vehicle Location (AVL) system on its buses that would provide vehicle tracking (in time and location) capability. However, arrival and departure time information is available from the Automatic Passenger Counter (APC) equipped “smart buses” that constitute about 15% of the fleet. The Automatic Passenger Counter is used in conjunction with the Global Positioning System (GPS) to determine the time and location of the passenger counts and the arrival and departure times at each stop. The logistical challenge is to allocate the smart buses of various vehicle types to the selected evaluation routes during the data collection period. UTA has agreed to provide all smart buses, including the 60 new buses procured in early 2003, in support of this evaluation. Specific arrangements will be made as part of the development of the detailed test plans. Based on the exploratory analysis, Battelle concluded that the CP message logs, and the train and bus data provided by the UTA are sufficient for conducting the analysis in support of the evaluation objectives described in Section 2.

The general approach for the proposed qualitative assessment is relatively intuitive. A survey is envisioned to collect passengers’ perceptions of connection reliability at transfer locations with and without CP system protection. The selection of with and without CP transfer locations for the passenger survey will be coordinated with the quantitative study, based on the results of the exploratory analysis. The challenge of the passenger survey is to recruit the subjects without jeopardizing their transfer and the rest of their journey. A number of possible intercept survey techniques were explored by the Battelle team during and after the evaluation kickoff meeting. The method and logistical arrangements will be identified with BYU staff and as part of the development of the detailed test plans. Other qualitative analyses will be based on key informant interviews with bus operators and supervisors, and logs of customer comments (complaints) with regard to connection experience.

More discussion of the specific evaluation activities is provided in Section 6: Proposed Evaluation Tests.

Section 5

Exploratory Analysis

To assist in the development of an appropriate “with-without” evaluation design, Battelle conducted an exploratory analysis using data collected by UTA during the last two months of 2002. The purpose of this analysis was to summarize which bus trips were CP protected, which ones received the most messages, and which TRAX stations had the most instances of late trains. An interim technical report covering the results of this exploratory analysis was delivered to DOT on March 7, 2003. Results of the analysis are being used as inputs for the evaluation design.

Five primary objectives, listed below, were defined for the exploratory analysis:

Identify which bus trips are protected by the CP system;
Determine numbers and percentages of bus trips that receive CP messages;
Determine numbers and percentages of late trains at each TRAX station;
Determine the average wait time of buses on each route and/or at each TRAX station;
Identify any unprotected bus trips that could serve as comparable “without” trips during the field evaluation.

5.1 Analysis Methods

UTA has systems in place to collect data related to the performance of the CP system. Collected data include arrival and departure times of buses and trains, a log of CP messages that are generated, and databases of bus and train schedules (including indications of which bus trips have been assigned CP protection). The exploratory analysis made use of data collected between April 2002 and January 2003. Note that three “change periods” (i.e., tri-annual schedule and bus route adjustments to respond to seasonal changes in travel) occurred over the time span covered by the data. This summary of the exploratory analysis focuses on data from the last change period. Additional analyses on the other two change periods are discussed in the full exploratory analysis report delivered to DOT on March 7, 2003.

Bus and train times were grouped into four time-of-day categories for some of the analyses. These categories were defined as:

AM rush = 4:00 am – 10:00 am

Mid-day = 10:00 am – 2:00 pm

PM rush = 2:00 pm – 7:00 pm

Evening = 7:00 pm – 4:00 am

Smart-bus wait times were calculated by subtracting the bus arrival time from the bus departure time at each stop. These arrival and departure times are logged by the GPS equipment on each bus when the bus passes by pre-defined points (i.e., independent equipment from the APC counters that count passengers).

5.2 Analysis Results

Tables showing the results of the various analyses are provided in the appendix (Tables A-1 – A-8). Figure 5-1 provides a UTA TRAX system map to facilitate the interpretation of the analysis results. A discussion of the results is provided below.

Figure 5-1. TRAX System Map

Figure 5-1. TRAX System Map

Protected Trips

The TRAX system is comprised of two lines: the University Line, which runs east/west between downtown and the University of Utah; and the Sandy/Salt Lake Line, which runs north/south between downtown and the suburb of Sandy. Major construction is occurring along the University Line, and bus routes are being detoured accordingly. As a result, CP protection along the University Line is not being implemented until the construction is complete. Also, TRAX stations in the downtown area are not configured to allow buses to meet the trains right at the station; passengers need to walk to the nearest bus stop to make a transfer. For this reason, CP protection is not implemented at the five downtown stations. Consequently, the only TRAX stations that are possible for inclusion in the current evaluation are the stations between Ballpark and Sandy Civic Center on the Sandy/Salt Lake Line.

Excluding the University Line and downtown TRAX stations, there are 1,227 possible weekday bus trips (unique route/station/time combinations) per day that could be CP protected (CP protection was not done on weekends). A total of 77 of these “daily” trips were protected during Change Period #3, representing 6% of the possible weekday trips.

Thirty-nine different bus routes intersect the TRAX stations of interest (including eight routes that intersect multiple stations). Only 15 of these routes had some of their trips protected by the CP system. Table A-1 shows the number of protected and unprotected trips on a given weekday for Change Period #3. A list of the routes where CP protection is implemented (along with their corresponding TRAX stations) is given below:

66 (Ballpark)
35 (Central Pointe)
37, 41 (Millcreek)
36, 42 (Meadowbrook)
33 (Fashion Place West)
88 (Midvale Center)
90, 94 (Historic Sandy)
41, 46, 47, 345, 811 (Sandy Civic Center)

The vast majority of trips that were CP protected were PM-rush trips (59 out of 77). Twelve, four, and two trips were CP protected during the AM-rush, mid-day, and evening, respectively. Table A-2 shows the number of protected and unprotected trips by time of day and frequency of bus route.

CP Messages

Four types of CP messages are generated by the CP system and recorded in the message log that is maintained by UTA. These include messages that are transmitted to appropriate buses (“Hold” messages); messages that are generated too late to be transmitted, as determined by the pre-defined thresholds (“Missed” messages); messages that should be transmitted but cannot, because the bus driver has not logged into the system correctly or some other problem has occurred (“Bad Transmission” messages); and repeat messages that are generated because of increasingly late trains and are not transmitted because the bus has already been instructed to hold (“Discarded” messages).

During November and December 2002, there were 41 possible days of travel (excluding Thanksgiving Day and Christmas Day, when there was no service; and weekends, when no trips were CP protected). Given that there are 77 protected trips each day, that yields a total of 3,157 possible protected events for this two month period. A total of 1,415 CP messages were generated during this time period; however, 897 of them were either bad transmissions, or missed or discarded messages. Table A-3 summarizes the 1,415 CP messages by message type. The 518 “Hold” messages transmitted during this period implies that 16.4% of the connected bus trips received “hold” CP messages during this period (518/3157 = 16.4%).

Excluding the 558 weekend “Bad Transmission” messages, there was an average of 20.9 CP messages per day (non-holiday weekday) in November and December. However, there were four days (9%) during this time when no messages were generated. Thus, on those days when at least one CP message was generated, there was an average of 23.2 CP messages per day.

In November and December, buses at Millcreek, Historic Sandy, and Sandy Civic Center received 77% of all of the non-“Bad Transmission” messages (complete location data were not available for the “Bad Transmission” cases). This makes intuitive sense, because these three TRAX stations also had the most protected bus trips associated with them (total of 72%). Another way of comparing the CP messages generated across TRAX stations is to look at the ratio of CP messages to protected trips. This comparison reveals that Millcreek, Historic Sandy, and Midvale Center had the highest relative percent of CP messages generated. While it was not investigated in depth, one possible explanation for this is that the time windows for connections at these stations are tighter than at some of the other TRAX stations, thus more of the protected trips receive CP messages. Table A-4 shows the number and percentage of CP messages generated, the number and percentage of protected trips (summed across the 41 possible days), and the percentage of CP messages relative to protected trips at each TRAX station.

Late Trains

Trains that were at least three minutes late were summarized according to TRAX station, time of day, and day of week (see Tables A-5, A-6, and A-7). Trains were more likely to be late in the evening and afternoon rush, and on Mondays, Fridays, and Saturdays. Also, Sandy Civic Center had the fewest late trains, which is notable because buses at this station received the most CP messages. In addition to Sandy Civic Center having the most protected trips, the obvious reason for the abundance of CP messages is that 14 bus routes intersect with Sandy Civic Center, compared to between one and six routes at other TRAX stations. As a result, one late train at Sandy Civic Center is likely to cause more CP messages to be generated so that all affected buses can be held.

The degree of lateness for these trains ranged from 3 minutes through 600 minutes, with an arithmetic mean of 10.5 and a median of 4. No major differences were seen in the lateness of trains at each of the TRAX stations.

Smart Buses

Over 9,500 trips were made by smart buses in November and December. On over half of these trips (56%), the buses apparently did not stop to drop off or pick up passengers at the TRAX station (i.e., the arrival and departure times were equal). Two sets of statistics were calculated on the wait times of the smart buses. The first included all of the trips, while the second included only those trips that actually stopped at the TRAX station. In the overall case, the buses waited for an average of 3.4 minutes at each TRAX station, while the average was 7.8 minutes if only those buses that stopped were included. No major differences were seen in the wait times at each of the TRAX stations.

Potential CP Assignments

In trying to identify trips that could be used in a “with and without” comparison for the field evaluation, the protected and unprotected trips were categorized by the time of day that they occurred, the frequency between bus trips on a particular route, and the TRAX station that they served. This comparison is presented in Table A-8.

Listings showing each individual bus trip that occurred in November and December then were generated and examined to determine trips that were not CP protected and that were similar in nature to protected trips. The goal was to identify unprotected trips that were scheduled to depart the TRAX station at least three minutes after the scheduled train arrival time and no more than one minute after the corresponding CP-protected bus was scheduled to depart the station. These trips were considered to be potential “without” cases (subject to additional input that UTA may have about them). Table A-9 lists all of these trips and their associated protected trips (including the number of “Hold” messages generated in November and December, and the train time associated with each CP-protected trip). In summary, the number of potential “without” cases at each of the TRAX stations for which cases were identified is as follows:

Fashion Place West – 5 trips over 2 routes
Millcreek – 8 trips over 2 routes
Meadowbrook – 6 trips over 1 route
Midvale Center – 3 trips over 1 route
Historic Sandy – 8 trips over 2 routes
Sandy Civic Center – 25 trips over 4 routes

5.3 Implication for Evaluation Design

It appears that there may be an opportunity to conduct a “with vs. without” comparison of UTA’s CP system. A total of 55 unprotected bus trips were identified that occur in close proximity to bus trips that are already CP protected. Additional trip characteristics including type of service (local vs. express) and passenger volume can be investigated to determine which trips would be best for inclusion in the evaluation.

Section 6

Proposed Evaluation Tests

Two types of tests will be performed to evaluate the performance of and satisfaction with the CP system. The first one will be a quantitative analysis of the performance of the CP system, while the second one will be a qualitative assessment of customer and UTA satisfaction with the CP system. This section describes these two tests in further detail. Complete details on how they will be conducted will be provided in the respective detailed test plans.

6.1 CP System Performance Test

An evaluation will be conducted to determine how well the CP system is operating in regard to preventing missed connections for train and bus passengers. The objectives of this analysis and the manner in which it will be conducted are described in this section.

6.1.1 Test Objectives

The primary objective of the evaluation of system performance data is to evaluate the effectiveness of the CP system. More specifically, the objective is to evaluate the ability of the CP system to prevent missed connections and its ability to improve the number of successful train-to-bus transfers for selected scenarios.

A second objective of the evaluation of system performance data is to evaluate operational aspects of the system performance. However, this will not include an evaluation of the specific hardware and software that comprise the CP system. Rather, particular emphasis will be on evaluating the extent to which the CP system consistently operates the way that it was designed from an overall perspective.

Finally, a third objective of the evaluation of system performance is to assess unforeseen or unintentional impacts resulting from implementing the CP system. In short, this portion of the evaluation of system performance data will be conducted to determine if the benefits of the CP system in terms of protecting riders have unforeseen costs associated with them. For example, does holding a bus via a CP message at one station cause missed bus-to-bus connections downstream at later stops?

6.1.2 Hypotheses and Measures of Effectiveness

There are many different measures that could be used to meet the objectives of the system performance evaluation. Table 6-1 contains some key measures and example hypotheses for each of the three objectives discussed above. These measures and hypotheses will be refined as part of the development of a detailed test plan.

Table 6-1. Objectives, Hypotheses and Measures of System Performance

Objectives	Hypotheses	Measures	Sources of Data
Assess the effectiveness of the CP System in reducing the number of missed connections	§ CP prevents missed connections that would otherwise be missed (i.e., CP increases the overall number of successful connections) § CP prevents missed connections in extreme situations (e.g., last bus of the day) § There are no bus trips that are not being currently protected that should be (i.e., cases where CP could have a significant impact)	§ Number and percentage of missed connections § Average bus wait time § Number and percentage of late train events that would trigger a CP message § Bus arrival and departure time § Train arrival time	§ CP Message Logs § Smartbuses § Train logs
Assess the operational aspects of the CP system, not to include software or hardware	§ Messages are being issued when they are supposed to be § When a message is issued, drivers receive the message § Drivers do not ignore the messages § CP messages are targeted for the correct trips	§ Number of messages issued vs. expected § Number of messages that were not received § Number of messages that were erroneously sent	§ CP Message Logs § Train logs
Determine if there are unanticipated impacts as a result of implementing the CP system	§ CP does not increase safety risks (i.e., drivers do not travel at faster speeds to “make up” time lost from a CP hold). § CP does not have lasting impacts on bus routes (e.g., buses are back on schedule within a few stops of the CP hold) § Buses have greater occupancy when CP messages are issued than without (more cost effective)	§ Bus occupancy § Number of “on-time” arrivals of buses § Average wait time § Number of missed bus-to-bus connections	§ Smartbuses

Objectives

Hypotheses

Measures

Sources of Data

Assess the effectiveness of the CP System in reducing the number of missed connections

§ CP prevents missed connections that would otherwise be missed (i.e., CP increases the overall number of successful connections)

§ CP prevents missed connections in extreme situations (e.g., last bus of the day)

§ There are no bus trips that are not being currently protected that should be (i.e., cases where CP could have a significant impact)

§ Number and percentage of missed connections

§ Average bus wait time

§ Number and percentage of late train events that would trigger a CP message

§ Bus arrival and departure time

§ Train arrival time

§ CP Message Logs

§ Smartbuses

§ Train logs

Assess the operational aspects of the CP system, not to include software or hardware

§ Messages are being issued when they are supposed to be

§ When a message is issued, drivers receive the message

§ Drivers do not ignore the messages

§ CP messages are targeted for the correct trips

§ Number of messages issued vs. expected

§ Number of messages that were not received

§ Number of messages that were erroneously sent

§ CP Message Logs

§ Train logs

Determine if there are unanticipated impacts as a result of implementing the CP system

§ CP does not increase safety risks (i.e., drivers do not travel at faster speeds to “make up” time lost from a CP hold).

§ CP does not have lasting impacts on bus routes (e.g., buses are back on schedule within a few stops of the CP hold)

§ Buses have greater occupancy when CP messages are issued than without (more cost effective)

§ Bus occupancy

§ Number of “on-time” arrivals of buses

§ Average wait time

§ Number of missed bus-to-bus connections

§ Smartbuses

6.1.3 Data Collection Plan

Data will be collected for a three-month period from September through November of 2003, plus an additional month if necessary. Most of the data required for the collection of system performance are already routinely collected as part of UTA’s normal course of operations. Therefore, we anticipate that some system performance data will be available for months proceeding the field period. We will capture and use as much of this earlier information as possible. This data will be used to provide more accurate measures of average wait times, bus occupancy, frequency of late trains, and the number of CP messages that are issued.

There are a number of different sources of data that will be used to evaluate system performance. The data that will be used for the evaluation of system performance include:

TRAX Schedules – scheduled train times at each of the TRAX stations.

Bus Schedules – scheduled bus times for each bus route that intersects a TRAX station.

CP Assignments – bus trips (route/time combinations) at each TRAX station that have been designated for CP protection. These data are prepared as part of UTA’s normal operations.

CP Message Logs – CP messages generated by the CP system. These data are currently captured by UTA as part of normal operations. However, the data need to be archived before they are overwritten.

Bus Arrival/Departure Times – Bus arrival and departure times at each TRAX station. These data are automatically collected by UTA for buses that are GPS/APC equipped (i.e., “Smart Buses”). If non Smart Buses are used, then these data may have to be manually collected by bus operators or by other methods.

Train Data – Arrival and departure times for trains at each of the TRAX stations. All light rail trains operated by UTA are equipped with GPS receivers that are used to record the real-time location of each train. UTA maintains a train tracking system that electronically collects these GPS data and monitors the performance of each train. The data are stored in a database and are retained for at least 40 days. Pertinent data (train identifier, scheduled and actual arrival/depart times, etc.) will be queried from the train tracking system.

As discussed above, all but one of these data sources are developed as part of the normal operation of the CP system. However, it will be important to capture the information for the evaluation before it is inadvertently erased or recycled as part of daily operations. With bus arrival and departure times, there may not be sufficient data collected as part of routine daily operations to facilitate a robust analysis. Should it prove necessary, we would collect supplemental information on bus arrival and departure times by asking the bus drivers to record these times on a log sheet. Existing log sheets would be used if available.

6.1.4 Data Analysis and Reporting

An ideal approach for examining system performance, as it can be used to measure the effectiveness and other impacts of the CP system, would be to use a “before and after” approach over comparable time periods. In this approach, each bus trip (i.e., unique route/location/time combination) serves as its own control. An alternative to this approach would be to employ a “with and without” approach, where comparable “without” cases serve as controls for the “with” cases. The first approach requires an assumption of comparable time periods, while the second approach requires the assumption of similar “with” and “without” bus trips.

For this evaluation, a multi-faceted approach will be used to evaluate the system performance information to assess the effectiveness of CP on reducing missed connections. The following describes the facets of this approach. In all cases, except where noted, the key outcome of interest is the number or percentage of cases where a missed connection was identified.

Analysis of “With” and “Without”

UTA has indicated that the trips that tend to be CP protected are those with longer intervals between buses or are during the evening rush hour or at the end of the day. Most of the CP-protected trips are at Historical Sandy and Sandy Civic Center. The statistical analysis would consist of comparing the percentage of missed connections (as defined by a bus departure time minus a train arrival time being less than one minute) among the “without” trips to the corresponding percentage among the “with” trips under different conditions, such as all days and instances where the train had a late arrival. If a significant difference in the percentage of missed connections can be identified, then this would be evidence that CP has/is making a difference among the CP-protected trips. However, the converse may not be true. Analysis of variance and logistical regression techniques will be used for the statistical analysis. For example, a logistic regression model will be employed to model the probability of not missing a transfer as a function of time of day of transfer, whether the transfer point was included in the CP system, and the transfer point itself.

Implementation of this approach would consist of collecting system data for all trips that are currently being CP protected (i.e., the “with” cases) and for a selected number of trips that are not currently being CP protected (i.e., the “without” cases). Overall, data will be collected simultaneously from those trips that are being protected and those that are not. This data collection activity will occur over a three-month period.

Analysis of “Before” and “After”

Estimates of the number or percentage of missed connections could be compared for some of the trips that are not currently being CP protected (“Before” cases) to similar estimates once CP has been activated for those trips (“After” cases). This has the advantage in that each trip serves as its own control, which increases the ability to statistically determine the impact of CP on these locations. The difficulty in this approach is that these are trips that are of a “lesser” concern, which is why they were not CP protected in the first place. However, if CP can be shown to be effective for these trips, then this would be strong evidence that it was likely effective for the routes already CP protected, though the converse would not necessarily be true. Analysis of variance and logistical regression techniques similar to those employed for the With-and-Without analysis also will be used in this statistical analysis.

This approach will be utilized for the two new bus routes being added to the Ballpark station, as well as for any other routes that can be identified. Information on these routes will be collected for a period of approximately 1.5 months without protecting any trips. Then, the CP system will be turned on for trips on these routes and data will be collected for another 1.5 months.

Analysis of Average Wait Times

Late train events will be examined for the trips that are CP protected. For each event, a determination will be made as to whether a missed connection was avoided because of the CP system. A historical average wait time will be applied to determine the likely outcome in the event that a CP message was not issued. This analysis would seek to answer questions such as: if the train was estimated to be 4 minutes late and a CP message was sent to the bus driver to hold the bus long enough for the train to arrive, would the driver have been likely to wait anyway, based upon the historical wait times? The challenge with this approach is that is relies upon applying a historical average to specific events and assumes that there has been no systematic change in the average waiting times of bus drivers. Ideally, information on the average wait time for each trip would be available prior to the implementation of CP. If this is not the case, then this will be estimated using data from non-CP events at the CP protected trips or from non-CP protected trips.

Analysis of Undeliverable Messages

There are a significant number of “Missed” messages that could occur during the three-month field period. For example, over the course of a 153-day period in late summer/early fall there were 263 messages that were “Missed” messages, which represents 16% of all messages that were issued. These messages represent cases where the bus operator had not logged into the system or cases where the scheduled departure time of the bus was before the CP message was issued. These cases will be examined to evaluate system operation as well as effectiveness of the system. Because these types of messages are uncontrollable, no action is needed to implement this method.

Analysis of Ancillary Impacts

Analysis of variance and descriptive statistics will be used to examine issues such as impacts to subsequent stops, speed, and bus occupancy. In all cases, comparisons will be made between events where a CP message was issued to events where a CP message was not issued. That is, this comparison would only be made using CP bus trips and each bus trip would serve as its own control. For example, the lateness of the bus (with respect to the schedule) at the next subsequent stop (time point) following the TRAX station will be compared between CP and non-CP events. This analysis would give insight into whether the bus driver tended to increase speed to “make up” for a CP hold.

6.2 CP User Satisfaction Test

This section describes the evaluation plan for the qualitative assessment of customer and operator satisfaction with the CP system. Complete and final details of the proposed evaluation tests will be provided at a later date after consultation with UTA and BYU staff in the form of detailed test plans.

6.2.1 Test Objectives

The main objective of the qualitative evaluation is to assess traveler and bus operator satisfaction with the CP system. Specific objectives are expressed in the following evaluation questions:

Do travelers perceive that their rail-to-bus connections are more reliable, either compared to pre-CP experience or between protected and unprotected connections?
Do travelers perceive that they are deriving value from the system?
Do travelers suggest any improvements or changes related to their connection experiences?
Do travelers save travel time due to the CP system, and if so, how much?
Are there any unanticipated effects associated with the CP system, and if so, what can be done to mitigate them?
Are system managers satisfied with CP system performance?
Are dispatchers satisfied with CP system performance?
Are bus operators satisfied with CP system performance?
Do bus operators suggest any improvements or changes that could enhance the operation or performance of the CP system?
Do riders indicate that connection performance is a factor in their decision to use transit?
Are complaints or comments regarding missed connections reduced in regard to routes that are CP protected?

To meet these objectives, the evaluation will involve a series of intercept surveys, with interviews of selected passengers on the UTA rail and bus system at specified transfer points to be conducted by trained and supervised BYU student interviewers. In addition, key informant interviews will be conducted with selected bus drivers, CP system operators and managers, and UTA customer comment logs will be examined and included in the analysis where appropriate. A final objective will be to add qualitative depth and interpretation to the system data being collected and analyzed under the system performance test.

6.2.2 Hypotheses and Measures of Effectiveness

Table 6-2 presents an initial understanding of the user satisfaction benefits, hypotheses and measures that will guide this component of the CP evaluation.

Table 6-2. Anticipated Benefits, Hypotheses, Measures and Data Sources

Objectives and Anticipated Benefits	Hypotheses	Measures and Data Sources
Increase satisfaction of transit users	- TRAX rail riders perceive that bus connections are reliable, and more reliable than they used to be - Bus riders are satisfied (not bothered by) connection delays at TRAX stations - Bus/rail rider complaints are reduced, or less related to connection issues - Transit users who have rail/bus transfers use transit for their travel more often	- Stated preference surveys of riders (Source: in-person interviews using intercept surveys at selected TRAX stations and on buses) - Content analysis of customer complaints (Source: complaint logs)
Increase satisfaction of bus operators, dispatchers, and transit system managers	- Bus drivers are more satisfied with connection performance - Bus drivers report fewer passenger complaints with connection performance - System operators/dispatchers/managers are satisfied with rail/bus system performance and perceive that the CP system does not have an adverse effect on bus schedule performance/adherence - Interviewees perceive a beneficial effect of CP on connection success and bus schedule performance/adherence	- Key informant interviews using structured questions (Source: selected bus drivers, dispatchers, managers)

6.2.3 Data Collection Plan

To evaluate user satisfaction with the connection protection program, data will be collected from various user groups by several methods that include the following:

Intercept surveys of TRAX rail riders who make transfers to buses (both on protected and on non-protected routes) to assess their experiences with the connection protection system.
Intercept survey of bus riders at the rail transfer points who did not arrive by train to assess how bus delays caused by this system may impact them.
Enumeration of the number of TRAX riders transferring to buses at the stations identified for interviewing, and measure time saved by CP.
Surveys and selected interviews with bus drivers, dispatchers, supervisors, and system operators and managers to assess their perceptions of the functionality and benefits of the CP system.
Review and analysis of UTA complaint logs.

The timing of the data collection period will need to take account of BYU’s staff and student interviewer availability, as well as avoiding UTA’s seasonal schedule changes. The data collection period is expected to be during September through November of 2003.

Site Selection. Two or three sites will be selected for the conduct of the intercept surveys. The underlying intent is to intercept and interview transit riders who travel on rail-bus routes that historically have experienced late trains and actual or potentially missed rail-to-bus connections. Rail-bus routes that meet these conditions will be further divided into those that have connection protection operating and those that do not. This with-without sample design will allow us to evaluate the role that CP plays in travelers’ experiences and satisfaction with transit service.

Two or three TRAX rail stations that experience the highest frequency of late trains but where only some of the bus routes/times are protected. This approach will allow for control of a number of factors that might influence customer satisfaction, including geography of the locations, some attributes of travelers, and trip timing. For this approach to be successful, we will need bus routes that are both protected and unprotected with schedules and headways that are similar. An advantage to this approach is the survey administration efficiency that is gained by having to supervise student interviewers at a few locations. Furthermore, this approach offers synergy with the system performance evaluation that intends to follow a similar evaluation design.

Rider Intercept Surveys. Data on rider satisfaction with the CP program will be collected through surveys with a sample of riders. This component of the test plan has several steps that include:

Design the survey questionnaires
Develop the sampling plan
Train and supervise BYU student interviewers
Collect, process, and analyze the data

Battelle will work closely with the BYU survey supervisor and student interviewers to design and carry out the intercept surveys. The plan is to use trained BYU student interviewers to conduct intercept surveys with riders of the UTA TRAX and bus system at a pre-selected rail stations. The primary target audience for the survey is those TRAX rail riders who intend to connect with a bus at one of the rail stations along the rail route. These riders will be sampled over a one-week period by day of week and time of day. Details of the sampling plan will be developed based on information about bus routes, ridership levels, arrival/departure schedules, and the analysis of historical CP system performance data. The sampling plan will be included in the detailed test plans. An effort will be made to create a representative sample of rail-to-bus transfer riders who are exposed to missed connections so that conclusions from this evaluation can be generalized to the population of all train-to-bus transfer riders of the system. In addition, a smaller sample of bus riders who are on buses that may be subject to delays due to the connection protection system will be surveyed in order to evaluate any potential issues associated with schedule delay due to the CP program experienced on those bus routes.

We expect that many transit riders are, and will continue to be unaware of the existence of the CP program, and more specifically of whether or not a CP message has been issued that may impact their travel experience. That is, they are likely to only be aware of whether or not they successfully make their rail-to-bus connection, and not why they may have succeeded or failed, especially when their TRAX arrival at the connecting station is known to be later than scheduled. Given this, the survey will be able to assess rider satisfaction with the connection (and other) service attributes, and with rider awareness of the CP program, but the focus will be on evaluating differences in rider satisfaction with their connection experience since the introduction of CP program.

During the next site visit, Battelle will work with UTA and BYU to finalize the survey design, interview formats, and the sampling plan. Prior to conducting the surveys, students from the BYU Department of Engineering will be recruited and given an orientation to the project and interview training. They will be assigned to the selected transfer station(s) at specified times and instructed to intercept a sample of riders who are transferring from rail to bus. Both protected and unprotected transfers will be sampled, but the protected transfers will be over-sampled. Because transfer time is expected to be very limited, riders are expected to be in a hurry. In order to avoid a sampling bias associated with differential value of time by these travelers, transferring riders will be identified and intercepted either while boarding the bus or once on the bus. An effort will be made to identify all persons on the bus at the connecting station who have just transferred from the arriving TRAX train. Persons who arrived at the connecting station on the target bus, or who are initiating their bus ride from that station but did not arrive by TRAX, will be identified and a sample of them selected to complete a second (somewhat different) questionnaire.

Respondents will be asked if they would be willing to complete a short written survey (self-administered questionnaire), and handed a packet containing an official introductory letter requesting their support, a brief explanation of the purpose of the survey, a copy of the appropriate survey depending on whether they are transferring from rail to bus or not, and a postpaid mailer to allow respondents to mail back their completed questionnaire. However, the student interviewers will be riding the bus with the intent of collecting the completed questionnaires prior to the next bus stop. A toll-free phone number will be provided for answering any respondent questions or responding to concerns. An incentive will be offered to those who complete the survey, and the details of this will be worked out with UTA. For example, it may be possible for the UTA to offer one or more reduced fare trips in the future to riders who complete the survey, with the bus drivers handing these to riders who turn in their completed questionnaire packet before disembarking.

Because the connection protection program was initiated in January 2002, there is no opportunity to collect pre-deployment baseline data. However, two strategies are available to provide for with and without comparisons of traveler perspective on this program. First, questions will be asked of the target respondents regarding their travel experiences prior to connection protection and to assess changes they perceive with their travel experiences since the CP program has been in place. We will address potential issues of recall bias in this approach. Second, some riders will be intercepted and surveyed on buses that do not have connection protection. These routes will be selected in consultation with UTA, as noted earlier, to match as well as possible routes and conditions where the new service is now being provided. In addition, some riders are expected to have experience with both conditions, to the extent that they transfer from rail to bus at different locations, depending on trip timing and purpose, though they are unlikely to be aware of whether or not connection protection service was operating at the time of their various transfers.

Rider Questionnaire Design and Implementation. Survey questions will be developed and reviewed with UTA and BYU during the forthcoming site visit, and then pre-tested with a sample of riders. This will test for clarity, understandability, and content of the questions, as well as for the feasibility of the intercept strategy and the response rate that can be anticipated. It also will constitute a dress rehearsal for the student interviewers. Refinements will be made before implementing the final survey plan. Sample size targets will be specified in the detailed test plan but are anticipated to be sufficient to achieve about 300-400 completed surveys.

Detailed rider survey questionnaires will be constructed to obtain the kinds of information listed below. However, because of the need to keep the questionnaires fairly short, these questions are presented as potential items for further consideration and prioritization. Most, if not all, of the questions will be presented as closed-ended, offering check box response categories to assure clarity and ease of completion. Draft mockup questionnaires will be prepared and reviewed with the BYU team and refined in a more final draft form.

· How often do you ride the bus? Light rail?

· How long a UTA rider (new, old, number of months)?

· Frequency of transit riding? Trips per week/month?

· How often transfer from rail to bus? At this location? Others?

· Time of day; day of week of transfer?

· Do you use rail/bus to commute? How often? Arrival/departure flexibility? Length of commute? Total number of transfers on your commute?

· Purpose(s) of trips? Commuting, pleasure/sightseeing/recreation, personal business/errands, other?

· Availability of alternative means of transportation? Have access to or own car?

· Is rider transit dependent or ride by choice?

· Perception of on-time performance? How often off schedule? Perception of any changes in experience before and after protection implementation?

· Experiences missing a connection from rail to bus? At this location? Others? Time of day? Day of week? Frequency? Perception of connection reliability?

· How important is it to you to be certain of connecting to the next bus? What is typical wait time if miss bus connection at times you travel?

· Attitudes (concerned about safety/security at transfer center, worry about being late, delay tolerance).

· Ever submitted a complaint to UTA? Current concerns or complaints related to experience with connections from rail to bus?

· Prior awareness of connection protection system from media or elsewhere?

· How does system benefit rider? Impact on riding behavior? Effect on riding frequency?

· Suggestions for improving the system?

· Demographic background (gender, age, education)

Operator Surveys and Key Informant Interviews. Working closely with UTA, we will obtain a list of key informants for surveying and interviewing. These will include drivers of buses on which connection protection has been implemented (and some where it has not been implemented), system supervisors, dispatchers, and managers or others who are in a position to offer perspective on how well this system is perceived to be working and to articulate in their own words the benefits or impacts of the system as they understand them. Given the large number of bus drivers in the UTA system, a sample of all drivers will be interviewed, but an effort will be made to interview all the other key informants. Interviews may be arranged individually or in groups for efficiency. If several individuals are interviewed together, they will be of the same type (drivers, supervisors, etc.), and the interviewer will exercise care to be sure that each individual can express his or her own opinion.

In addition to interviewing a sample of bus drivers, it is expected that a short survey will be distributed to all the bus drivers in the system, coded in order to track which routes are covered and which drivers have connection protection experience. This will allow us to collect a consistent set of data from all drivers who respond to the survey. The drivers will then be sampled and selected into several interview groups to allow more detailed inquiry into the issues associated with the connection protection program and their practical experiences with it. These discussions will go into greater depth than is possible with the short survey questionnaire.

Complaint Logs. In addition to the survey data, comment/complaint logs will be obtained from UTA and analyzed in terms of the issues raised that are potentially pertinent to the connection program. The complaint categories will be reviewed, and UTA will be requested to provide selected information on the nature and details of the complaint for those complaint categories that are likely to be related to transit riders seeking to connect from rail to bus. No information that would reveal the identity of the individual filing the complaint will be requested or provided. Thus, these will be anonymous records of the complaints. Coverage of the complaint logs is believed to go back several years.

6.2.4 Data Analysis and Reporting

Data acquired from the surveys and interviews will be coded, manually entered into an MS Access database, verified, and cleaned. Depending on the sampling strategy selected for identifying intercepts, the data may need to be weighted prior to analysis in order to provide representative results. Analyses will be conducted using SPSS[1] to generate marginal frequency distributions for all the variables in the survey. Relationships among the variables will be explored using cross-tabulations, analysis of variance, or multiple regression, as appropriate, and the results will be presented and interpreted. Where possible, rider perceptions will be compared with system performance data that are discussed in the system performance test plan. This may include, for example, rider perceptions of system delays with objectively measured deviations from schedule. BYU offers no-cost statistical support to all its research departments and will support this team on such statistical details as determining a proper sample size; selecting locations, times and respondents for the intercept surveys; designing a data weighting scheme, and developing analytic strategies. Tabular and graphic data will supplement the report narrative to present results in a clear and understandable way.

Where possible, the complaint log files will be coded by bus route and location. It is recognized that these data are anecdotal and may not be amenable to formal statistical analysis, but rather will offer additional qualitative understanding of issues related to connection protection. Also, these logs will be content-analyzed early on in the research process and mined for issues and ideas that can help shape the structuring of the surveys and interviews. To the extent possible, after inspecting the available data, comparisons will be made station by station (or bus route by bus route) to look for differences in issues and differences between protected and non-protected bus-rail connections. That is, depending on our ability to determine the circumstances to which these complaints apply, they will follow the with-without comparative approach. Also, they may be amenable to a limited before-after analysis using the time stamps on the log files.

Section 7

Evaluation Management

7.1 Evaluation Schedule and Deliverables

Figure 7-1 presents the evaluation schedule and major deliverables. UTA has recommended that the system performance data collection be conducted during the fall schedule change period between August 25, 2003 and November 23, 2003, as opposed to the summer period (April 14, 2003 to August 24, 2003) where low ridership would be experienced due to the absence of student riders. The fall period represents the typical ridership of the UTA systems and provides a good scenario for measuring the effectiveness of the CP system. As a contingency, the evaluation team decided to include an additional month for data collection to insure that enough sample size (i.e., CP messages) will be achieved to entail statistical comparison of connections made with and without the CP.

Figure 7-1. Evaluation Schedule and Deliverables

Figure 7-1. Evaluation Schedule and Deliverables

7.2 Evaluation Management Structure

The management structure of this evaluation is presented in Figure 7-2. The evaluation team is managed by Dr. Jeffrey Jenq of Battelle who reports to Mr. Terrell Williams of Federal Transit Administration (FTA) who serves as the Federal task manager for the CP evaluation. Mr. Williams reports to Dr. Joseph Peters of ITS Joint Program Office (JPO) who is the manager of the ITS Program Assessment (IPAS) program. Mr. Williams is supported by Mr. Steve Mortensen of Mitretek who provides various evaluation supports to the USDOT. Dr. Jenq also will coordinate with the UTA evaluation committee through the ITS Project Manager, Mr. Richard Hodges.

Dr. Chris Cluett of Battelle serves as the qualitative study leader and is supported by the Brigham Young University evaluation team supervised by Professor Mitsuru Saito. Mr. Ben Pierce of Battelle leads the quantitative study and is supported by Mr. Alan Pate of Battelle who serves as the database manager in charge of system data collection, processing, and analysis.

Figure 7-2. CP Evaluation Management Structure

Figure 7-2. CP Evaluation Management Structure

7.3 Work Breakdown Structure

A work breakdown structure (WBS) has been prepared for use in managing all aspects of this earmark evaluation project. The discrete task elements of the project are listed in this work breakdown structure, displayed in Figure 7-3. Battelle’s project manager will refer to this work breakdown structure to track and monitor all tasks and deliverables for the project.

Figure 3. Work Breakdown Structure

Figure 3. Work Breakdown Structure

Figure 7-3. CP Evaluation Work Breakdown Structure

7.4 Estimated Level of Effort

Table 7-1 presents the estimated level of effort for the completion of the evaluation. Note that, per DOT approval, additional efforts were made in performing an exploratory analysis which was not in the original scope. An estimated 270 hours were spent in the planning, execution, and reporting of the exploratory analysis. Additional funding might be needed to complete the remaining tasks of this evaluation.

Table 7-1. Estimated Level of Effort

Task	Estimated Hours
Task	Senior Analyst	Middle Analyst	Junior Analyst	Total
0. Exploratory Analysis	90	120	60	270
1. Evaluation Plan	150	20	0	170
2. Test Plans	115	55	0	170
3. Data Collection	140	244	666	1050
4. Data Analysis	35	300	100	435
5. Interim Briefing	50	0	0	50
6. Evaluation Report	161	85	0	246
Total	741	824	826	2391

7.5 Data Management

This project will collect large amount of system data in the electronic form. Those data will be acquired and archived using an industry standard format such as SAS^® or Microsoft Access. This evaluation will preserve all raw and processed system data as well as the final numeric analysis outputs generated from such data. Mr. Alan Pate of Battelle is the database manager for this evaluation and will perform all data transfer, processing, and archival throughout the entire study. These electronic data will be saved on Battelle’s data server during the evaluation. Daily backup of all data is automatically performed by the Battelle computer system. At the end of the project, all electronic data will be transferred to the USDOT using approved media such as CD ROM discs.

Appendix A

This section contains the exploratory analysis results referenced in the Section 5 Exploratory Analysis.

Table A-1. Number of Protected and Unprotected Trips on a Given Weekday for Each Bus Route

TRAX Station	Bus Route	Number of Protected Trips	Number of Unprotected Trips
Ballpark	66	4	4
Central Pointe	30	-	79
	35	5	14
	442	-	38
Millcreek	31	-	81
	37	3	28
	41	4	16
	131	-	4
	137	-	7
Meadowbrook	15	-	19
	36	3	16
	39	-	70
	42	3	14
	142	-	4
	442	-	19
Murray North	40	-	70
Murray North	140	-	8
Murray Central	10	-	56
	12	-	18
	84	-	26
Fashion Place West	22	-	58
	24	-	26
	33	3	17
	124	-	10
	125	-	9
	222	-	27
Midvale Fort Union	82	-	30
Midvale Fort Union	85	-	27
Midvale Center	25	-	23
	27	-	16
	88	4	2
	125	-	9
	222	-	54
Historic Sandy	24	-	52
	90	6	13
	94	7	12
	124	-	10
Sandy Civic Center	12	-	16
	24	-	52
	33	-	19
	41	1	20
	46	4	-
	47	4	5
	124	-	5
	125	-	4
	133	-	4
	143	-	5
	222	-	27
	345	4	4
	811	22	1
	816	-	2
Total		77	1150

Table A-2. Distribution of Weekday Bus Trips by Time of Day vs. Frequency of Bus Route

Bus Route Frequency	Number of CP Protected Trips				Number of Non-CP Protected Trips				Total
Bus Route Frequency	AM rush	Mid-day	PM rush	Evening	AM rush	Mid-day	PM rush	Evening	Total
< 25 min	0	0	3	0	126	62	140	0	331
25-55 min	12	0	52	0	271	156	221	14	726
> 55 min	0	4	4	2	0	64	14	73	161
Undefined*	0	0	0	0	1	1	0	7	9
Total	12	4	59	2	398	283	375	94	1227

*only 1 trip departs on a given route during time period, thus frequency cannot be calculated

Table A-3. CP Messages Generated During November and December 2002

CP Message Type	Number of CP Messages
CP Message Type	November	December	Total
Hold	202	316	518
Missed	37	78	115
Bad Transmission*	0	770	770
Discarded**	0	12	12
Total	239	1176	1415

* 558 of these occurred on weekends, even though no trips are CP protected on weekends

**not generated by CP System until Version 2 was installed on (or about) 12/12/02

Table A-4. November/December CP Messages Generated for Buses at Each TRAX Station*

TRAX Station	Number (Percent) of CP Messages	Number (Percent) of Protected Trips**	CP Messages as Percent of Protected Trips
Ballpark	29 (4.5%)	164 (5.2%)	17.7%
Central Pointe	44 (6.8%)	205 (6.5%)	21.5%
Millcreek	105 (16.3%)	287 (9.1%)	36.6%
Meadowbrook	10 (1.6%)	246 (7.8%)	4.1%
Murray North	0 (0.0%)	0 (0.0%)	--
Murray Central	0 (0.0%)	0 (0.0%)	--
Fashion Place West	18 (2.8%)	123 (3.9%)	14.6%
Midvale Fort Union	0 (0.0%)	0 (0.0%)	--
Midvale Center	45 (7.0%)	164 (5.2%)	27.4%
Historic Sandy	149 (23.1%)	533 (16.9%)	28.0%
Sandy Civic Center	245 (38.0%)	1435 (45.5%)	17.1%
Total	645 (100%)	3157 (100%)	20.4%

* 770 “Bad Transmission” messages not included

** based on 41 days and Change Period #3 schedule

Table A-5. Late Train Trips During November and December by TRAX Station

TRAX Station	Number (Percent) of Late Train Trips	Number (Percent) of Scheduled Train Trips*	Late Train Trips as Percent of Total Train Trips
Ballpark	265 (8.3%)	7999 (9.0%)	3.3
Central Pointe	230 (7.2%)	7999 (9.0%)	2.9
Millcreek	251 (7.9%)	7999 (9.0%)	3.1
Meadowbrook	241 (7.7%)	7999 (9.0%)	3.0
Murray North	238 (7.5%)	7999 (9.0%)	3.0
Murray Central	376 (11.8%)	7999 (9.0%)	4.7
Fashion Place West	335 (10.5%)	7649 (8.6%)	4.4
Midvale Fort Union	346 (10.9%)	8398 (9.4%)	4.1
Midvale Center	398 (12.5%)	8348 (9.4%)	4.8
Historic Sandy	357 (11.2%)	8348 (9.4%)	4.3
Sandy Civic Center	146 (4.6%)	8348 (9.4%)	1.7
Total	3183 (100%)	89085 (100%)	3.6

* based on Change Period #3 schedule

Table A-6. Late Train Trips During November and December by Day of Week

Day of Week	Number (Percent) of Late Train Trips	Number (Percent) of Scheduled Train Trips*	Late Train Trips as Percent of Total Train Trips
Sunday	160 (5.0%)	6705 (7.5%)	2.4
Monday	582 (18.3%)	14877 (16.7%)	3.9
Tuesday	400 (12.6%)	14877 (16.7%)	2.7
Wednesday	246 (7.7%)	11571 (13.0%)	2.1
Thursday	331 (10.4%)	11571 (13.0%)	2.9
Friday	666 (20.9%)	14877 (16.7%)	4.5
Saturday	798 (25.1%)	14607 (16.4%)	5.5
Total	3183 (100%)	89085 (100%)	3.6

* based on Change Period #3 schedule

Table A-7. Late Train Trips During November and December by Time of Day

Time of Day	Number (Percent) of Late Train Trips	Number (Percent) of Scheduled Train Trips*	Late Train Trips as Percent of Total Train Trips
AM rush	606 (19.0%)	22608 (25.4%)	2.7
Mid-day	537 (16.9%)	19877 (22.3%)	2.7
PM rush	895 (28.1%)	25585 (28.7%)	3.5
Evening	1145 (36.0%)	21015 (23.6%)	5.4
Total	3183 (100%)	89085 (100%)	3.6

* based on Change Period #3 schedule

Table A-8. Distribution of Bus Trips by Time of Day vs. Frequency of Bus Route

Bus Route Frequency	CP Protected (# trips)				Non-CP Protected (# trips)				Total
Bus Route Frequency	AM rush	Mid-day	PM rush	Evening	AM rush	Mid-day	PM rush	Evening	Total
Ballpark
< 25 min	-	-	-	-	-	-	-	-	0
25-55 min	4	-	-	-	-	-	-	-	4
> 55 min	-	-	-	-	-	-	4	-	4
Undefined*	-	-	-	-	-	-	-	-	0
Central Pointe
< 25 min	-	-	-	-	24	24	28	-	76
25-55 min	-	-	5	-	21	-	19	2	47
> 55 min	-	-	-	-	-	12	-	-	12
Undefined*	-	-	-	-	-	-	-	1	1
Millcreek
< 25 min	-	-	3	-	30	22	38	-	93
25-55 min	-	-	4	-	19	8	4	7	42
> 55 min	-	-	-	-	-	4	-	4	8
Undefined*	-	-	-	-	-	-	-	-	0
Meadowbrook
< 25 min	-	-	-	-	26	-	26	-	52
25-55 min	-	-	6	-	26	16	26	-	74
> 55 min	-	-	-	-	-	16	-	4	20
Undefined*	-	-	-	-	-	-	-	2	2
Murray North
< 25 min	-	-	-	-	26	-	26	-	52
25-55 min	-	-	-	-	-	16	-	-	16
> 55 min	-	-	-	-	-	-	-	8	8
Undefined*	-	-	-	-	-	-	-	2	2
Murray Central
< 25 min	-	-	-	-	-	-	-	-	0
25-55 min	-	-	-	-	35	24	36	-	95
> 55 min	-	-	-	-	-	4	-	-	4
Undefined*	-	-	-	-	-	-	-	1	1
Fashion Place West
< 25 min	-	-	-	-	20	16	22	-	58
25-55 min	-	-	3	-	26	16	24	5	74
> 55 min	-	-	-	-	-	4	-	14	18
Undefined*	-	-	-	-	-	-	-	-	0
Midvale Fort Union
< 25 min	-	-	-	-	-	-	-	-	0
25-55 min	-	-	-	-	19	16	18	-	53
> 55 min	-	-	-	-	-	-	-	4	4
Undefined*	-	-	-	-	-	-	-	-	0
Midvale Center
< 25 min	-	-	-	-	-	-	-	-	0
25-55 min	-	-	-	-	36	20	29	-	85
> 55 min	-	-	4	-	-	4	5	9	22
Undefined*	-	-	-	-	-	1	-	-	1
Historic Sandy
< 25 min	-	-	-	-	-	-	-	-	0
25-55 min	-	-	13	-	30	16	22	-	81
> 55 min	-	-	-	-	-	8	-	10	18
Undefined*	-	-	-	-	1	-	-	-	1

Sandy Civic Center
< 25 min	-	-	-	-	-	-	-	-	0
25-55 min	8	-	22		59	24	42	-	155
> 55 min	-	4	-	2	-	12	5	20	43
Undefined*	-	-	-	-	-	-	-	1	1
Total	12	4	60	2	398	283	374	94	1227

*only 1 trip departs on a given route during time period, thus frequency cannot be calculated

Table A-9. Potential Bus Trips that Could Be Used for a “With vs. Without” Evaluation

TRAX Station	Potential “Without” Case				Corresponding CP-Protected Trip(s)			# CP Msg	Train Time
TRAX Station	Bus Route	Bus Direction	Bus Time	Bus Frequency	Bus Route	Bus Direction	Bus Time	# CP Msg	Train Time
Fashion Place West	22	Inbound	16:20	<25 min	33	Southbound	16:20	5	16:14
	222	Southbound	16:20	25-55 min*	33	Southbound	16:20	5	16:14
	22	Inbound	16:50	<25 min	33	Southbound	16:50	3	16:44
	222	Southbound	16:50	25-55 min*	33	Southbound	16:50	3	16:44
	222	Southbound	17:20	25-55 min*	33	Southbound	17:20	5	17:14
Millcreek	31	Eastbound	16:38	< 25 min*	37	Westbound	16:40	7	16:35
	31	Westbound	16:40	< 25 min*	37	Westbound	16:40	7	16:35
	31	Eastbound	16:54	< 25 min	41	Southbound	16:55	9	16:50
	37	Westbound	16:55	< 25 min	41	Southbound	16:55	9	16:50
	31	Eastbound	17:40	< 25 min*	37	Westbound	17:40	8	17:35
	31	Eastbound	17:55	< 25 min	41	Southbound	17:55	8	17:50
	31	Eastbound	18:25	< 25 min	41	Southbound	18:25	8	18:20
	31	Westbound	18:25	< 25 min	41	Southbound	18:25	8	18:20
Meadowbrook	39	Westbound	16:25	< 25 min	36	Westbound	16:31	0	16:22
	39	Westbound	16:45	< 25 min	42	Southbound	16:46	0	16:37
	39	Westbound	16:55	< 25 min	36	Westbound	17:01	1	16:52
	39	Westbound	17:15	< 25 min	42	Southbound	17:16	2	17:07
	39	Westbound	17:25	< 25 min	36	Westbound	17:31	1	17:22
	39	Westbound	17:45	< 25 min	42	Southbound	17:46	1	17:37
Midvale Center	27	Inbound	15:06	> 55 min*	88	Westbound	15:08	2	15:03
	27	Inbound	17:06	> 55 min*	88	Westbound	17:08	8	17:03
	27	Inbound	18:06	> 55 min*	88	Westbound	18:08	9	18:03
Historic Sandy	24	Southbound	15:54	25-55 min*	94	Eastbound	15:55	11	15:50
	90	Westbound	15:55	25-55 min*	94	Eastbound	15:55	11	15:50
	24	Southbound	16:24	25-55 min*	90 94	Westbound Eastbound	16:25 16:25	9 7	16:20
	24	Southbound	16:54	25-55 min*	90 94	Westbound Eastbound	16:55 16:55	8 7	16:50
	24	Southbound	17:24	25-55 min*	90 94	Westbound Eastbound	17:25 17:25	5 4	17:20
	24	Southbound	17:54	25-55 min*	90 94	Westbound Eastbound	17:55 17:55	8 7	17:50
	24	Southbound	18:24	25-55 min*	90 94	Westbound Eastbound	18:25 18:25	12 9	18:20
	24	Southbound	18:54	25-55 min*	90 94	Westbound Eastbound	18:55 18:55	11 10	18:50
Sandy Civic Center	24	Northbound	6:29	25-55 min*	41	Northbound	6:35	0	6:23
	24	Southbound	6:35	25-55 min*	41	Northbound	6:35	0	6:23
	33	Northbound	6:43	25-55 min*	811	To Provo	6:42	11	6:38
	47	Northbound	6:43	25-55 min*	811	To Provo	6:42	11	6:38
	24	Northbound	6:59	25-55 min*	811	To Provo	6:59	4	6:53
	33	Northbound	7:13	25-55 min*	811	To Provo	7:14	3	7:08
	47	Northbound	7:43	25-55 min*	811	To Provo	7:44	1	7:38
	33	Northbound	7:44	25-55 min*	811	To Provo	7:44	1	7:38
	33	Northbound	8:14	25-55 min*	811	To Provo	8:14	0	8:08
	24	Northbound	9:29	25-55 min*	811	To Provo	9:29	2	9:23
	24	Northbound	10:29	25-55 min*	811	To Provo	10:29	1	10:23
	24	Northbound	11:29	25-55 min*	811	To Provo	11:29	1	11:23
	24	Northbound	12:29	25-55 min*	811	To Provo	12:29	5	12:23
	24	Northbound	13:29	25-55 min*	811	To Provo	13:29	4	13:23
	24	Northbound	14:29	25-55 min*	811	To Provo	14:29	5	14:23
	24	Northbound	15:29	25-55 min*	811	To Provo	15:29	5	15:23
	33	Northbound	16:14	25-55 min*	811	To Provo	16:14	4	16:08
	33	Northbound	16:42	25-55 min*	47 345 811	Southbound Outbound To Provo	16:43 16:44 16:44	2 11 2	16:38
	24	Northbound	16:59	25-55 min*	811	To Provo	16:59	2	16:53
	33	Northbound	17:12	25-55 min*	46 811	Southbound To Provo	17:13 17:14	8 4	17:08
	24	Northbound	17:29	25-55 min*	345	Outbound	17:28	4	17:23
	33	Northbound	17:42	25-55 min*	46 47 811	Southbound Southbound To Provo	17:43 17:44 17:44	10 5 5	17:38
	24	Northbound	17:59	25-55 min*	345 811	Outbound To Provo	17:58 17:59	7 5	17:53
	33	Northbound	18:12	25-55 min*	46	Southbound	18:13	9	18:08
	12	Northbound	18:41	> 55 min	46 47	Southbound Southbound	18:43 18:44	9 3	18:38

* same frequency as protected trip

[1] SPSS is the Statistical Package for the Social Sciences, an analytic software tool that reads standard database formats and allows for a full range of statistical procedures to be applied to the data, including frequency distributions and associated statistics, cross-tabulations to examine relationships among variables in the data, and even more sophisticated forms of analyses.

6.2.4 Data Analysis and Reporting

Task

Estimated Hours

Total