1. Introduction

One of the primary motivations of the Bus Rapid Transit (BRT) Demonstration Program is to assess the effects of demonstration projects through scientific evaluation. Only by carefully documenting and analyzing the effects of the demonstration programs and their features will it be possible to determine which BRT features are most effective in which contexts, that is, the type of service offered, the level of transit demand, the size of the region, and other factors. The various demonstrations will serve as learning tools and as models for other locales throughout the country. In order for these demonstrations to have maximum effectiveness in their respective operational capacities, a consistent, carefully structured approach to project evaluation is desirable.

The Federal Transit Administration (FTA) plans to conduct full-scale evaluations of the ten projects awarded BRT Demonstration Program grants, and is preparing Evaluation Guidelines. This document provides a brief overview of the evaluation component of the program.

2. Evaluation Roles

The major players in a BRT Demonstration evaluation are the local transit agency and its co-sponsors implementing the BRT system, the FTA, and a third-party evaluator designated by the FTA. The diversity of activities and generally long (three to four years) time frame for a demonstration necessitate close and continual coordination among the participants. Their roles break out as follows:

• The evaluator is responsible for developing a comprehensive evaluation plan, including the data collection plan, in conjunction with the local transit agency. As a partner in the demonstration effort, the evaluator will work with the transit agency in monitoring the collection of data and troubleshooting when necessary. The evaluator will prepare interim reports as needed and the final evaluation report.
• The transit agency and its co-sponsors in the demonstration will implement the BRT system as planned, participate in BRT consortium meetings, and cooperate with the evaluator in the development of an appropriate evaluation plan. They will provide data collectors and supervise their efforts. They will also make available any operating data and information needed by the evaluator for assessing the effectiveness of the demonstration.
• The FTA will provide overall guidance for the demonstration project, and will conduct workshops and seminars on relevant subjects for the BRT consortium.

3. Evaluation Objectives

The following are the FTA's objectives for the evaluations:

1. Determine the benefits, costs, and other impacts of individual BRT features, including ITS/APTS applications, and of the system as a whole.
2. Characterize the successful and unsuccessful aspects of the demonstration.
3. Evaluate the demonstration's achievement of FTA and agency goals.
4. Assess the applicability of the demonstration results to other sites.

Specific impacts the FTA would like to examine are listed below. The transit agencies, local transportation organizations and state and local governments may add to this list.

1. The degree to which bus speeds and schedule adherence improve.
2. The degree to which ridership increases due to improved bus speeds, schedule adherence and convenience.
3. The effect on other traffic.
4. The effect of each of the components of Bus Rapid Transit on bus speed and other traffic.
5. The benefits of ITS/APTS applications to the demonstration.
6. The effect of Bus Rapid Transit on land use and development.

To meet these objectives, it will be necessary to collect sufficient data on several aspects of the projects, as described in the following section. The data collected should derive from the measures necessary to calculate the anticipated project impacts (costs, benefits, and transfers). The expected project impacts (listed above) are discussed in Section 4. Program benefits can be achieved by using a combination of several potential BRT features. Section 5 outlines the BRT features and their expected impacts. FTA would like to know the separate impact of each feature. However, it is also possible that some features work better in combination with others as a system. The synergy among BRT features is an important element of the evaluation. Section 6 provides an overview of the evaluation process.

4. Project Impacts

As described in the program announcement, the BRT program is designed to provide the rest of the nation with information for considering BRT in the planning process and for engineering, designing, and implementing bus rapid transit projects. A thorough analysis of the impacts of the projects in the demonstration program will be essential for this purpose. Impacts can be classified as costs, benefits or transfers.

4.1 Travel Time

Passenger travel time is a key parameter. It is important to consider total trip time, which consists of access, wait, transfer, in-vehicle, and egress time. Further, we know that most people attach different values of time to the different components, with in-vehicle time felt to be the least onerous, and waiting time the most onerous. Because travel times are inherently stochastic, one should consider the distribution of travel times, characterized (at a minimum) by both a mean and a standard deviation. Further, because travel times vary by origin and destination and time of day, the measuring procedure should consider these and other forms of systematic variation in travel time. Reducing travel time provides a direct benefit to passengers and is also a principal means of attracting more passengers.

The "reliability" of service can either be seen as an independent parameter or as a component of the measurement of travel time. Irregular service leads to increases in mean waiting times. By definition, irregular service also increases the variance of travel times, which may in itself be an important factor. For example, if a bus is usually on time, but occasionally is very late, some riders may seek other means of transportation rather than being late.

Irregular service also affects vehicle loading. Heavy "crush" loads due to gaps in service can lead to further delays, as boarding and alighting are slowed. At the extreme, passengers are rejected and must wait until the next vehicle arrives. Transit headways are inherently unstable; a control strategy is generally required to insure regular service and reduce waiting times. Punctual service is also important when transfers are common.

Reducing the number of stops decreases in-vehicle time but increases access time. This trade-off must be evaluated carefully. On the other hand, reducing vehicle dwell time (the time spent at a stop while passengers are boarding or alighting or while waiting to get into traffic) has no downside. Reductions in dwell time can come from changes in vehicle design, fare collection policy, stop location, and stop design. A "priority merge" rule can speed the movement of buses back into the traffic lane. Improvements in the regularity of service can also reduce dwell time by reducing the incidence of crush loading.

4.2 Efficiency and Productivity

Reductions in travel time allow transit agencies to provide the same amount of service with fewer operator and vehicle hours. This improves transit efficiency and productivity. However, these savings may be realized only when the changes in travel time are large, since other constraints on the deployment of resources may prevent a reduction in the labor force or in the fleet size. It may be possible to redeploy resources, i.e., to use the faster speeds to provide additional service.

4.3 Ridership

Travel time is perhaps the most important determinant of transit ridership (along with out-of-pocket costs such as fares or parking costs avoided). Reductions in travel time will generally increase transit passenger trips.

4.4 Non-users

Some BRT policies may have either positive or negative effects on non-users of transit. Giving transit priority in terms of street design, traffic signals, or merging may increase travel times for other road users. On the other hand, such measures may actually reduce travel times for non-transit users. For example, eliminating on-street parking (even just in the peak hour in the peak direction) may disproportionately benefit transit users but improve travel for all road users. This change of course must be balanced against the cost of losing on-street parking.

4.5 Land Use

The structure of the urban environment can have a dramatic effect on people's willingness to use public transit. One component of the BRT evaluation will examine the extent to which transit-supportive land use policies can be instituted along with changes in transit service. These policies include those which make the pedestrian environment friendlier, and which encourage a range of mixed uses adjacent to transit. They may also permit more intense development near high-capacity transit stops. Land use policies may have ancillary benefits (permitting more low-cost housing or improving the quality of the walking experience). However, their primary transit benefit is their effect on current and future levels of transit ridership.

5. BRT Features

There is a wide range of features which are used in BRT projects to achieve some of the benefits described in the previous section. Every project relies on a combination of features, but no two projects use the same combination. The deployment of features may be consecutive instead of simultaneous. Consecutive (or parallel on different routes) deployment aids the evaluation process since it facilitates comparing the individual impacts of different features. However, another aspect of the evaluation will be to consider the joint impacts of features deployed in combination. Some features may work best when combined with others. The following is a list of some of the major BRT features and their expected impacts.

5.1 Express Rights of Way

Rapid transit service can be provided on bus-only roadways (busways), on bus lanes or HOV lanes on expressways, or on uncongested expressways. Where expressway ramp metering is in place, buses can sometimes use separate queue by-pass lanes. These facilities are primarily designed to reduce in-vehicle time. However, it is possible that access time may increase due to the location of available right-of-way and the spacing between stops. A busway is the type of facility which is most amenable to intensified station area development.

5.2 Bus Lanes on Arterials

To improve movement through surface streets, bus-only lanes can be designated. These lanes can either be permanent or only in effect during certain periods (typically peak hours). They can be with-flow or against the flow of normal traffic (contraflow). In some instances two bus-only lanes are preferable (typically the curb lane and the next lane over); this permits buses to pass each other easily.

5.3 Stop Location, Design and Spacing

Fewer stops can mean faster speeds, but at the expense of increased access time or waiting time (as in skip-stop service). The location of stops can affect travel time, given the placement of traffic signals, on-street parking, and other features which can delay transit. Stops can also be designed to reduce dwell time; for example, if the bus stops in the stream of traffic (either in the curb lane or along a stop which bulbs out into the parking lane), less time is lost in restarting. Stops should also be designed to improve passenger personal security and comfort.

5.4 Traffic Signal Priority

Traffic signals have a large effect on bus travel time, both its mean and its variance. Adjusting signals to benefit transit, either passively or actively, can have large benefits for transit users. Recent advances in ITS technology enable cost-effective deployment of automatic vehicle location and identification systems which permit the use of sophisticated algorithms for controlling traffic signals based on real-time transit schedule adherence and traffic volumes. Signal priority can be combined with short bus-only lanes to enable buses to jump out in front of waiting traffic.

5.5 Improved Vehicle Design

Vehicle design can be optimized to reduce dwell time. Low-floor vehicles have been shown to reduce boarding and alighting time. Wider doors, or more doors per vehicle, can further reduce dwell time, especially if combined with a fare payment policy which seeks to reduce the use of cash. Providing sufficient aisle space can also reduce dwell time when loads are high. When demand is very high, the use of larger vehicles, such as articulated buses, can also be crucial to maintain dwell time at a minimum.

5.6 Improved Fare Collection

Fare payment is one of the largest components of dwell time. Putting in place incentives to reduce the use of cash can reduce boarding time greatly. SMART card technology can also reduce fare payment time while improving the reliability of fare collection and the recording of boarding data.

5.7 Improved Passenger Information

Providing a high level of information at stops, on board vehicles, and away from the system is important in attracting and maintaining riders. Reducing the number of stops can go along with improving the quality of each stop, including the provision of information. Automatic vehicle location systems can be used to provide real-time data on next vehicle arrival times. New technologies such as the World Wide Web, cell phones, and cable TV can be used for remote access of customized transit information.

5.8 Improved Transit Marketing

Marketing is important to make potential riders aware of improved services. In addition, providing a consistent and visible identity for BRT-type service can increase public acceptance and awareness. Such marketing strategies might provide a consistent name, logo, and color schemes on vehicles, stops, publications, and press releases. Marketing can be combined with passenger information to provide a convenient, user-friendly service.

5.8 Transit-supportive Land Use Policies

Complementary land use policies can help increase transit ridership as land uses change over time. As mentioned, such policies can help to maintain or increase density and diversity of land uses, pedestrian-friendly road design, and pedestrian-oriented land uses (e.g., with parking behind structures). Moreover, land use policies can help facilitate the incorporation of transit in suburban areas. For example, subdivisions can be designed to provide more convenient transit access. Shopping malls can be designed around a transit station, rather than having transit stops at the periphery of vast parking areas.

6. Overview of the Evaluation Process

The evaluation process can be thought of conceptually as a link between the BRT demonstrations and technology transfer portions of the Bus Rapid Transit Demonstration Program. That is, it serves as a bridge between the implementation of a Bus Rapid Transit system at a particular site and the understanding of its actual performance at that site as well as its potential effectiveness in other locales. The quality of the evaluation process directly influences the accuracy and perceptiveness of the demonstration assessment and ultimately affects the applicability and transferability of the findings.

The figure below is a flow diagram representing the evaluation process for a Bus Rapid Transit demonstration. The diagram is divided into four major sections: the evaluation frame of reference, evaluation planning, evaluation implementation, and potential evaluation spin-offs. The first and fourth sections can be thought of, respectively, as input to and output from the active phases of the evaluation process, which are planning and implementation. A discussion of each of the four sections follows.

6.1 Evaluation Frame of Reference

The four elements of the evaluation frame of reference are: the scope of the Bus Rapid Transit demonstration; the FTA Bus Rapid Transit Program goals and objectives (described above); and external influences. The frame of reference sets the stage for the evaluation.

The scope of the BRT demonstration refers to the comprehensiveness of the project: which of the features described above are part of the demonstration; how extensive is the demonstration site; what agencies are participating.

External influences refer to circumstances external to the demonstration that may affect the demonstration's performance. The effects of such things as major increases or decreases in population, economic recessions or booms, major highway construction projects, and natural and other disasters can be easily confounded with the effects of the demonstration. For example, a major urban redevelopment project may attract new residents and new bus riders regardless of the improvements in service due to the BRT demonstration. To the extent possible, the evaluation analysis must endeavor to control for these influences or separate these effects.

6.2 Evaluation Planning

Evaluation planning translates the evaluation frame of reference into a detailed, structured program for conducting the evaluation. This should be completed long before the demonstration begins to allow for adequate time to collect "before" or baseline data for measuring demonstration performance.

The evaluator will have to collect a significant amount of data. In many cases, these data will be items already normally collected by the transit agency. In consultation with the transit agency, the evaluator will develop a data collection plan customized for that agency's operations and designed to insure that all necessary information is collected and that the collection process is as cost-effective as possible. The transit agency will receive a BRT Demonstration Program grant from the FTA to cover the costs of administering and implementing the data collection plan.

The data collection plan will address these elements:

1. Baseline data to measure the "before" case.
2. Control cases to insure that there are no confounding effects over time as the project is implemented. The control could be a comparable route where no BRT treatments are planned, or the rest of the system as a whole.
3. Statistical sampling techniques to reduce the amount of data to be collected while insuring that sample sizes are adequate for cases that are known to be non-homogenous, e.g., different times of day, days of week, and passenger origins and destinations.
4. Already available (and historical) data, such as data collected for the National Transit Database or for agency service and operations planning.
5. The number and availability of personnel needed to collect the data.
6. Careful analysis of the specific measures needed to calculate project benefits and costs (and not collecting data on measures not needed).
7. With respect to cost data, distinguishing between recurring and non-recurring costs.
8. Sufficient "after" data to account for the delay between service changes and ridership responses.
9. Consistent reporting of data across all projects in the Bus Rapid Transit Demonstration Program.

6.3 Evaluation Implementation

The evaluation implementation phase is the period during which the evaluation plan is implemented. Activities during this phase include the collection and analysis of data relative to project goals and issues, the collection and analysis of data on site characteristics, the compilation of a chronology describing the implementation and operation of the demonstration, and the recording of external factors which might influence operational test findings and results.

This phase not only generates information on which the final assessment of the demonstration is based but also provides feedback information relative to ongoing transit operations. The ongoing evaluation activities, while adding to the cumulative body of quantitative and qualitative information regarding the project impacts, provide interim indications of costs and functions of BRT features and the preliminary effects of these features on transit system performance. These interim findings may serve as useful input to the local agency responsible for implementing and operating the demonstration by suggesting the need for operational modifications, as well as to other transit agencies in the consortium who are just embarking on their demonstrations.

The culmination of the evaluation implementation phase is a final evaluation report, which presents the following types of findings:

1. Evaluation of the project in terms of its attainment of relevant BRT Demonstration Program goals and objectives.
2. Insight into project issues associated with operational feasibility and characteristics of the BRT features.
3. Assessment of the influence of site-specific characteristics and external factors on demonstration results.
4. Lessons learned, based on practical experience, relative to the implementation of the BRT system (possibly to include recommendations for project modifications in the demonstration site or for future implementations in other locales).
5. Appraisal of the evaluation procedures employed in terms of effectiveness, cost, accuracy, etc.

In essence, this report presents an assessment of the impact of the BRT system at the site and provides guidance for the transferability of results to other locales.

6.4 Potential Evaluation Spin-offs

It is anticipated that each BRT demonstration will give rise to potential implementation and analytical spin-offs. The final evaluation report, while essentially documenting the history and effects of a single project, also serves the broader function of increasing the understanding of and stimulating the application of the demonstrated BRT features and technologies in other localities. Information presented in the report provides a versatile basis for comparing the effects of a particular BRT feature with those of other similar projects, suggesting modifications to the applications for future use, and predicting the effectiveness and utility of the BRT features in other cities. In addition, they may provide input to cross-cutting reports using examples from all projects participating in the consortium. Moreover, the report's assessment of project evaluation procedures can serve as a stimulus for improving the state-of-the-art of evaluation techniques. These broader functions of the final evaluation report generally materialize after the demonstration period.

7. Evaluation Guidelines

The FTA is preparing Evaluation Guidelines for planning, implementing and reporting the findings of sites participating in the Bus Rapid Transit Demonstration Program. The guidelines will foster consistency of evaluation philosophy and techniques, and comparability and transferability of results to improve the quality and utility of information obtained from the Program. The guidelines will not, however, cover every conceivable situation that will arise in the demonstrations. The evaluator will use them as a guide, but must tailor each plan to the individual site being evaluated.

To assure that adequate "before" data are collected for each site, the FTA will select an evaluator and, when possible, initiate the evaluation process at least one year before the anticipated date of BRT Demonstration implementation.