This is the accessible text file for GAO report number GAO-05-423SP entitled 'Highlights Of An Expert Panel: The Benefits and Costs of Highway and Transit Investments' which was released on May 9, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. U.S. Government Accountability Office: Highlights Of An Expert Panel: The Benefits and Costs of Highway and Transit Investments: May 2005: GAO-05-423SP: GAO Highlights: Highlights of GAO-05-423SP: Why GAO Convened This Expert Panel: The nation's economy and its citizens' quality of life depend on our transportation system. While all government levels have made significant investments in transportation, projections of future passenger and freight travel indicate that considerable investment will be needed to maintain the system. However, this comes amid growing concern about the size of the federal budget deficit and increasing demands on state and local government revenue. As a result, careful decisions will need to be made to ensure that transportation investments maximize the benefits of each dollar invested. The House Appropriations Committee report accompanying the fiscal year 2004 Departments of Transportation and Treasury and Independent Agencies Appropriations Bill, required GAO to review the benefits and costs of various transportation modes. (See GAO-05-172.) As part of this study, GAO convened an expert panel that included some of the leading transportation economists and practitioners from throughout the nation. The panel discussed the benefits and costs of highway and transit investments. What Participants Said: GAO asked expert panel participants to discuss how to conceptualize, measure, improve, and use information about the benefits and costs of highway and transit investments. The expert panel was not designed to reach a consensus on these issues, but several themes emerged from the panel's discussion, including the following: * Benefit-cost analysis can be a useful tool to inform transportation investment decisions. * Requiring benefit-cost analysis can be useful if it is fully integrated into the decision making process and not seen as a compliance checklist. * Transportation investments seldom are compared across modes. * Better analytic tools are needed to evaluate land use and distributional impacts of investments. * Quality of state and local transportation data needs to be improved so that travel models can accurately predict patterns, trends, and needs. www.gao.gov/cgi-bin/getrpt?GAO-05-423SP. To view the full product, including the scope and methodology, click on the link above. For more information, contact Katherine Siggerud at (202) 512-2834 or siggerudk@gao.gov. [End of section] Contents: Abbreviations: Message from the Comptroller General of the United States: Introduction: Expert Panel on the Benefits and Costs of Transit and Highway Investments: Highlights of the Expert Panel Discussion: Appendix I: Scope and Methodology: Appendix II: Overview of Expert Panel and Opening Remarks: Appendix III: How Should We Think About Transportation Benefits and Costs? Opening Comments by Dr. Lewis and Dr. Wachs: Panel Discussion: Summary of Panel Responses to Audience Questions: Appendix IV: How Are Benefits and Costs of Transit and Highway Investments Best Measured? Opening Comments by Dr. Small and Dr. Pickrell: Panel Discussion: Summary of Panel Responses to Audience Questions: Appendix V: How Could Benefit-Cost Analysis Be Improved? Opening Comments by Dr. Forkenbrock and Dr. Kirby: Panel Discussion: Summary of Panel Responses to Audience Questions: Appendix VI: How Can Benefit-Cost Analysis Be Most Useful In Investment Decisions? Opening Comments by Dr. Meyer and Dr. Gomez-Ibanez: Panel Discussion: Summary of Panel Responses to Audience Questions: Appendix VII: Panelists' Closing Remarks: Appendix VIII: Selected Bibliography and Related GAO Products: Appendix IX: GAO Contacts and Acknowledgements: GAO Contacts: Acknowledgments: Endnotes: Abbreviations: AASHTO: American Association of State Highway and Transportation Officials: CAAA: Clean Air Act Amendments: CMAQ: Congestion Mitigation and Air Quality: DOT: Department of Transportation: EIS: Environmental Impact Statement: EPA: Environmental Protection Agency: FHWA: Federal Highway Administration: FTA: Federal Transit Administration: GDP: Gross Domestic Product: GPS: Global Positioning System: HOT: High Occupancy Toll: HOV: High Occupancy Vehicle: ISTEA: Intermodal Surface Transportation Efficiency Act: MPO: Metropolitan Planning Organization: NAS: National Academy of Sciences: NCHRP: National Cooperative Highway Research Program: NEPA: National Environmental Policy Act: OMB: Office of Management and Budget: ROI: Return on Investment: TANF: Temporary Assistance for Needy Families: TCRP: Transit Cooperative Research Program: TEA-21: Transportation Equity Act for the 21st Century: TRB: Transportation Research Board: Message from the Comptroller General of the United States: In speeches and presentations over the past several years, I have called attention to our large and growing long-term fiscal challenge and the risks it poses to our nation's future. Simply put, our nation's fiscal policy is on an unsustainable course. As long-term budget simulations by GAO and others show, we face a large and growing structural deficit over the long term due primarily to known demographic trends and rising health costs. These trends are compounded by the presence of near-term deficits arising from new discretionary and mandatory spending as well as lower revenues as a share of the economy. Continuing on this unsustainable fiscal path will gradually erode and may suddenly damage our economy, standard of living, and ultimately our national security. Given the size of our projected deficit, we will not be able to grow our way out of this problem--tough choices will be required. We need nothing less than a fundamental reexamination of all major existing spending and tax policies and priorities. While prompted by fiscal necessity, such a fundamental review of major program and policy areas also serves the vital function of updating the federal government's programs and priorities to meet current and future challenges. Many current federal programs and policies were designed decades ago to respond to trends and challenges that existed at the time of their creation. The transportation sector is one of many areas where emerging challenges necessitate difficult decisions about investments and priorities. I want to thank the distinguished experts who participated in our panel on the benefits and costs of highway and transit investments for their willingness to share their knowledge and time to examine issues with immediate importance and serious concern. Their insights about maximizing the benefits of federal dollars invested in transportation will be of value to Congress and the transportation community. I look forward to working with the panelists and others on this and other issues of mutual interest and concern in the future. Signed by: David M. Walker: Comptroller General of the United States: [End of section] Introduction: Expert Panel on the Benefits and Costs of Transit and Highway Investments: The nation's economic vitality and its citizens' quality of life depend substantially on the soundness, security, and availability of its transportation system. The transportation system provides people with access to goods, services, recreation, and jobs; provides businesses with access to materials, markets, and people; and promotes the movement of personnel and material to meet national defense needs. Given the importance of the transportation system, all levels of government have made significant investments in the system. However, future decisions about investments in the transportation system are set to collide with new realities and emerging trends. In particular: * Securing funding for transportation investments is becoming increasingly difficult. Federal transportation grant programs-- including the nation's highways and transit programs--are funded by the Highway Trust Fund. Revenues to the Highway Trust Fund are drawn from fuel taxes and user fees. The purchasing power of these revenues is declining, and future fuel tax revenues could be further eroded by the increasing fuel efficiency of vehicles. Many experts question whether the current financing scheme for transportation is ultimately sustainable. As a result, decision makers are increasingly looking more to the general fund to finance transportation programs, and state and local governments are increasingly relying on property and sales taxes to fund transportation improvements. Attempts to secure funding for transportation from these other sources come amid growing concerns about the size of federal budget deficits and future Social Security and Medicare commitments that will consume a greater share of the nation's resources. Moreover, transportation faces increasing competition from education, Medicaid, and other public uses for state and local government revenues.[Footnote 1] * The gap between travel needs and the transportation system's condition and capacity is growing. Increasing passenger and freight travel has generated substantial congestion throughout the national transportation system. Travel projections indicate that considerable investment will be needed to prevent congestion from overwhelming the system, while maintaining system safety and condition. The Department of Transportation (DOT) estimated that the nation's highway and transit systems will require $90.7 billion in annual capital investment through 2020 to maintain their current level of conditions and performance, while up to $127.5 billion in annual investment would be required to improve conditions and performance.[Footnote 2] * The transportation investment decision-making process is increasingly complex. Transportation investment decisions are inextricably linked with land use, economic, environmental, and energy policy concerns, among other things. Therefore, when making investment decisions, decision makers must consider a number of factors as well as the diverse, and sometimes conflicting, interests of numerous stakeholders. For example, the Intermodal Surface Transportation Efficiency Act of 1991 (ISTEA)[Footnote 3] and the Transportation Equity Act for the 21st Century (TEA-21)[Footnote 4] require that a number of factors-- including safety, environmental impacts, system connectivity, and accessibility--be considered in investment decisions and that state and local transportation agencies involve numerous stakeholders in the decision-making process. ISTEA and TEA-21 also gave local and state transportation agencies greater discretion in planning for and selecting transportation investments that meet local needs and priorities. As a result, the transportation investment decision-making process has been broadened to a wider range of viewpoints and interests. Moreover, new security imperatives in a world after September 11, 2001, present additional challenges for the transportation system that must be considered in the decision-making process. These trends raise questions about how to make transportation investment decisions in an increasingly fiscally constrained and complex environment. GAO and other federal agencies--including the Office of Management and Budget (OMB) and DOT--have identified benefit- cost analysis as a useful tool for integrating the social, environmental, economic, and other effects of investment alternatives and for helping decision makers identify projects with the greatest net benefits. In addition, the systematic process of benefit-cost analysis helps decision makers organize and evaluate information about, and determine trade-offs between, alternatives. The use of benefit-cost analysis in transportation investment decision making was the subject of an expert panel that GAO convened on June 28, 2004, to discuss four key issues--how to conceptualize, measure, improve, and use information about benefits and costs of highway and transit investments. We convened the expert panel, in collaboration with the National Academy of Sciences, as part of a larger study of the benefits and costs of transit and highway investments. The panel included top transportation economists and practitioners from throughout the country, including David J. Forkenbrock, Jose A. Gomez- Ibanez, Ronald F. Kirby, David L. Lewis, Michael D. Meyer, Donald Pickerell, Kenneth A. Small, Brian D. Taylor, and Martin Wachs. (See app. I for the methodology we used in convening the panel and a profile of each panelist.) We included the major themes that emerged from the panel in our January report to the House and Senate Appropriations Committees.[Footnote 5] However, the panel produced many additional important insights that were beyond the focus of our January 2005 report. Given the importance of these issues to decision makers and the transportation community, we decided to publish a separate report devoted exclusively to the results of the panel. We conducted the work to prepare this report from February through April 2005 according to generally accepted government auditing standards. Following is a summary of the discussion among the panel participants. The summary reflects the major themes that surfaced at the panel, and we used boldface type in the report to highlight points that these experts emphasized. Appendixes II to VII contain an edited transcript of the panel's discussion as well as subsequent comments received from the panelists based on a draft of this report. The views expressed by the panelists do not necessarily represent the views of GAO or the National Academy of Sciences. Appendix VIII contains a select bibliography and a list of related GAO products. This report will be posted on our Web site at www.gao.gov. For additional information on our work related to transportation decision making, please contact Katherine Siggerud on (202) 512 2834 or at siggerudk@gao.gov. Key contributors to this report are listed in appendix IX. [End of section] Highlights of the Expert Panel Discussion: Several themes emerged as the panelists responded to the four major issues that we presented for discussion--(1) conceptualizing, (2) measuring, (3) improving, and (4) using information about the benefits and costs of highway and transit investments. Although the expert panel was not designed to reach a consensus on these issues or specific questions that we presented, a number of themes emerged from the panel's discussion, as shown in table 1. Table 1: Major Themes from the Expert Panel Discussion: * Benefit-cost analysis can be a useful tool to inform transportation investment decisions. * Transportation investments are not often compared across modes. * Requiring benefit-cost analysis can be useful if it is fully integrated into the decision-making process and not seen as a compliance checklist. * Better analytic tools are needed to analyze land use and distributional impacts of transportation investments. * Quality of state and local transportation data needs to be improved so that travel models can accurately predict patterns, trends, and needs. Source: GAO analysis of expert panel discussion. [End of table] * Benefit-cost analysis can be a useful tool to inform transportation investment decisions. Benefit-cost analysis can provide important information to transportation decision makers about transportation investments and a structure for discussing benefits and costs of alternative investments with public and private stakeholders. In particular, it provides an analytic framework that decision makers can use to consider a range of factors in a systematic manner and clarifies what is and is not known about the impacts of a transportation project. However, efforts to increase the use of benefit-cost analysis need to be tempered with the knowledge that the results of benefit-cost analysis represent only one factor of many that are considered in investment decision making. Factors such as federal funding, public comment, limitations imposed by existing infrastructure, and political considerations also influence investment decisions.[Footnote 6] * Transportation investments are not often compared across modes. Alternatives in other modes are seldom systematically analyzed to determine how efficiently and effectively they could meet the transportation need. The highly compartmentalized structure and funding of federal highway and transit programs work against an advantage of benefit-cost analysis--the ability to evaluate how well alternative investments meet transportation problems. Separations between federal programs and funds give state, regional, and local agencies little incentive to systematically compare the trade-offs between investing in different transportation alternatives to meet passenger and freight travel needs because funding can be tied to certain programs or types of projects. In addition, the modal structure of federal programs gives rise to advocacy for specific modes or investments. * Requiring benefit-cost analysis can be useful if it is fully integrated into the decision-making process and not seen as a compliance checklist. Since systematic analyses of the benefits and costs of highway and transit investments are not often conducted voluntarily,[Footnote 7] requiring a benefit-cost analysis for new highway and transit investments could be useful. However, past experience with federal benefit-cost analysis requirements shows that they can either be treated in a pro forma way or "gamed" by the affected agencies. This experience indicates that mandates alone are not sufficient. Both incentives to conduct analysis and enforcement mechanisms would be needed to ensure that the analytic requirement is fully integrated into the decision-making process, thereby ensuring meaningful compliance. Experts noted that lessons can be learned from other federal analytic requirements. One expert also noted that the National Environmental Policy Act of 1969 (NEPA) [Footnote 8] requirements are not typically manipulated because such manipulation could result in a lawsuit. * Better analytic tools are needed to evaluate land use and distributional impacts of transportation investments. Land use impacts are often major drivers of investment choices. However, benefit-cost analysis and other types of economic analysis usually pay limited attention to land use issues, in part, because land-use issues--as well as other indirect benefits--are difficult to estimate. The panel also highlighted the importance of taking into account which groups benefit from a project and which bear the costs. Although the distribution of transportation investments' benefits and costs is an important local concern, it is frequently not considered adequately in the evaluation of a project's benefits and costs. * Quality of state and local transportation data needs to be improved so that travel models can accurately predict patterns, trends, and needs. Local and state transportation agencies require valid, reliable data in order to conduct analyses, including benefit-cost analysis. Yet, experts expressed concerns about the quality of local and state transportation data. Data quality is a pivotal concern in transportation modeling, as the available data provide critical input for travel models. For example, data about traffic flow throughout the day, rather than at a single time, are crucial to producing valid representations of travel needs and problems. However, reliable and complete data are not always available--which can result in forecasting errors. Collecting the data needed for modeling is growing more expensive and difficult. For instance, a home survey of travel habits, which identifies basic transportation needs and travel patterns of a region and is the foundation of transportation modeling, is now beyond most local transportation agencies' annual budgets, according to one expert. [End of section] Appendix I: Scope and Methodology: We contracted with the National Academy of Sciences (NAS) to convene a balanced, diverse panel of experts to discuss the use of benefit-cost analysis in highway and transit project decision making and gather views about options to improve the information available to decision makers. The NAS Transportation Research Board (TRB) identified potential panelists who were knowledgeable about benefit-cost analysis, transportation policy and planning, highway and transit use, and transportation decision making. We worked closely with TRB to select panelists who could adequately respond to our general and specific questions about conceptualizing, measuring, improving, and using benefit and cost information in investment decisions. In keeping with NAS policy, the panelists were invited to provide their individual views, and the panel was not designed to reach a consensus on any of the issues that we asked them to discuss. We also asked the panelists to submit two published articles related to the subject, which were disseminated to the audience the day of the panel. (See app. VIII for the list of articles submitted by the panelists.) The panelists convened at the National Academy of Sciences' Keck Center in Washington, D.C., on June 28, 2004, after reviewing discussion questions that we provided in advance. To start the day, the panel moderator, Brian Taylor of the University of California, Los Angeles, provided an overview of the issues to be discussed; and during the remainder of the day, the panelists addressed the questions we had provided for their consideration. We did not verify the panelists' statements, although we did ask the panelists, in some instances, to clarify certain details. The views expressed by the panelists do not necessarily represent the views of GAO or NAS. After the expert panel was conducted, we used a content analysis to systematically analyze a transcript of the panel's discussion in order to identify each expert's views on key questions, and we used boldface type in the report to highlight points that they emphasized. We also used the content analysis to highlight principal themes that emerged from the panel's discussion. To ensure that we accurately represented the panelists' comments in our report, we provided each panelist the opportunity to review and comment on the edited transcript. We incorporated changes or clarifications provided by the panelists to the draft report. Finally, we added endnotes to the transcript to define terminology used by the panelists, where appropriate, and to reference cited publications, laws, and programs. The discussion summarized in this report should be interpreted in the context of two key limitations and qualifications. First, although we were able to secure the participation of a balanced, highly qualified group of experts, there are other experts in this field who could not be included because of the need to limit the size of the panel. Although many points of view were represented, the panel was not representative of all potential views. Second, even though GAO, in cooperation with NAS, conducted preliminary research and heard from national experts in their fields, a day's conversation cannot represent the current practice in this vast arena. More thought, discussion, and research must be done to develop greater agreement on what we really know, what needs to be done, and how to do it. These two key limitations and qualifications provide contextual boundaries. Nevertheless, the panel provided a rich dialogue on the benefits and costs of transit and highway investments, and the panelists provided insightful comments in responding to the questions posed to the panel. Participants in the expert panel included the following: David J. Forkenbrock is Director of the Public Policy Center, Director of the Transportation Research Program, Professor in Urban and Regional Planning, and Professor in Civil and Environmental Engineering at the University of Iowa. His research and teaching interests include analytic methods in planning and transportation policy and planning. From 1995 through 1998, Dr. Forkenbrock chaired a National Research Council-appointed committee to review the Federal Highway Administration's (FHWA) Cost Allocation Study process. He is a member of the College of Fellows, American Institute of Certified Planners, and a lifetime National Associate of the National Academies. He is chairman of the TRB Committee for Review of Travel Demand Modeling by the Metropolitan Washington Council of Governments and a member of the TRB Committee for the Study of the Long-Term Viability of Fuel Taxes for Transportation Finance. In 2004, he received the first-ever TRB William S. Vickrey Award for Best Paper in Transportation Economics and Finance for his work on mileage-based road user charges. He received the Michael J. Brody Award for Excellence in Faculty Service to the University and the State, from the University of Iowa in 1996. He earned a Ph.D., from the University of Michigan; a Master of Urban Planning, from Wayne State University; and a B.A., from the University of Minnesota. Jose A. Gomez-Ibanez is Derek C. Bok Professor of Urban Planning and Public Policy at Harvard University's John F. Kennedy School of Government and Graduate School of Design. His research interests are primarily in the areas of transportation policy and urban development and privatization and regulation of infrastructure. He has served as a consultant for a variety of public agencies. His recent books include Regulating Infrastructure: Monopoly, Contracts, and Discretion; Regulation for Revenue: The Political Economy of Land Use Exactions (with Alan Altshuler); Going Private: The International Experience with Transport Privatization (with John R. Meyer); and Essays on Transport Policy and Economics (ed.) Ronald F. Kirby is Director of Transportation Planning for the Metropolitan Washington Area Council of Governments. He began his career in the United States as a Senior Research Associate with Planning Research Corporation. He joined the Urban Institute as a Senior Research Associate and became a Principal Research Associate and Director of Transportation Studies. He has served on several TRB committees and is currently a member of the TRB Executive Committee. He has a B.S. and a Ph.D., in applied mathematics, from the University of Adelaide, South Australia. David L. Lewis is President and CEO of HLB Decision Economics. His credits include a range of widely adopted applications in cost-benefit analysis, productivity measurement, risk analysis, and approaches to establishing public-private investment partnerships. He has authored three books, including Policy and Planning as Public Choice: Mass Transit in the United States (Ashgate Press), 1999. His past positions include Partner-in-Charge, Division of Economics and U.S. Operations, Hickling Corporation; Chief Economist, Office of the Auditor General of Canada; Executive Interchange Program and Principal Analyst, U.S. Congressional Budget Office, Congress of the United States; and Senior Economist and Director of the Office of Domestic Forecasting, Electricity Council. He has a Ph.D. and an M.S., in economics, from the London School of Economics; and a B.A., in economics, from the University of Maryland. Michael D. Meyer is Professor of Civil and Environmental Engineering at the Georgia Institute of Technology. Prior to coming to Georgia Tech in 1988, he was the Director of the Bureau of Transportation Planning and Development at the Massachusetts Department of Public Works for 5 years. Prior to his employment at the Massachusetts Department of Public Works, he was a professor in the civil engineering department of the Massachusetts Institute of Technology. His research interests include transportation planning and policy analysis, environmental impact assessment, analysis of transportation control measures, and intermodal and transit planning. He is a Professional Engineer in the State of Georgia, and a member of the American Society of Civil Engineers and the Institute of Transportation Engineers. He has chaired TRB's Task Force on Transportation Demand Management, the Public Policy Committee, the Committee on Education and Training, and the Statewide Multimodal Transportation Planning Committee. He is a former member of the National Research Council policy study Panel on Statistical Programs and Practices of the Bureau of Transportation Statistics. Currently, he is a member of TRB's Executive Committee and Standing Committee on Statewide Multimodal Transportation Planning. Donald Pickrell is DOT's Volpe Center's Chief Economist. Prior to joining DOT, he taught economics, transportation planning, and government regulation at Harvard University. While at the Volpe Center, he also was a lecturer in the Department of Civil Engineering at the Massachusetts Institute of Technology. He has authored over 100 published papers and research reports on various topics in transportation policy and planning, including transportation pricing; transit planning and finance; airline marketing and competition; travel demand forecasting; infrastructure investment and finance; and the relationships of travel behavior to land use, urban air quality, and potential climate change. He received his undergraduate degree in economics and mathematics from the University of California at San Diego; and Master's and Ph.D. degrees, in urban planning, from the University of California at Los Angeles. Kenneth A. Small is Professor of Economics at the University of California at Irvine, where he served 3 years as chair of the Department of Economics and 6 years as Associate Dean of Social Sciences. He previously taught at Princeton University and was a Research Associate at The Brookings Institution. He has written numerous books and articles on urban economics, transportation, public finance, and environmental economics. He serves on the editorial boards of several professional journals in the fields of urban and transportation studies and has served as coeditor or guest editor for four of those boards. In 1999, he received the Distinguished Member award of the Transport and Public Utilities Group of the American Economic Association. During 1999 to 2000, he held a Gilbert White Fellowship at Resources for the Future. He has served on two TRB policy study committees--the Committee for a Review of the Highway Cost Allocation Study and the Committee for a Study on Urban Transportation Congestion Pricing. Brian D. Taylor (Moderator) is Associate Professor of Urban Planning and Director of the Institute of Transportation Studies at the University of California at Los Angeles as well as Vice-Chair of the Urban Planning Department. His research centers on transportation finance and travel demographics. He has examined the politics of transportation finance, including the influence of finance on the development of metropolitan freeway systems and the effect of public transit subsidy programs on system performance and social equity. His research on the demographics of travel behavior has emphasized access- deprived populations, including women, racial-ethnic minorities, the disabled, and the poor. He also has explored relationships between transportation and urban form, with a focus on commuting and employment access for low-wage workers. Prior to coming to the University of California at Los Angeles in 1994, he was Assistant Professor in the Department of City and Regional Planning at the University of North Carolina at Chapel Hill. Prior to that, he was a Transportation Analyst with the Metropolitan Transportation Commission in Oakland, California. Martin Wachs is Professor of Civil and Environmental Engineering and City and Regional Planning, and Director of the Institute of Transportation Studies at the University of California at Berkeley. He was formerly Professor of Urban Planning and Director of the Institute of Transportation Studies at the University of California at Los Angeles where he served three terms as Head of the Urban Planning Program. Dr. Wachs' research interests include methods for evaluating alternative transportation projects; relationships among land use, transportation, and air quality; and fare and subsidy policies in urban transportation. Most recently, he chaired the TRB policy study Committee for the Study on Urban Transportation Congestion Pricing. He is past Chairman of the TRB Executive Committee. Dr. Wachs holds a Ph.D., in transportation planning, from Northwestern University. [End of section] Appendix II: Overview of Expert Panel and Opening Remarks: Brian D. Taylor is an Associate Professor of Urban Planning and Director of the Institute of Transportation Studies at the University of California at Los Angeles as well as Vice-Chair of the Urban Planning Department. Dr. TAYLOR (Moderator): To examine the use of benefit-cost analysis in transportation decision making, the panel will examine four issues-conceptualizing the benefits and costs of transit and highway investments; measuring benefits and costs; improving benefit-cost analysis as an evaluation tool; and using benefit-cost analysis to inform public decisions. Two panelists will offer comments to begin the discussion of each issue. A key question is implied in these four major issues that GAO asked the panel to discuss: Why does benefit-cost analysis play a relatively limited role in transportation decision making? I see three possible answers: * it's the wrong analytical tool-we have other, better tools for this purpose; * it's the right tool, but often improperly applied; or: * it's the right tool, but hard to apply-in other words, we need better data, a more sophisticated application of this tool, or a more formal incorporation of this tool into decision making. Regardless of what may explain the relatively limited role of benefit- cost analysis in transportation decision making, there are at least four unstated premises to this issue that warrant reflection: * first, that transportation investments sometimes are misguided; * second, that improper evaluations or failure to conduct evaluations have played a role in misguided investments; * third, that improved analyses can better inform transportation decision making (or, put another way, if decision makers have better information, it will be harder for them to make bad decisions); * and fourth, that better informed decision making can reduce the number of misguided transportation investments. Collectively, these premises suggest links between information, evaluation, and decision making that are far from settled in my view. The panel faces an important question: Will public officials actually find better, more transparent evaluations of the transportation merits of proposed projects "threatening" to the current, well-established processes of transportation decision making? I would contend that benefit and cost comparisons do, in fact, guide all public investments in transportation, but not in the way that students of benefit-cost analysis might expect. In practice, such comparisons center on geopolitical benefits and costs--that is, they concern bargaining by elected officials over the distribution of limited public resources. In such a world, transportation benefits and costs are secondary. Thus, geopolitical benefits and costs trump consideration of transportation benefits and costs, so that programs and projects become the ends of public investments, rather than means to transportation ends. Further, the rise of legislative earmarking,[Footnote 9] which bypasses many evaluation processes, increases the extent to which concerns over geopolitical distribution of benefits and costs trump transportation project analyses. * ISTEA in 1991 included earmarks for 40 rail transit projects. * TEA-21 in 1998 increased the number of earmarked rail transit projects to 191, many of which were in places not normally viewed as ideal environments for rail investments. * All three versions of legislation pending in Congress in June 2004 to reauthorize surface transportation programs contain significant increases in earmarking over TEA-21. Why the earmarking? First, earmarking bypasses evaluation processes that vest bureaucrats with significant authority over transportation investments. Second, most (though not all) earmarks are capital projects that provide good "ribbon-cutting" opportunities, and attendant media attention, for elected officials. Further, projects like new rail transit lines, highway bypasses, and maintenance facilities generate local economic benefits that are clear and unambiguous to both public officials and the people who elect them. But while earmarked projects may be the products of a careful geopolitical calculus, they may provide few transportation benefits in relation to their costs. Our goal is to recognize, and separate, our consideration of these two effects--political and transportation--in analyzing the benefits and costs of public investments in transportation. Sidebar: "Will public officials actually find better, more transparent evaluations of the transportation merits of proposed projects threatening to the current, well-established processes of transportation decision making?" -Dr. Taylor: [End of sidebar] Elected officials and transportation analysts think about transportation investments in different ways, and this is the source of conflict about analytical techniques. Transportation analysts and economists have long advised us to focus on the transportation effects of public investments, and not on the expenditure effects of such investments. The former concern whether and how public investments lower transportation costs--such as by reducing congestion, increasing safety, reducing emissions, etc.--and the latter concern the direct effects of spending public dollars to hire construction workers, pay truck drivers, and so forth. Indeed, most analysts would argue that transportation investments should be judged, first and foremost, on how they reduce transportation costs, rather than on their local expenditure effects. Such transportation benefits make it cheaper to produce current goods and services, make new forms of goods and services possible, and benefit the economy by lowering transportation costs for system users and society at large, as described in table 2. Table 2: Differing Views of Transportation Investments: Elected Officials-Expenditure Effects: Focus on how resources are collected from and distributed to jurisdictions, and, in the process, how these resources redistribute economic activity among jurisdictions. Analysts/Economists-Transportation Effects: Focus on how transportation improvements stimulate economic activity by lowering transportation costs, which allows current activities to be accomplished less expensively and makes new activities economically feasible. Source: Dr. Taylor's presentation at GAO's June 28, 2004 expert panel. [End of table] For transportation analysts, the redistributive effects of expenditures are largely a zero sum game. Although transportation expenditures can generate significant local economic activity, much of it is simply redistributed from other taxpayers and places that lost out in the geographic competition for subsidy dollars. From this point of view, policy makers are simply missing the point when they focus almost exclusively on the local expenditure effects of transportation investment decisions. Despite such admonitions from analysts, however, many elected officials and other policy makers view the transportation effects of public investments as abstract, arcane, and arbitrary. While a new freeway ramp metering project might smooth traffic flows, which in turn lower production costs for a particular set of firms, which in turn increase sales, which in turn add to total employment, such effects are difficult to unambiguously link to the highway investment. In contrast, the consequences of the public expenditures on transportation projects in a given congressional district are clear and unambiguous--dollars get spent, projects get built, people get hired. New highways and transit investments are dramatic and highly visible and generate economic activity, especially during construction. That much of this activity is simply shifted from taxpayers in other jurisdictions is almost beside the point to most elected officials. For most elected officials responsible for transportation taxation and spending, the overriding concern is with the equity of transportation funding among states, districts, and jurisdictions. Concerns over who pays and who receives are paramount. This concern ensures a political focus on the expenditure effects of transportation investments and makes it all but impossible for elected officials to consider the transportation effects of investments. From the perspective of most public officials, it's the transportation analysts and economists who miss the point by focusing on transportation effects and tools like benefit-cost analysis in making investment decisions. A Member of Congress from a western state, for example, may find a study showing that rail transit investments in a densely developed, older east coast city are likely to yield far greater transportation benefits than those in his/her city all but irrelevant to debates over the equitable geographic distribution of federal transportation funds. These divergent views pose several related questions: * How do public officials view the benefits and costs of transportation benefit-cost analyses? Are transparent evaluations of transportation benefits seen as conflicting with and a direct threat to the geopolitical logic of political bargaining? If so, does this conflict explain why many benefit-cost analyses are conducted after the fact to gather evidence to support decisions, and why many analyses are of an already preferred alternative and some straw men? In my experience as a metropolitan planner during the 1980s, it was evident that alternatives were selected very carefully to ensure that they would not be too effective in competing with the clearly preferred alternative. * If public officials perceive benefit-cost analyses as shifting decisionmaking power and authority to analysts, does this help to explain some of the criticisms leveled against the technique? For example, unpopular benefit-cost analyses frequently are dismissed for excluding factors that are difficult to measure. While such criticism may be well founded, the results often are not very sensitive to the excluded factors. But, this frequently is lost in "attack and defend" debates over unpopular analysis results. * How can we cope with deep conflicts over what constitutes good transportation systems and good cities? While most transportation analysts see lowering transportation costs (both for travelers and shippers, and for society at large) as a principal objective, many transportation activists and environmental advocates view declining transportation costs as a problem. This is a vexing, often unspoken issue that underlies many debates over benefit-cost analyses. * Can evaluations focus more on clearly defined problems and less on solutions to poorly defined problems? Analysts rarely are asked to generate and evaluate alternative approaches to address a transportation problem in the current project-focused political climate. Instead, they usually are asked to evaluate/compare poorly defined solutions--rail transit, increased highway capacity, high occupancy vehicle (HOV) lanes, or bus rapid transit--to poorly defined problems.[Footnote 10] [End of section] Appendix III: How Should We Think About Transportation Benefits and Costs? GAO Questions: * What types of benefits and costs are associated with public investments in transit and highways and how should they be reflected? * What types of externalities are associated with these investments? * What is known about cross-modal comparisons at the national, state, and/or local levels? Opening Comments by Dr. Lewis and Dr. Wachs: David L. Lewis is President and CEO of HLB Decision Economics: [See PDF for image] [End of figure] Dr. LEWIS: There are two key points about conceptualizing benefits and costs. * Analysts and economists need to help decision makers look at their choices, including highway versus transit choices, on a level playing field--something we palpably lack today. * We should reinvent benefit-cost analysis so that it facilitates decision by discussion. Benefit-cost analysis needs to shift from a study presented in a report and delivered by remote experts who stand aloof from the decision-making process to a facilitated analysis framework in which stakeholders can participate in formulating values. Decision makers have many single choices or combinations of choices, yet we rarely help them to look at their choices on a level playing field. As Dr. Taylor pointed out, decision makers very rarely and certainly never systematically ask for--nor do analysts provide--a comparative analysis of the payoffs associated with investment alternatives. These may be alternatives in design, scope, or mode pricing and various other alternatives. These also may be investments in education, health, or even tax reductions. Nor are alternatives analyzed in relation to timing--a consideration because there are many good projects whose time has not come. When Dulles Airport opened in 1963 it was empty. Today, it is unbelievably crowded. (See Fig. 1 for passenger traffic trends at Washington Dulles International Airport.) Sidebar: "Benefit-cost analysis needs to shift from a study presented in a report and delivered by remote experts who stand aloof from the decision-making process to a facilitated analysis framework in which stakeholders can participate in formulating values."-Dr. Lewis. [End of sidebar] Figure 1: Passenger Traffic at Washington Dulles International Airport, 1962 to 2004: [See PDF for image] Source: GAO presentation of data from the Metropolitan Washington Airports Authority. [End of figure] Dulles Airport was empty for the first 25 or 30 years of its life. Does that mean we were visionary in anticipating the huge crowds that would ultimately use it? No. We could have used those billions of dollars (in current prices) in much better ways in the meantime and still have beaten inflation by a lot in building the facility 20 or 30 years later. There are alternatives in scope, design, and time and ways to compare alternatives. Everybody understands and is generally comfortable with Return on Investment (ROI) calculations.[Footnote 11] Any and all options and combinations can be boiled down to their ROI's. Decision makers could be treated to a clear, honest portrayal of a "risk-adjusted" comparative ROI of widening a highway versus building a light rail line down the corridor, versus doing a bit of both, versus doing nothing, versus delaying bits and pieces of it, etc. Maximizing ROI is a good, very accessible way for most decision makers and stakeholders to appreciate how alternatives differ. Second, benefit-cost analysis needs to shift from studies delivered by remote experts who stand aloof from decision making to reinventing benefit-cost analysis as a means of facilitating decision by discussion. We analysts take fundamental values--the value of human life, the value of reducing environmental emissions and greenhouse gases, the value of a job--as data. We have sophisticated techniques for measuring how people feel about things and expressing those feelings in the form of people's willingness to pay monetary equivalent values by empirically measuring transactions in the marketplace and through survey data. But these values are not data. Some modern economists and philosophers, like Amartya Sen,[Footnote 12] argue convincingly that discussion is the melting pot in which values tend to form. Analysts and economists can help the public, decision makers, and stakeholders discuss values by using benefit-cost analysis as a powerful framework and facilitation tool. Welfare economics[Footnote 13] has provided an incredibly powerful way of thinking that people drink up when it's presented to them in a digestible format. The conceptual benefits and costs of transportation solutions to congestion, development, and mobility and the macroeconomic effects of these solutions on jobs, income, and the tax base--including the redistributional or expenditure impacts that Dr. Taylor discussed--can be laid out to enable people to discuss things in a logical, reasonable way. This process can help isolate the minority who wish to game the system or bend the discussion to suit a particular outcome. Economic analysis, benefit-cost analysis, welfare economics--whatever you want to call it--brings reason to a debate if it is transformed into a facilitation tool. Martin Wachs is Professor of Civil and Environmental Engineering and City and Regional Planning, and Director of the Institute of Transportation Studies at the University of California at Berkeley. [See PDF for image] [End of figure] Dr. WACHS: Two examples from California show the different poles at which benefit- cost analysis is being discussed and used. First, the California Department of Transportation (CALTRANS) provides a benefit-cost analysis template on its Web site. It assists local planners and decision makers with problems at the project level--an intersection, a small corridor, something in a metropolitan area, or a grade crossing. Project alternatives can be evaluated using CALTRANS's list of benefits and assumptions about the value of time, of a life, of property damage, of accidents, etc. It is possible to get an answer about the benefits of highway widening versus traffic signal timing improvement options. It is very useful to compare alternatives when you have limited resources and can approach decision making with an analytical framework that is readily available on line. However, enumerating benefits (such as time savings, which often is the largest benefit category) raises enormous questions and important assumptions that require answers. Do we believe that non-work and work travel should have the same value of time? We may not believe that they should, but we make assumptions and operationalize these assumptions on the Web site. Do we see the value of time as linear or nonlinear with respect to the amount of time saved? I do not--I cannot usefully use one minute saved in the same way that I can use 20 minutes saved. Do rich and poor people have the same value of time? I think not. Do we believe that the value of an old person injured or killed in an accident is the same as the value of a young person? Yet, these important, implied questions are addressed by assumptions and set aside for the purpose of analysis. At the narrow scale of an intersection or a mile, this does not do much harm. The second example from California is that benefit-cost analysis is being advanced as the appropriate way to debate a major state public policy issue. The question is whether California should build a high- speed rail system among San Diego, Los Angeles, the San Francisco Bay Area, and Sacramento at a cost of about $35 billion and 30 years to build. Voters will be asked to tax themselves by voting on a statewide proposition. The high-speed rail debate, couched in benefit-cost analysis terms, is highly politicized. Proponents state that the project will provide enormous time and travel time savings for high- speed rail users and car and air travelers, based upon assumptions about the growth of car and air travel over the next 20-30 years. Some say it will achieve smart growth by concentrating new community development for a population growth of 30 million people in these corridors, preserving open space, and reducing development and preserving agricultural land outside of these corridors. What does this really mean? Are there really large transfers involved? If one saves time using one mode, is this somehow a net saving for the state or a transfer of benefits from one system to another, one set of users to another, and/or one geographic area to another? We're told that one enormous benefit will be reduced air pollution and energy consumption. But Dr. Forkenbrock's article asks whether this is actually a secondary effect of travel time savings.[Footnote 14] How can we conceptualize the difference between costs that will accrue to those who use highways, air transportation, and rail 30 years from now aside from making rather heroic assumptions? Are we not saying it is the secondary effect that is very important in policy terms--the secondary effect of the principal effect? Isn't that all an artifact of the assumptions that we make? I have no difficulty making assumptions when I am comparing one intersection configuration to another because assumptions are necessary to get useful outcomes. Here, assumptions are being made about matters of such enormous ethical, moral, and political consequence that I am much more uncomfortable saying that benefits exceed costs by a substantial margin, as proponents say. Their argument is based upon assumptions that are reasonable if you are a proponent, but not reasonable if you are an opponent. This argument is entirely about redistribution, but the benefit-cost analysis is entirely about the benefits to whomsoever they accrue, while the total cost is borne by every state citizen. Dr. Lewis said in the paper he shared with us that the benefit-cost framework enables us to have a debate because it facilitates dialogue.[Footnote 15] But at what level? It does so at the intersection or corridor level where you might have bus rapid transit versus standard bus transit. But does the benefit-cost framework confound or clarify the dialogue about an enormously important statewide project? Can we focus on making benefit-cost analysis more transparent and useful so that it plays a positive role in such debates? Currently, benefit-cost analysis appears to be limiting rather than enhancing this debate. Panel Discussion: Dr. SMALL: Dr. Wachs's last remark raises the question of why highspeed rail proponents have couched their arguments in benefit-cost analysis terms, particularly if the analysis is overshadowed by politics or other considerations. If we can decide why other people are using benefit-cost analysis, perhaps we can figure out how we might structure it to make better decisions. Dr. MEYER: Conceptualizing benefits and costs depends on scale. At the intersection or perhaps corridor levels, benefit-cost analysis is a very important force in decisions. The benefits and costs are quite clear, but the congressional earmarking level is off this scale. My state and metropolitan government experience has indicated that geographic distribution and political considerations increase as the scale increases. We need to be careful in presenting benefit-cost analysis as the primary tool for determining trade-offs, because it is only one piece of information in decision making at a higher scale. In fact, cost effectiveness may be preferable to benefit-cost analysis. Dr. LEWIS: I disagree. The question of scale is a red herring because you can find benefit-cost analysis--not necessarily great applications of it--conducted at huge scales. The Three Gorges Dam, which changed the lives of millions of people along China's Yangtze River, became the framework for a great deal of political and divisive discussion. It was not a paragon of benefit-cost analysis and should have been done much better, according to many critics. But it did not cause controversy because of anything inherent in benefit-cost analysis theory or practice. Benefit-cost analysis is something we offer to help people to think through problems at any scale. If we lose political reality in doing that, it is because we are not looking at the options, alternatives, and complex policy arrangements that people want to include in benefit- cost analysis. Nothing about benefit-cost analysis limits the scope or range of choices that we look at--including comparing a rail system from Los Angeles to San Francisco to options that would reconcile the losses or redistributional problems that people far from the Los Angeles/San Francisco corridor might perceive. Dr. TAYLOR: Does the definition of potential benefits change or evolve at higher scales? Dr. LEWIS: What changes are functions of the scope, policy dimensions, and range of choice. The problem with California's high-speed rail analysis is that it is not compared to a baseline of not having that investment. Since many other things could happen to those transportation dollars, this should be explicit. Dr. KIRBY: Dr. Lewis's notion of benefit-cost analysis as an instrument of discursive democracy in the paper he shared with us is a truly valuable concept and a very constructive way for us to bring these techniques into decision making. Benefit-cost analysis is about more than just informing discussion--it also is about facilitating a decision. I also liked the statement that the objective of discursive democracy is to reach consensus without minority dissent, but I feel less optimism about that. In many major and even smaller projects, we have minority dissent that does not preclude a decision to move forward. Lawsuits also may affect decisions, but that is the nature of the process. Good analysis and communication with the public could resolve a lot of issues through facilitation and discussion. A major project to widen Washington's Wilson Bridge is an example of how information and facilitation can help get people on the same page about a problem. (See Fig. 2 for photographs of the Woodrow Wilson Bridge Project.) In this instance, opposing views of the problem were resolved by simply looking at the data. A citizen group was determined that the solution to congestion was building another bridge to handle all Miami to Boston truck traffic. This posed the empirical question-- what is the composition of traffic on the Wilson Bridge? Data from our consultants, models, surveys, and counts indicated that trucks were only 15 percent of bridge traffic. Only 4 percent of trucks were coming into the region and exiting at the other end of the region. The other 11 percent were trucks doing business here, with one origin or destination in the region--often at a local grocery store. The citizen group reacted with disbelief and decided to do its own traffic count. At the next meeting, they agreed with our data and our conclusion that this was primarily local traffic and very much a local issue. Figure 2: The Woodrow Wilson Bridge Project: [See PDF for image] Source: Woodrow Wilson Bridge project. The Woodrow Wilson Bridge Project is a 7-1/2 mile corridor that begins in Maryland and connects to Virginia by a bridge over the Potomac River. The project consists of the replacement of the existing Woodrow Wilson Bridge, among other things. Pictured on the left: Traffic crosses the existing structure as the new bridge rises in October 2004. Pictured on the right: One of two V-shaped piers that will support the draw span of the new bridge is constructed in December 2004. [End of figure] Dr. WACHS: This relates to the scale debate because it occurred at the scale of an individual bridge and there was something to count. It was very relevant to the success of the technique in this deliberative, democratic setting. If the debate were about the benefits of high-speed rail versus airport expansion over 40 years, could the citizens have done the same thing? Dr. KIRBY: It definitely would be more difficult--especially in the longer term. Regarding scale, it is very often the project's nature rather than scale that determines the degree to which different groups are engaged. A left turn lane to improve safety can be as contentious as a major highway if it moves traffic onto a local street. It will bring the neighbors out. A huge highway interchange project--hundreds of millions of dollars--in our metropolitan area generated virtually no public comment. This was because the project had virtually no local impact. Vast amounts of land around the interchange already were owned, and there were only a few houses that owners did not mind leaving that needed to be demolished. It is really a question of the impacts and whom the impacts affect. It just depends on the situation, as Dr. Meyer said. However, the idea of discursive democracy is relevant. Objectively presenting data analysis can help people focus on real issues, as opposed to arguing about unrealistic issues, even in longer term decisions. Dr. GOMEZ-IBANEZ: I echo Dr. Kirby's comments and make two points. First, Dr. Small raised one of the most interesting questions--why do proponents of California's rail transit system justify it by benefit- cost analysis when it is so hard to do a benefit-cost analysis of something so large and long-term? It may reflect the appeal of systematic thinking and rationality in modern Western culture, i.e., if you propose to spend $35 billion, you should have good reasons that you can explain. This is a very important, powerful leverage. It means that people are open to the challenge of making explicit their assumptions in thinking that this $35 billion investment is viable or in the public interest. They must appeal to something beyond parochial self-interest. Their appeal essentially must be similar to benefit-cost analysis-- benefits to society will be larger than costs. Sidebar: "One thing that discredits benefit-cost analysis is that it does not pay much attention to the land use effects of major transportation capital investments...transportation planners do not join that debate because they have such poor tools for forecasting the land use effects and they find it hard to adapt benefit-cost analysis to that context." -Dr. Gomez-Ibanez. [End of sidebar] Second, I agree with Dr. Kirby that scale does not mean less controversy, but also with Dr. Wachs that the broader the scale, the more difficult it is to apply benefit-cost analysis and reach a single number with confidence. This does not mean that benefit-cost analysis or something like it is not extremely helpful at a large scale. The California rail transit program requires listing and thinking systematically about a set of assumptions. * Is it true that you would concentrate all the growth in this corridor? * What does our experience suggest--is rail transit enough or is highspeed rail better? * What are the benefits of Smart Growth?[Footnote 16] * Since one benefit usually is reducing air pollution, would the project result in less driving? * Would it lead to less local infrastructure? * What does the literature say? * Are we going to save $35 billion on cheaper sewers, sidewalks, and narrower roads or not? Although you might not be able to reduce the discussion to a single benefit-cost ratio or internal rate of return, you would force both sides to be systematic about the assumptions that they are making and focus the discussion in an extremely helpful way. One thing that discredits benefit-cost analysis is that it does not pay much attention to the land use effects of major transportation capital investments. There may be enormous benefits from density or sprawl, depending on your side of the argument. But transportation planners do not join that debate because they have such poor tools for forecasting the land use effects and they find it hard to adapt benefit-cost analysis to that context. They understand the travel time savings, but how should we think about the land use impacts? Are land use impacts just a reflection of the travel time savings? If infrastructure is under-priced, should it be? Doing more land use evaluation of transportation investments rather than straight transportation evaluation would help because it would force an open discussion about the assumptions on both sides. Dr. FORKENBROCK: I agree and would build on these points with three words--visioning, scale, and assumptions. Visioning is the key. * What do we want our community to become? * What do we want our region to aspire to? * What sorts of economic progress are we pursuing through the project being considered? In Lewis Carroll's "Through The Looking Glass," one character asked another, how do I get there? The other character said, where do you want to go? Well, I don't know. Then how do you expect me to tell you how to get there? This is a difficult problem with benefit-cost analysis, i.e., what do we want our city to become? What urban form do we aspire to? If we spend all our time on the "big three measures"--the value/reliability of travel time, safety, and vehicle operating costs- -we can miss the whole point. We can get a nice benefit-cost ratio for the wrong project. It is something we have to worry a great deal about. Getting things right is a function of scale. It is much easier to vision what will happen with a left turn lane than with Boston's Central Artery or some other very large project that will have a big impact on urban form, travel patterns, activity patterns, and quality of life. (See Fig. 3 for photographs of Boston's Central Artery Project.) Questions about the valuation of external costs or the effects on air quality in an area, or about putting a value on air quality changes versus travel time saved, all bring us back to scale. We should spend more time identifying the problem and less time analyzing the wrong problem, especially as we get into big projects. Once we have done that, we need to worry about assumptions and attaching values to key parameters. This follows the major point on visioning. What are we trying to accomplish? What will be the long-term effects? Dr. LEWIS: I sense at least two broad questions on the table: what is the nature of benefit-cost analysis and what is the nature of transportation benefits and costs? They are two different questions-- both are very operational, practical, and important questions. Figure 3: Boston's Central Artery/Tunnel Project: [See PDF for image] Source: Central Artery/Tunnel Project. The Central Artery/Tunnel Project replaced the six-lane elevated highway (Central Artery Highway) that ran through downtown Boston, MA, with an eight-to-ten lane underground expressway directly beneath the existing road. The project spans almost 8 miles of highway, about half in tunnels. [End of figure] Sidebar: "Benefit-cost analysis gives us a frame within which to logically address…effects to which we attach value… and their costs and benefits."-Dr. Lewis. [End of sidebar] There is no reason why the answer to the first question cannot be that benefit-cost analysis is technically broad enough to frame a discussion about the kind of community we want, rather than, given the kind of community we want to be, which option is the most efficient and cost effective to get us there. Benefit-cost analysis is a framework to think about big questions. When it is all done, the accuracy of numbers becomes less relevant than the framework or the fact that a community has been able to systematically work through enough options to take a course of action that commands some broader support. But what is the nature of benefits and costs? We are hearing that in addition to traditional time, reliability, and safety effects, other effects have value and benefit-cost analysis is remiss in not dealing with them. If it is hard to predict the effect of a transportation program or project on achieving a land use outcome that the community values, then benefit-cost analysis is a good framework for admitting that the decision you take or delay has the following risks and rewards and that there are things we do and do not know. Benefit-cost analysis gives us a frame within which to logically address any and all of the effects to which we attach value, both positive and negative, and their costs and benefits. Dr. TAYLOR: What would happen if there is such profound disagreement about the visioning that Dr. Forkenbrock discussed that there is essentially a tacit agreement to disagree, and individual projects that collectively may be at odds with one another are pursued to satisfy different groups? For example, some may want a compact, transit- oriented city and projects that make sense on those merits. Others may want single-family detached dwellings outside of town and to drive cars and projects to pursue that. Dr. GOMEZ-IBANEZ: Where I disagree and think Dr. Small's earlier comment has power is that proponents of different viewpoints usually see the need to explain their position--it may be why a compact city is desirable or why sprawl is desirable--in terms that you can test. This may be whether a more compact city generates less pollution, has less infrastructure expenditures per capita, etc. Sprawl proponents say sprawl will result in lower housing prices and more housing choices. Those are testable propositions. People feel they should have reasons for their positions, and benefit-cost analysis offers a way to structure the debate that Dr. Lewis wants. Dr. KIRBY: Land use impacts are absolutely critical. One reason for conflict between local transit project proponents and the Federal Transit Administration (FTA) is that proponents (e.g., the transit agency, local politicians, and land developers) have land use development goals. But when they go to FTA, these groups are supposed to explain how they are saving travel time and cost, which can be expressed in terms of "generalized time savings." A lot of creative work is done to turn a land development project into a travel time savings project. This is where a lot of technical conflict arises. FTA now requires a procedure that tacks a travel time savings component on the end of a four step modeling process. However, it has a problem dealing with land development impacts, as the following examples illustrate. A new rail station in the Washington region is going into an old, dilapidated warehouse area where the current land use forecast predicts no activity. This station will stimulate much new development (evidenced by the fact the developers are paying one-third of the station's cost). But we are told that the same land use forecast is needed to evaluate both alternatives--building the station and not building the station. This is somewhat problematic. Building commuter rail lines is another example. A transit user moves to a distant suburb and takes commuter rail, thereby substantially increasing the user's transit costs and trip time. It seems unreasonable, but actually this is a true benefit of the commuter rail project. From the regional perspective, focusing development along these transit corridors offers choices and is what we are trying to do. Yet, this does not match up with FTA's evaluation criteria, which are overly focused on travel time benefits and used to compare projects around the country. That is a real glitch in this process and one of many conflicts. Dr. TAYLOR: Panelists seem to be saying that analytical tools do a poor job of addressing a range of benefits that are central to local concerns and local decision makers. Is there a fundamental problem emerging in applying these tools? Dr. KIRBY: Absolutely. Tools that are being mandated from the top are not always appropriately applied. For example, DOT lists 13 environmental streamlining projects around the country to be moved expeditiously through the environmental process to demonstrate that you should not get bogged down with paperwork. One that got through the process recently was blocked in court because the Environmental Impact Statement (EIS)[Footnote 17] process did not address the land use impacts of a circumferential roadway proposed in the project. Despite the project having gone through the entire federal review and approval process, some local groups felt that the land development impacts of this project were adverse. They readily identified an EIS provision on secondary and cumulative impacts and went to court. The judge basically said he would defer to the Federal Highway Administration (FHWA), unless a legal requirement was not addressed. He concluded that secondary and cumulative impacts were not addressed and stopped the project. It is a technical process failure when a special streamlined project goes through formal review and is blocked in court for what appears to be a very valid reason. Dr. SMALL: I endorse the idea that land use is a major category that is omitted in benefit-cost analysis. The difficulty is that analysts worry about not double-counting benefits and distinguishing between transfers of benefits and real net benefits. Land use involves considerable transfers. It goes to Dr. Taylor's point that you are in the middle of everybody's self-interest and their effort to get their piece of the pie when you talk about land use. So, land use arguments that are couched in terms of benefits are difficult to distinguish from the transfers of benefits. That is where analysts could help. While we do not have great tools for doing this, we understand some things about agglomeration[Footnote 18] and its value as well as positive and negative externalities from one parcel of land to a neighboring one.[Footnote 19] Improving benefit-cost analysis to distinguish between the net benefits and transfers deserves attention. [End of section] Summary of Panel Responses to Audience Questions: What is the effect of transit--particularly rail transit--on reducing congestion and on land use? * Dr. Lewis said that unlike highway travel times, which slow as more people use the system, transit travel times remain similar regardless of the number of people using the system due to transit's fixed schedules; this creates congestion stability rather than reducing congestion. He added that the level of benefits created through this stability varies. * Dr. Small noted that transit can (a) encourage consolidation of employment in a downtown area and (b) enable people to live further out and travel downtown along the transit corridor. In contrast, he said highways encourage residents and employment to spread out to the countryside. He added that a model might show that transit would be good for downtown employment and suburban residences, whereas highways would tend to encourage suburban employment and residences. [Footnote 20] * Dr. Pickrell added that all rapid transit capacity investments therefore promote decentralization--at least of residences. He said that the conventional thinking that transit will create more urban area density or that highways will promote decentralization is incorrect. When comparing highway versus rail benefits for both freight and passengers, what additional user benefits--beyond travel time savings- -and additional costs do we need to consider? * Dr. Forkenbrock mentioned a number of issues related to freight transportation investment,[Footnote 21] including (1) whether in some circumstances it can make sense to invest in truck-only lanes on interstates and major highways, (2) whether states can invest in rail to reduce the need for investment in facilities that serve trucks, (3) the concern of the business community that a rail system for freight would need to have a comparable arrival time reliability as trucking, when the trucking industry is one of the most competitive industries in the country while rail is exactly the opposite; and (4) the lack of data to create a good investment analysis comparing the two modes. * Dr. Pickrell said that there is a whole separate category of freight benefits from the construction and use of highway infrastructure. [Footnote 22] * Dr. Lewis conjectured that highway network improvements could encourage freight shippers to reorganize their production and logistics technology, resulting in a gain in productivity and a net gain in Gross Domestic Product (GDP).[Footnote 23] * Dr. Forkenbrock stressed that trucks do not pay their full cost responsibility for operating on the road and that if billions of dollars are to be spent on truck-only lanes, it would be better to have a proper pricing mechanism upon which to base those decisions. [End of section] Appendix IV: How Are Benefits and Costs of Transit and Highway Investments Best Measured? GAO Questions: * What are the most common problems in measuring the benefits and costs of transit and highway investments? * Which problems pose the greatest obstacles to accurate measurement? * What are the best approaches to examining benefits and costs? * To what extent can benefits and costs of transit and highway investments be measured at the national level? Kenneth A. Small is Professor of Economics at the University of California at Irvine. [See PDF for image] [End of figure] Opening Comments by Dr. Small and Dr. Pickrell: Dr. SMALL: I would like to raise some problems that are likely to generate a discussion of techniques for measuring costs and benefits: * Measuring the value of life. * Measuring how benefits and time vary with income. * Identifying the real decision being analyzed. The first problem is measuring the value of life. Dr. Pickrell gave us a reading on DOT's guidance on the value of life and time, showing that DOT backs away from a literal application of benefit-cost methodology regarding these measurements. [Footnote 24] Although the guidance recognizes and articulates the value of life concept quite well in the summary memo that guides DOT staff on its use, the guidance is very, very cautious about using it. It basically says--don't let anyone know you are valuing life. Instead, it suggests using what is chiefly a cost-effectiveness analysis--you analyze all other parts of benefit-cost analysis but leave out the value of life. Then you rank projects by safety and the cost of each life saved. Following this guidance poses a difficulty for benefit-cost analysts because it asks us to treat differently one area that is considered too hot to handle--or at least too hot to handle explicitly. The second problem is that DOT's guidance backs away from a literal application of benefit-cost methodology with respect to how benefits vary with income. It is well recognized that the value of time varies with income. In fact, the guidance directs us to value time as a percentage of wage rates. Yet the guidance declares that we do not want the value of time to vary with income in project analyses. This is understandable--we do not want any variations on the value of life--and this is not unique to the United States. A little more surprising is that value of time recommendations do not vary by mode. The rationale for this in the guidance is that the information is not good enough. This may be true, but the real rationale may have been the distributional issue again--that is, not wanting to get into the issue that more poor people take the bus and their value of time is lower, so we do not have to pay as much attention to them in terms of valuing time saved via bus travel. While all these positions have political explanations, they create difficulties in performing benefit-cost analysis. The third problem is that understanding the real decision being analyzed is a critical, continuing problem for benefit-cost analysis. Figuring out how to use a value of life calculation is a good example. You can say someone's life is worth $2.5 million if you make that explicit and recognize that you are not analyzing a decision that will take someone's life away--you are analyzing a decision that will change the safety level that people perceive. Usually, you are considering very small changes in the safety level, for example, going from a 0.001 to a 0.0012 probability of some adverse event. This is well articulated in DOT's basic guidance. While this process may suggest a way of successfully incorporating the value of life into an analysis, it brings up another problem related to the issue that Dr. Wachs raised when he questioned whether, when analyzing the amount of time saved, you can use 1 minute versus 20 minutes. The question is what decision is being analyzed. If you are considering an infrastructure improvement that will last 20, 30, or 40 years, are you really giving some identifiable person a minute and asking them how much do you care about this minute over 40 years? No. You are changing the environment. Many people will buy and sell houses and change jobs and alter their daily lives over many years. All those people are going to face a 1-minute difference in some parameter that affects them, and analysts will try to identify what effects will occur. This changes the way of looking at it from whether somebody can use 1 minute versus 20 minutes to the broader context of how decisions will affect the environment over time. I generally advocate that benefit-cost analysis should not overconcretize the benefits. We all tend to use ourselves as examples. While this can keep us connected with the real world, it is a little dangerous. We tend to make decisions more concrete to a particular situation--yours, your daughter's, etc. In fact, these decisions are likely to be altered and to affect circumstances in many ways that might not be immediately obvious. Donald Pickrell is DOT's Volpe Center's Chief Economist: [See PDF for image] [End of figure] Dr. PICKRELL: An invaluable first step in identifying and measuring benefits and costs is to develop explicit descriptions of the cause and effect paths by which alternative decisions or investments may lead to their anticipated effects. This step has hidden value for several reasons: * It requires decision/investment advocates to articulate not only the impacts that they expect and alternatives they prefer, but also the pathways through which they expect those impacts to occur, as Dr. Gomez- Ibanez said. * It forces us to think clearly about how and how extensively each alternative is expected to produce speed, safety, quality, or other desirable transportation service characteristics. * It helps clarify the magnitude and timing of specific resource commitments that would be necessary to produce these service characteristics. In this process, it is important to develop quantitative estimates of as many of each alternative's expected impacts as practical. This is not to say that whatever cannot be quantified and denominated in dollars should be ignored, as many critics of the practice occasionally argue. The fact that we do not know how to monetize or quantify important impacts should not be a reason to avoid quantifying impacts that we know how to quantify and whose monetary value we can estimate. It generally is useful to measure impacts as close to their source as possible. This ensures the correct attribution of the impact to the characteristics of the alternative that we expect to cause it. For example, it is better to measure travel time savings produced by an investment that extends some transportation service into a relatively undeveloped area than to measure property value increases that theoretically will result. We have discussed the importance of clarifying land use impacts of transportation investments. But, as Dr. Small reminded us, doing so involves wading into the morass of transfers and multiple counting of benefits that are best, most reliably measured at their primal source. Land use impacts represent the consequences of these primal benefits working their way through the complex urban economy in which we make most investments. Measuring impacts at their source will greatly improve prospects for heeding another deceptively simple sounding, but often violated guideline--count each impact only once. Measuring impacts as close to their causal sources as possible and stopping there helps to avoid the most obvious multiple counting that tends to occur primarily on the benefit side of the ledger. Similarly, one needs to be extremely circumspect about including indirect impacts[Footnote 25] of infrastructure investments because most of these represent multiple counting. While we want to think about how primal project benefits are transformed into other forms through the urban economy, we should not count them in a variety of places where it may be convenient to identify them. One also needs to be especially careful of including employment and so- called multiplier induced benefits in benefit-cost analyses.[Footnote 26] Here is an example of what not to do: Seven Midwestern states that were purchasing rail branch lines being abandoned by their operators were surveyed. The survey showed that employees' wages from shippers that would remain in business along these lines (as a result of state acquisition and operation of these lines) was the major benefit category for continued operation of these branch lines. So the states moved costs of the employees' wages to the benefit side of the ledger. To make matters worse, regional impact multipliers were applied to these payments. Not only were costs that were completely irrelevant to the decision included as benefits, they often were counted twice or more in evaluating the advantages of doing so. One needs to be very careful about indirect impacts--they often count something that we already have counted and included. A related issue is that measuring impacts in their naturally occurring or generic terms enormously facilitates comparisons among alternatives with different designs or technologies--such as highways and rail systems. Generic measures are those denominated in units--a person's hours of travel or expected injuries--customarily used to measure consequences of most transportation projects. There is considerable hand-wringing in public debate and literature about the difficulty of evaluating intermodal investments. However, transportation is quite well equipped to do intermodal comparisons because of the benefit measures that we customarily collect. There still are pressing questions about how to value service quality differences in different modes, but we are making progress in identifying specific, measurable dimensions of quality such as travel time reliability and its more esoteric aspects like privacy and security. A final caution: do not confuse the effects of seemingly related but ultimately separate decisions in evaluating benefits. The most common violation is including the costs of proceeding with an alternative investment as a benefit of making a completely separate decision-- although the first may bear on the subsequent desirability of the second, as shown in the following example: Proponents of constructing high-speed rail lines in inter-city travel corridors through a state argue that the benefits include savings from avoiding new highway and airport investments that otherwise would be needed to accommodate growing travel volumes. This contorted logic suggests that investing heavily to move people in the corridor is the only alternative. This is incorrect, given the degree to which public infrastructure investments commonly are mispriced. It suggests that competing investments have only costs but no benefits. If this logic is pursued to its extreme and we avoid spending on airport capacity by building a high-speed rail line, we forego any benefits that expanding airport capacity might have had without having considered those benefits. The importance of separating decisions that can and will be made separately and have logically separable impacts relates to what Dr. Forkenbrock wanted us to think about--the collective impact of our infrastructure investment decisions on evolving urban forms. In high- speed rail corridors, the impacts may extend even to the forms of development in corridors connecting major urban areas. While it is important to consider the cumulative impact of separate investments on metropolitan area forms and their desirability, it is very important to logically separate decisions to avoid confusing their benefits and costs. Panel Discussion: Dr. TAYLOR: You just cautioned us against focusing too much on secondary effects, double-counting effects that are too often double- counted, and using the cost of avoided expenditures in some other mode but not the associated benefits in an analysis. Are these mistakes the result of incompetence, error, or a desire to count as many benefits as possible for a project? Does this speak to motivations of the people doing the analysis? Dr. PICKRELL: I don't know. The modal structure of most state agencies and the federal DOT gives rise to more advocacy in evaluating alternative infrastructure investments than one would wish to see. As Dr. Gomez-Ibanez said, many participants in debates about major public infrastructure investments focus on what benefit-cost analysts regard as secondary impacts. These impacts are primary concerns for noneconomists who, after all, represent the majority of the world. Including these impacts is often motivated by a desire to refocus on decision consequences that noneconomists view as the most important, contentious, or debatable impacts. Often simple advocacy is at work. Finally, financing for many locally planned/selected infrastructure projects in which the federal government has a significant share tends to transform costs into benefits before the eyes of local political officials. Some focus on indirect benefits, such as job creation, is an expected consequence of this financing structure. Sidebar: "…by compressing everything into a single metric, you lose the ability to discuss impacts that cannot be quantified."-Dr. Forkenbrock. [End of sidebar] Dr. FORKENBROCK: Regional economic models are one of the main sources of double-counting--the problem is outsourcing. This often happens when an operating agency such as a Council of Governments or state transportation agency subscribes to a very good model like REMI.[Footnote 27] First, the traditional travel time savings analysis for freight carriers like trucks is added to the model. The model then generates employment changes, tax revenue, etc., that result from economic activity moving into the region by virtue of lower transportation costs. Lower transportation costs experienced by trucking are then added to the economic tax revenue and employment. This is blatant double-counting that probably reflects poor understanding of how the model actually works, rather than incompetence. Four more words--behavior, dread, philosophy, and rhetoric--are relevant to this discussion. Dr. Small's comments about valuing time, etc., have several important elements that relate to behavior. There is very good research about how people value their own time. However, when we simplify behavior, we miss a lot. For example, people have a five times greater disutility of time when they are waiting for the bus than when they are riding it and know they are en route to their destination.[Footnote 28] The highest disutility of time occurs when people get off a bus in a strange neighborhood and wait for another bus to come. People hate that. It really is a question of different kinds of disutility. Ian Savage at Northwestern University wrote an excellent piece that illustrates dread.[Footnote 29] Dread is willingness to pay to avoid the risk of losing your life--it is essentially the way we value saving lives. However, his survey respondents were much more willing to pay to avoid dying from stomach cancer than from the similar risk of dying in a car crash. People regard themselves as better drivers than someone who would be involved in an accident. Discount rates illustrate the philosophy issue. If your philosophy favors small government, you will want a high discount rate because benefits accrue over a long period of time. When Richard Nixon raised the discount rate to 10 percent, he declared that there was too darned much money going into the public sector and that we were building too many things. This is a really good way to reduce public sector investment because many studies show how sensitive a project using benefit-cost analysis is to the discount rate. It really matters. Finally, rhetoric. We cannot measure some things, as Dr. Pickrell said, such as the impact that a pound of carbon dioxide has on global climate change. There has been much debate about the right value to give to such intangibles. Should we exclude them? The answer is no. If we want a single measure at the end of the day--net present value, benefit-cost ratio, or internal rate of return--how can we do this? I would choose benefit-cost analysis because, as Dr. Lewis said, the purpose of benefit-cost analysis is to help improve decision making and allow us to discuss impacts that cannot be quantified but need to be part of the dialogue. An important problem is that by compressing everything into a single metric, you lose the ability to discuss impacts that cannot be quantified. Dr. GOMEZ-IBANEZ: All of us probably would agree that there are indirect impacts, such as employment or land use impacts, from transportation projects. If you create a job for someone who is unemployed, most of us also would agree that that person's wages are not a real social cost because there is no opportunity cost of putting that person to work.[Footnote 30] The opportunity cost is not the wages, as it normally is with someone who is otherwise employed. And there are legitimate land use benefits. The problem of valuing these indirect impacts centers on mispriced complements to transportation, especially the possibility that other complementary infrastructure such as water and sewer needed to develop a project is under-priced. But zoning made historic land use choices. It is difficult to argue that markets for transportation complements and substitutes are perfectly competitive and function smoothly since project developers, users, and residents often do not pay the full cost of infrastructure that their projects use. While each viewpoint about land use ought to be included in the debate because it has some legitimate role, there are several reasons that we fight so hard against including them. * First, land use effects are often taken much farther than the limited circumstances in which we believe they should be included. * Second, it is hard to forecast the path between policy and predicted outcome. Land use transportation models are by far the weakest part of urban and inter-city transportation demand forecasting. * Third, most of us believe that if you systematically measured these indirect benefits, they would be small. This presents a strategic decision--how to handle areas that are so confusing and difficult to forecast that they become the refuge for scoundrels who want to exaggerate a project's benefits. Discussing indirect land use benefits is their opportunity to do so with the least chance of being embarrassed by demonstrably true facts. Yet it is a real problem for us because the public has enough sense to see some legitimacy to these arguments. We run the risk of discrediting the entire process by denying them. How can you let these views in and still force the conversation about them to be reasonably clear sighted and hard nosed? Dr. LEWIS: That goes to my point that we need to consider subjective probabilistic analysis as a method or framework of analysis.[Footnote 32] One reason that we analysts get cold shoulders from stakeholders and decision makers is that they perceive us as pretending to know that which we do not and cannot know. Weather forecasters caught onto this a long time ago when they started talking about the probability of rain. People are much happier working with precipitation or hurricane probabilities than trying to suspend their disbelief in the forecaster's ability to know what will occur. We could take our lead from the biomedical statistical world. The U.S. Food and Drug Administration and National Academy of Sciences do good work with panels that bring solid statistical data to the table. Then experts, stakeholders, and people who work in the field develop another estimation layer called subjective or Bayesian--what you believe to be true, based upon your experience.[Footnote 33] For example, the Journal of the American Medical Association recently published the study of two heart attack suppressive drugs based on 10 years of double-blind trial data brought to the table by the National Institutes of Health. One drug costs $1 per vial; one costs $10 per vial. Which drug is more effective? A substantial amount of empirical, hard-nosed data indicated that one drug was 1 percent more effective than the other. However, a facilitated subjective probabilistic study with paramedics, nurses, cardiologists, and other experts concluded that the efficacy of the two drugs was indistinguishable. This study remains the foundation for the American Medical Association's recommendation on the efficacy of these two drugs. We can allow benefit-cost analysis to be a more powerful, useful technique for thinking through a topic based on values and defuse the "expert versus expert" mistrust in numbers. We can do this by bringing probability distributions of what we think is the value of a statistical life, a small time saving, or the likelihood that someone unemployed will have a new job along with the basis for these statements to the table. In doing so, we present everything we know in the context of how wrong we might be. People deal with this very comfortably, even if they do not have any statistical background. While Dr. Pickrell's point about going to the root source of benefits to avoid double-counting is very well put, there is a source of danger in the land use issue. Empirical evidence suggests that the value of time saved does not entirely account for property value increases that seem to be the capitalization of time savings. In suburbs where commercial and residential developments spring up around transit stations, when you account for measured increases in property value based on proximity or time-related benefits, there seems to be a residual that is left over. People appear willing to pay a premium to live or work in a transit-served area and the density, diversity, walking distances, etc., that come with that. Even if they do not use the transit system, they are willing to pay a premium for office space or to live in these areas. One might call it an existence benefit or option value as distinct from a user value. There are land use and property values that are not entirely accounted for by using the system for time savings. Dr. TAYLOR: If you saw a similar cluster of disparate elements in places without transit investment, how do you know that the residual is caused by transit investment time savings and not the agglomeration itself? Dr. LEWIS: You can never know. You could hypothesize that some people are willing to bid up commercial or residential land prices because of the direct or indirect effect of the rail station. Then you can study neighborhoods that are similar except for the transit and measure the change in land values. You can determine to what extent that increment may be accounted for by time savings, but you will never know if it was actually something else. This is why risk and probability are so important. People may accept a 20 percent chance that this is the explanation as a basis for accepting that value in a benefit-cost analysis. We do not know unless we ask them, and this is a fundamental part of facilitation. Are you willing to take an 80 percent risk that this value might be a cause and effect link that will produce a certain transit rate of return and some effects that you like, or an 80 percent chance it might be wrong? My premise is that people do this in their minds anyway. Benefit-cost analysis makes it formal and makes the size of the risk understandable. If the question is how to distinguish the need for values of time that vary with income--the answer is that you must allow for it because values of time do vary. Demand studies and forecast accuracy would be wrong if we blur over that. The answer is to distinguish between positive economics--the predictive end of benefit-cost analysis--46 and normative economics--the valuation end.[Footnote 34] When you assume that everyone has the same value of time because we believe it is fairer, you need to be very explicit about what you are doing and ask whether others agree. If not, let the market reveal values in the benefit-cost process. Then you will get a different result for transit that helps poorer people, and people will deal with that information. Dr. MEYER: Two of GAO's questions--what are the most common problems in measuring the benefits and costs of transit and highway investments and which problems pose the greatest obstacles to accurate measurement--may take us in different directions. Much of our discussion has related to valuing benefits--that is, the value of time, reliability, and life. My field experience suggests that although people may raise their eyebrows at valuing human life, they more or less accept it. The problem is change--especially over a 25-or 30-year period. How do you predict reductions in fatal crashes? How do you predict air quality changes such as tons of X emitted? We mentioned developments in intermodal comparisons, but that presupposes that our models can look at person hours of travel by different modes with some confidence over a 25-or 30- year horizon. While value issues are very important, existing technical tools and models at the regional, state, and even more at the national level are unable to predict change. In my transportation safety course, we spend a considerable amount of time on safety, value of life, etc. When asked what will be the reduction in fatalities for a city bus in 2025, students accept the value of a life but say there is no way to answer the question. In my area, we are examining how to put value and reliability into a time perspective so that we can measure this when we evaluate the context of our updated regional transportation plan. However, without the full microsimulation model[Footnote 35] that can look at freeway system performance it is very difficult to predict transportation system reliability in 2030. Dr. LEWIS: I think our difficulty in forecasting lies more on the engineering and science side than the value side of the equation. An analysis conducted for the National Cooperative Highway Research Program sought to deconstruct uncertainty in estimates of the economic benefits of reduced highway congestion. Analysts found that about 80 percent of the uncertainty is associated with potential error in the shape of the speed-flow relationship (an engineering problem), while the balance of error lies in uncertainty about the value of time (an economic problem). We are presenting the public with our best guess, but I do not think they want our best guess. I think they want to know the likelihood that the sun will shine. If we are honest and peer into our models for standard hours and extract probability information, we can let them know that the time savings of a proposal has about a 10 percent chance of being greater than 3 minutes per person. If you want 8 minutes per person, it may have a 5 percent probability of happening. I think we know how wrong we are. Dr. MEYER: I agree and view this as a way to improve benefit-cost analysis. On the other hand, it often is very difficult to explain to nontechnical, elected officials that it is within plus or minus 5 percent of the most likely result. Dr. TAYLOR: Are you getting around uncertainty by presenting probability? Certainty and probability are different things. Dr. LEWIS: No. I think including probability makes information more useful to decision makers, given the state of the information at any given time. The bond insurance industry has been using traditional "four step" transportation demand studies[Footnote 36] for revenue forecasts as the basis for determining whether and at what price the project bonds are insurable. It was something of a surprise to the bond industry to learn that such forecasts have fully a 50 percent chance of being too high or low. To obtain forecasts with a 99.5 percent certainty of being met or exceeded, the four-step model must be quite considerably modified. The probabilistic presentation is not a way of eradicating uncertainty, but of making the information more appealing, truthful, and useful. This is especially the case when it is used to compare a transit project with widening a highway, major corridor, or other alternatives. You may get similar results, but my experience is that one option has a much higher risk of going underwater than the other, even though the risk profile of the projects is very different. If we get this information to people, it can be more useful. Dr. WACHS: Dr. Lewis has made the point that benefit-cost analysis is extremely important to us because it gives us a language for our discourse and enables deliberative democracy to take place. I return to that question and ask whether what we have just discussed gives us any confidence that it helps creates a deliberate discourse and helps us govern ourselves as a democracy? Or, does it actually distract us? Does discussing whether or not we are double-counting really help elected officials and citizens who represent us come to better decisions? This group is experienced and committed to benefit-cost analysis and would like to believe that the answer is yes. However, there are critics. Sidebar: "Does discussing whether or not we are double-counting really help elected officials and citizens who represent us come to better decisions? … we have to…demonstrate that these important questions actually help make better decisions."-Dr. Wachs. [End of sidebar] I have been reading a book called Priceless: On Knowing the Price of Everything and the Value of Nothing.[Footnote 37] It argues that if you want to invest enough public resources to prevent humanity from becoming extinct, then the benefit-cost calculus is irrelevant and not helpful. If a public decision was made to preserve a species or build a rail line, surely you can retrospectively conclude that benefits must have exceeded costs. In Joseph Berechman's benefit-cost analysis of investing in the Appian Way,[Footnote 38] he concluded that if you valued intangibles at a certain level, then it was worth doing; if you valued them at less, it was not worth doing. Clearly, their society made a deliberative decision to invest in the Appian Way without quantitative analysis. The relevant question is whether the measurement questions that we have been discussing are so difficult to address that they hinder our ability to have a deliberative, democratic discussion. If GAO is asking whether we can encourage Congress to be more attentive to this, we have to carry this discussion one level farther and demonstrate that these important questions actually help make better decisions. Dr. GOMEZ-IBANEZ: All evaluation is relative. I agree with your concern about things becoming a distraction and the duel of the analysts being irrelevant to most things. What is an alternative? What would you propose? Dr. WACHS: I am not sure. I certainly would not give up analysis, modeling, prediction, using probabilities, etc. However, I might weaken the requirement that everything be done in monetary terms. This requirement may force us to make assumptions that govern what we are doing and obscure the dialogue. Ordered, analytical approaches are very helpful, but it is healthy to keep asking whether our society benefits so much from a dollar framework for benefits and costs that we should devote all this effort to trying to get the numbers right. Dr. SMALL: The problem with looking solely at the primary benefits, rather than the benefits where they finally accrue, is that doing so, although absolutely correct in analytic terms, conflicts directly with the mandate we've been discussing--making benefit-cost relevant to decision makers who are not experts in benefit-cost analysis. Land use is one of the first things that people consider as a transportation project impact, but one of the last things on the chain of causation. We cannot afford to ignore it by just presenting final benefit numbers, saying that benefits were measured correctly at the source of the primary benefit and therefore we can ignore land use effects. Benefit- cost analysis could be used in deliberative discussions so that people actually understand that the impact they passionately care about is a little further down the chain. When you actually do the numbers, you can see if there are some real indirect benefits. I hope that terminology can be developed to convey this whole difficulty. Regarding employment as another indirect effect, I disagree a bit with Dr. Gomez-Ibanez. We agree if somebody who is unemployed becomes employed, that is an extra benefit. However, we may be ignoring other related things by being too concrete. If a transportation project throws someone out of work, we count it in a certain way. But, other things may be happening. The Federal Reserve may be tightening monetary and employment policy because it is afraid that the economy will grow too quickly. Even something that looks like an obvious benefit, such as putting an unemployed person to work, may not be in a broader context where there are other policy objectives that cause that person to be out of work in the first place. Sidebar: "We are very focused on quantifying dollar benefits and do not want to talk about transfers, distributional issues, or externalities….But… this is what people out there really care about."-Dr. Kirby. [End of sidebar] Dr. KIRBY: Regarding Dr. Wachs's concern about whether monetary terms are just getting in the way, I think that if we see benefit-cost analysis as distilling everything to a dollar number, then monetary terms are getting in the way. There is much work that can illuminate impacts in an extraordinarily useful way, but we cannot push it too far. We are very focused on quantifying dollar benefits and do not want to talk about transfers, distributional issues,[Footnote 39] or externalities--sort of second class in terms of benefit-cost analysis. But the reality is that this is what people out there really care about. We can illuminate those issues enormously as long as we do not overstep what we can really do. Distributional issues regarding affected populations are what decision makers discuss. Land development is very important to local jurisdictions that are watching their tax base and plays through the decision-making process to the governor. If a transportation investment will take an attractive new technology into the neighboring jurisdiction, rather than mine, I care a whole lot. Dr. Lewis' example of people paying a premium to live near a transit station raises such an issue. Lower income people for whom we are trying to provide transit are being bid out of those areas and must move to areas without transit. This issue, which is a distributional issue, comes to the table in the political discussion. We may not be comfortable with including it in our benefit-cost analyses, but it is a very important issue. The value of time in the context of who uses a toll or high-occupancy toll (HOT) lane[Footnote 40] is a tremendously important issue that Dr. Small's paper addresses. [Footnote 41] Most highway expansions proposed in our region now are toll financed roads because we lack the ability to do them any other way. Private firms proposing to build these facilities want investment grade analysis from those who will issue and service the bonds. They want to know who and how many people will use the road, usage at various toll levels, and how much will be paid to investors. This moves travel demand forecasting to a whole new dimension and puts new demands on modelers. When we could keep drawing down on the federal highway trust fund to finance transportation projects, it was a different decision. Now, the project will not go forward unless there is real comfort with forecasts. The distributional aspect of HOT lanes is fascinating. At first, these were seen as Lexus Lanes for rich people who had high values of time-- the assumption was that poor people would not use them and therefore they were bad. This was a showstopper for quite a while. However, monitoring these lanes' usage showed that it is not just the rich-- people that value reliability at all income levels use them. Women are disproportionately represented because they have child care and other responsibilities. Dr. Small demonstrates that complex values cause people to use these facilities, but that these values can be analyzed. This analysis will play into policy and decisions about moving forward with some toll facilities because who will use them is a critical issue. There is a tremendous amount that we can contribute by focusing on these issues that appear to be somewhat less tractable. Who else is going to do it? The discourse will occur anyway, but in a much less informed atmosphere if we do not wade in on these issues. Dr. PICKRELL: To be clear, I did not advocate ignoring downstream effects of transportation infrastructure investments. I advocated clearly articulating the paths through which primary and secondary benefits that become the main focus of public debate are generated so that they can be reliably measured. Keeping these paths conceptually separate allows us to respond to people who claim that land use consequences of their preferred investment have been ignored. To some extent, we get in these binds because we do not do our homework. Research on taxi cab use that we read as graduate students clearly demonstrated that they were not used exclusively by the rich. Taxis were used by elderly people going to medical appointments and a variety of people making trips that were highly valued. We are our own worst enemies because we react to criticisms about so-called Lexus Lanes in a way that does a disservice to what we otherwise, in other circumstances, would claim that we already knew. Dr. LEWIS: I agree with Dr. Wachs's concern that seeking to monetize everything could be distracting. As I tried to say in my paper, benefit- cost analysis has diverged from what might be perceived as useful because it is not consistent with some realities of the last 50 years. One such reality is that most people are not satisfied with some of the restrictive assumptions of conventional benefit-cost analysis. One convention is the idea that a project can be declared a welfare improvement if benefits are sufficiently large to enable beneficiaries potentially to compensate losers and still remain better off. Today, it matters to people whether such compensation actually is paid. Furthermore, it matters whether the project and compensation, as a package, garner community consensus. The debate about making all transit systems accessible to people in wheelchairs versus creating a separate transit service for people in wheelchairs is an excellent example. When the benefits of creating a separate transit service for those in wheelchairs were found to be greater than making all transit systems wheelchair accessible, something had gone wrong with the analysis. Benefit-cost analysis had not stepped into the situation's reality and facilitated a discussion about the value we put on accessibility above and beyond monetary considerations. John Rawls' book, A Theory of Justice, [Footnote 42] emphasizes that there are certain rights, duties, or obligations that are not necessarily enshrined in the Constitution, but which we believe we have acquired. These include environmental rights, disability rights, obligations regarding greenhouse gas production, etc. They ought to be treated as givens, and benefit-cost analysis should then ask how we can optimize our world in that context. Economists need to be flexible to facilitate this discussion, and benefit-cost analysis needs to morph into a framework in which that discussion actually occurs. The Americans with Disabilities Act [Footnote 43] would have been passed 10 years sooner if full accessibility had been discussed in that way. There is no reason why a benefit-cost process could not have led to that result had we not been confined by our view of the analytic process. Converting benefits and costs into money is important, but not the Holy Grail. Dr. GOMEZ-IBANEZ: The benefit-cost analysis of universal transit access compared to separate transit systems fits into Dr. Lewis's deliberative model. People realized its limitations through discussion and decided to do something else--that is, make all transit systems accessible to the disabled. Although a narrow benefit-cost analysis said that the benefit of making all transit systems accessible to the disabled is not worth the cost, people responded that this is not strictly a benefit- cost question. Dr. PICKRELL: It provoked a discussion about whether making all transit systems accessible is a right and the value of enforcing that right. Dr. TAYLOR: Dr. Kirby said that distributional issues are absolutely central. We heard that benefit-cost analysis sometimes does not deal with them effectively or effectively enough. Why is this so important? There is one issue where we can make normative judgments about redistribution--we are going to redistribute wealth from the haves to the have-nots. Dr. Kirby also described redistribution as a competition where there are winners and losers. Such a redistribution may not been seen as a loss to anyone at the regional, state, or national level, but stakes are enormous at the local level. Can consideration of these stakes--that is, winners and losers--be incorporated into the analytical process? Dr. FORKENBROCK: Sometimes the opposite may be true, as with regional economic models. We discuss how jobs will come into the region if we make a transportation investment, but not where the jobs came from. The rule of thumb is that if job redistribution occurs in the same jurisdiction that is funding the project, there is no net gain. But when you use federal funds, redistribution occurs in the same jurisdiction. A state that funds redistribution of jobs from one community to another has made a mistake. The difficulty is what I call the moral imperative--that is, decision makers know that they have nothing to lose by arguing strongly for their project--the federal gas tax will not go up or down as a result of a single project. And if the model does not indicate where the jobs came from, no other politician is going to tell me that these are jobs off my plate. There is an advantage to obscuring this, and models do that so well. Sidebar: "…distributional issues…are at the heart of benefit-cost analysis and are the central reason why we need it."-Dr. Small: [End of sidebar] Dr. SMALL: I have strong views about distributional issues--they are at the heart of benefit-cost analysis and are the central reason why we need it. Some decisions are made with unanimity--perhaps everyone's incentives are aligned. Benefit-cost analysis is needed when real differences in the outcomes to people must be adjudicated. This usually occurs through the hurly-burly of politics. But we are agreeing that benefit-cost analysis can help this adjudication process lead to better results. Technically, the adjudication process is a way to get potential Pareto improvements[Footnote 44] transformed into real Pareto improvements. This is because, when a project has positive net benefits, there are still some people who are winners and some who are losers--although there are more on the winning side. If you apply this methodology consistently to many different projects and ensure that it is not always the same group of people that loses, then there is an increased chance that you will have real improvements for everybody. These distributional issues are so fundamental for benefit-cost analysis that in practice we often have to carry them out and make them explicit in order to move forward with a project. Dr. PICKRELL: I agree that distributional issues are of paramount importance. Any analytic process is embedded in an inherently political decision-making process that turns on considerations of who pays and who benefits. I personally believe that is for the better. I was adamant about systematically tracing the mechanisms by which public infrastructure investments are expected to produce direct and indirect benefits because this naturally leads us to tally the benefits and costs borne by different participants in the political decision calculus. At the very least, it helps provide systematically forecasted rather than speculative or woefully misrepresented information about the impacts' nature and magnitude. It also produces the advantage that Dr. Small is describing--we can examine whether the succession of decisions from this political and analytical process tends to work to the advantage of some groups or others over time. Dr. KIRBY: I do not disagree, but I am concerned that we tend not to focus on things that we are less comfortable with analytically or less able to quantify. As a result, we may be vulnerable to a challenge that we failed to comprehensively assess a project and a judge is going to stop it. If it was a good project that does not get built because the process failed to address a legitimate issue, it is a real criticism of us as practitioners and something to be very concerned about. You cannot make every project a winner for everybody--and do not need to. If projects are in a large enough context to show overall net benefits and that no group is systematically without benefits, then I think you can proceed. However, in our very elaborate public involvement process, you may be unable to take action if anyone loses. The electorate is very well informed. People articulate the disbenefits[Footnote 45] to them and this can be enough to overwhelm the more diffuse beneficiaries. This may be where we are in transportation. Project after project goes on the shelf--it really is a big concern. A county transportation director recently said that numerous studies--but not one single project--were proceeding in his county. High benefit transportation improvements were not going forward--paralysis by analysis. If we do not recognize that there are some disadvantaged groups early in the process, we will not be able to deal with the issues they raise in an effective manner. Dr. TAYLOR: Perhaps we are not doing a good job of compensating the losers if we have situations where a concentrated set of opponents can kill projects for which the benefits overwhelmingly outweigh the costs. Are there legal or procedural limitations to the ability to compensate losers? Dr. KIRBY: No. But we often fail to recognize that there are those who do not benefit, never mind thinking about how we can compensate them. These groups do not feel that their issues have been fully addressed and sometimes use legal challenges to stop a project. Our failure to address this is rather critical, as is shown in the following example: These issues came up in a large project that involved land use questions. People were concerned about the induced demand[Footnote 46] caused by a major highway expansion project completed 20 years ago. We went back 20 years and looked at the forecast. Traffic did exceed the original projections, largely because the development shifted to that corridor away from other areas relative to the forecast. There was no question--it was not from the outside, it was a shift. Another jurisdiction used the study to argue that a proposed new highway facility would take development away from that jurisdiction. Based on that earlier study we are now saying that each jurisdiction must revisit its land use and activity forecast under the assumption that this proposed new highway facility will be built. Employment forecasts relative to the project may be reduced for some areas and come out on the table as part of the political discussion. There is no way of avoiding that. But, we must put that topic on the table and recognize that there will be some marginal effect. We cannot expect everyone to win on everything--otherwise, nobody will get anything. If we discuss this type of issue explicitly, we will be much better off. Dr. FORKENBROCK: The issue of compensating losers is fascinating, but extremely elusive. A new highway is going to raise noise levels and people are upset. But, what if the access value has raised property values? Looking at many different dimensions, deciding on winners and losers, and summing it all up could show that overall, the people who are upset lose little or not at all, but the analysis would be extremely difficult. Dr. LEWIS: I agree with Dr. Kirby--there is evidence to suggest that making the redistributional effects and the winners/losers more explicit and transparent is likely to speed decisions more than slow them down. We did a large discursive benefit-cost exercise[Footnote 47] for building a new airport runway in a large Canadian city. The proposal had been on the table for nearly 20 years and modifications had been knocked down by noise advocates. Although the airport authority was reluctant, we engaged the community in a very scientific discussion of measuring noise and the empirical evidence on depreciation. We even used loss of household as surplus in addition to financial losses of value. Through an analytical process, we got quite wide assent that there would be $100 million of property value losses over 30 years due to the additional noise over a specific area. But that was put in the context of about $4.5 billion in economic benefits. The benefits lost without the runway were mainly in time savings and resources, not jobs. This precipitated some compensation proposals that were integrated into the solution--noise insulation, etc.--nothing terribly dramatic. There is a runway there now, for better or worse. I think that the numbers created political will that did not exist before in the environmental review office, the litigation domain, and runway supporters. The numbers simply made the case. I think you can make things real and the truth sort of bubbles up to the surface. Dr. FORKENBROCK: The problem is that the objector is the same as the loser. Dr. LEWIS: In airport cases, they often are. When you get to valuable development around transit stations, perhaps you are gentrifying the community and what was an affordable housing community is no longer. What do you do with that? In this case, objectors are rarely the people who would be displaced. If there is awareness that this could happen, then simply acknowledge and analyze it. This was not done in the Model Cities program, otherwise the removal of ghetto residents to even worse, poorer ghettos might have been anticipated and Model Cities would not have been the disaster that it was. On the other hand, if people living near a station now have to walk a little bit further, we might acknowledge that this is a reality of the market system. Being explicit about it makes all the difference. Dr. GOMEZ-IBANEZ: I will throw a little cold water on this love fest for distributional analysis. I do not disagree that keeping track of projects' distributional consequences is ethically important--the whole Pareto Principle depends on compensation--as well as politically prudent. If you ignore them, you're going to fail. However, I worry that we are not being honest about the technical difficulty of some of what we are proposing. We are discussing several kinds of redistributions. One is between different income groups; the other is between different jurisdictions or locations. Both of them are really difficult. What you need is some general equilibrium model that traces out how these direct effects get transferred to other parties. The classic example has been mentioned--that is, a lot of transportation improvements' value does not end up with a traveler, but is capitalized into the land values of an owner's property when the improvement is made. But that is just one example. In addition, economists are not very good at tracing the subtle ways in which things get passed along. If the chain gets too complicated, as when you make downtown more accessible and try to assess how much goes into downtown property values or higher wages and profits for downtown workers and businesses, it is very tough for us to give you an answer with much confidence. Focusing on jurisdictions is at least as bad and is compounded by confusion about whether growth necessarily benefits a jurisdiction. Often it is not clear--at least to existing residents-- that job or residential growth is such a benefit. So a probable benefit to landowners may not be a benefit to all the citizens. In addition, we are not very sophisticated about the implications of some of the distributional effects, particularly geographically. On one level, I agree that the arguments for tracing distribution are pretty strong. On another level it is garbage in, garbage out. You may end up with some quite naive and misleading estimates of distributional consequences. The example of noise from airport runways probably is one of the places where it is easiest and clearest. Many other things we deal with are more complicated. Sidebar: "…the geographic and the population distribution issues…are very, very difficult, and benefit-cost analysis is only part of the total evaluation framework."-Dr. Meyer. [End of sidebar] Dr. MEYER: I agree with what Dr. Gomez-Ibanez just said because I face both the geographic and the population distribution issues. They are very, very difficult, and benefit-cost analysis is only part of the total evaluation framework. For example, to get the message about real distribution issues in regional investment, we did not look so much at benefit-cost analysis as we indicated on a map where the investment was going. We also indicated how this investment related to the poor and minority population. This distributional impact got people's attention. This visual relationship initiated a discussion about equitable distribution of investment vis-á-vis population location and where the revenues are coming from. We really should be talking about evaluation and the role for benefit-cost analysis in this much broader framework. [End of section] Summary of Panel Responses to Audience Questions: Could benefits and costs be measured at the national level so that Congress had a framework for deciding, for example, whether to spend $1 billion on transit or highway projects? If it is possible, is it a good idea? * Dr. Forkenbrock stated that the only way to do such an analysis would be to use major projects undertaken in the past decade as sample data- -and this would be precarious. * Dr. Pickrell said that the Highway Economics Requirements System (HERS)[Footnote 48] model attempts to do this for highways by simulating the most beneficial improvement projects on a large sample of U.S. highway segments, adding the benefits of these projects, determining the spending level required to generate those benefits, and expanding to a national estimate of how much you would have to spend on highways to achieve various criteria such as maximum benefits. He said that there is no corresponding model for transit investments or one that incorporates both potential highway and transit investments. * Dr. Meyer said that we do not have the analytical tools needed to provide Congress such a framework. * Dr. Taylor noted that evaluating transit investments at the national level is particularly problematic because the consumption of transit services is so spatially asymmetric: since most transit riders are in the centers of the largest and oldest U.S. cities, most of the benefits of transit projects will be in these locations. [End of section] Appendix V: How Could Benefit-Cost Analysis Be Improved? GAO Questions: * How can data availability/quality be improved and uncertainty reduced in estimating the benefits and costs of transit and highway investments? * Is there a basis for standardizing benefit--cost methods and is this desirable? * What is the best way to help ensure the objectivity of benefit--cost analyses for transit and highway investments? Opening Comments by Dr. Forkenbrock and Dr. Kirby: David J. Forkenbrock is Director of the Public Policy Center, Director of the Transportation Research Program, Professor in Urban and Regional Planning, and Professor in Civil and Environmental Engineering at the University of Iowa. [See PDF for image] [End of figure] Dr. FORKENBROCK: To stimulate discussion, each of you has my handout, "Improving Benefit-Cost Analysis."[Footnote 49] Several concerns have developed from my practical work with state agencies to refine benefit- cost analysis for major investments and with a team that conducted feasibility analyses of three of the national interest corridors[Footnote 50] identified by ISTEA. These problems occur particularly at the metropolitan and state agency level, rather than the national level. * One problem is excessive dumbing down of benefit-cost analyses. When a colleague surveyed state agencies about their current benefit-cost analysis practices, he found that a series of them do not even discount their benefit and cost streams[Footnote 51]--they just add them up. Many state agencies and metropolitan planning organizations (MPOs) [Footnote 52] use computer software that contains a range of assumptions that are not explicitly chosen and that the user does not understand. A good example is using national averages for the value of time. Similarly, some analyses ignored network effects--that is, a project's effects on the larger area if, for example, traffic is diverted. Safety and modal shift effects also were ignored. * A second problem concerns questions of who benefits, who loses. We have talked today about the Kaldor-Hicks criterion--that is extremely important. [Footnote 53] We have just finished a National Cooperative Highway Research Program guidebook on taking into account distributional effects and incidence that are very important, but very commonly ignored, in benefit-cost analysis. [Footnote 54] * A third problem is that parameter values are very poorly chosen.[Footnote 55] In state agency project evaluations, the basis for fatality or injury values rarely is explicitly considered. Many states use very low values. One corridor of national significance runs between two states. One state uses a million dollars per life; the other uses $500,000 per life. My colleague asked the states surveyed why they used whatever discount rate they used and found that the basis was not well understood. As a result, the impacts of different discount rates were very rarely considered. Yet, the choice of discount rate and value of time can make a bad project into a very good project or a very good project look infeasible. These parameter values are incredibly important when evaluating specific projects. Moreover, time values rarely are tied to local wage rates, and there is no consideration of how the value of time might be varied. The "second best" issue[Footnote 56] is almost never considered. This means that if you are estimating demand on the basis of heavily subsidized prices, you are not going to get an efficient evaluation. This fact is almost never taken into account by benefit-cost analysis users at the state and MPO levels. When I chaired the National Research Council oversight committee on the National Highway Cost Allocation Study in 1997, we found that certain vehicles--particularly heavy trucks with fewer axles--have a cost responsibility that is perhaps 20, 30, 40 times the amount that they pay in user fees. There will be very different results if you build a road to serve current traffic levels and forecasted traffic increases than if you price the road at full cost and see what the demand level is, using that as the basis for your investment. The "second best" issue is a major and difficult problem. It occurs to me that one solution to these problems is the American Association of State Highway and Transportation Officials' (AASHTO) new manual on benefit-cost analysis--the Redbook.[Footnote 57] It contains good, useful information on doing benefit-cost analyses in transportation. Other solutions may be practical courses on benefit- cost analysis sponsored by FHWA, AASHTO, or some other group. Courses could pay special attention to using sensitivity analyses[Footnote 58] to test the impacts of different parameter values and increasing the use of state or regional travel demand analyses to estimate network effects. Many times, travel modeling that is the basis for determining whether investments are called for is done on a partial basis--the models do not actually look at the entire travel corridor. State cost allocation studies[Footnote 59] have been improving over recent years, but they still have many conceptual and data-driven limitations. And, without knowing cost allocation levels, you have no idea of exactly how close different vehicles are to paying their full cost. This information can drive investment analyses and become a critical element in doing confident benefit-cost analyses. Ronald F. Kirby is Director of Transportation Planning for the Metropolitan Washington Area Council of Governments. [See PDF for image] [End of figure] Dr. KIRBY: From the perspective of an MPO that does a lot of number crunching, modeling and analytical work, what we do does not map very well onto an ideal benefit-cost framework. Like other MPOs, we have spent the last few years very focused on air quality and computing regional emissions to meet Clean Air Act Amendments (CAAA) conformity requirements. [Footnote 60] These requirements are very stringent--they can block and limit project development. MPOs are required to meet fixed "emission budget" levels that are part of a much larger regional air quality analysis that looks at all kinds of different sources. We must meet a fixed air quality standard. If we are below it, we pass; if we are above it, we fail. That is a benefit-cost framework for you. The U.S. Environmental Protection Agency (EPA) currently mandates that we use the MOBILE6 model. [Footnote 61] This model requires us to model 28 different vehicle classes (the previous model had eight different vehicle classes). It puts vehicles into different weight categories, engine types, etc., and requires information that goes way beyond the data that we have for our region. Air quality modeling has been one of our major preoccupations because it could be a fatal flaw. If we do emissions calculations incorrectly, we could be challenged by those who might not like a particular project and see air quality requirements as a way of slowing things down. Air quality and transportation investment studies are insulated from each other--this is a big problem, as Dr. Forkenbrock noted. The requirement applies to individual projects and involves intensive work at the corridor level. While it draws on regional travel modeling, the EIS is a separate undertaking--often employing consultants who take the regional models and use them at the corridor level. The focus is on a project and corridor--and the process does not look as comprehensively as it should at regional implications. In a case in which citizens requested a study of alternative sites for a bridge crossing, the project study team initially responded that this was outside the project study scope. Yet, alternative sites were within 10 miles of the existing bridge. A month later, the study team thought better of its answer and agreed to test where traffic would come from and go to at other crossings. This is the philosophy that surrounds some corridor studies. However, as scrutiny of these studies increases, we are being driven to look at things much more comprehensively. Travel modeling is a big part of MPO work and much is being asked of modeling work. Sidebar: "We have a tendency to focus too much on improving travel models and not enough on improving the data that drive the models."-Dr. Kirby: [End of sidebar] We now are being asked to look at pricing issues in addition to air quality issues. Pricing is a whole different approach to project planning that puts a different angle on issues and poses very tough questions in forecasting the impacts of different pricing strategies. For example, we are doing a major corridor study in which the project is a new managed roadway--18 miles, 6 lanes, with 50 miles per hour peak period speeds in 2030. Our job is to find out what the toll is supposed to be, see what the revenues will be, and match this up with additional revenue sources. This will press our modeling capabilities more than in the past. In addition, groups who have various views about the outcomes they do and do not want will scrutinize our modeling procedures. This will be one line of challenge to the modeling outcome, and we are going into this project knowing that the technical work will be challenged. Elected officials and citizens now have a very substantial interest in knowing everything that is behind modeling. Many things that formerly were in-house technical issues are now scrutinized by citizens and elected officials. For example, values of time and how well the model replicates existing travel patterns, root mean square errors in terms of matching model results to traffic counts are going to be very important. This is good, but it requires us to explain many more things than before. Data availability and quality need to be improved to improve benefit- cost analysis. In my view, we have a tendency to focus too much on improving travel models and not enough on improving the data that drive the models. For example, there is growing interest in time of day modeling--how peak travel periods are spreading at congested locations on the freeway systems and how people react to congestion by changing their travel behavior. If people react to congestion by traveling later or earlier and you increase capacity, those same people will revert to the time they traveled before, and congestion will return. Has your project failed to eliminate congestion? Modeling travel at different times of day is important, but travel models are not terribly good at dealing with it or other traffic operational effects. An obvious question is whether we have good time of day counts. The answer is that we do not. We have found instances where the published counts are clearly inconsistent. Although this is an important issue, we have very few permanent count stations in our region and some are not operating. It is difficult to get data from state agencies because it is not a priority for them to collect counts at the level of detail that we need. Much more attention has to be focused on data if MPO modeling and analysis is going to get to this new level of scrutiny. There are obstacles--home interview surveys are increasingly harder to do, telephone surveys are difficult, all survey response rates are going down. Those people who have only cell phones are not included in a random selection of households. Should we put global positioning system (GPS)[Footnote 63] devices on the vehicles and track them around to get data in the future? Refreshing our data poses really tough issues. We really are in a new ball game--dealing with issues that we have never been asked to handle before. For example, we need analytical methods to help us predict how people will respond to various prices on a new lane or--even more complicated--on one or two tolled lanes on an existing freeway with lots of entry and exit points along the way. When the private sector wants to know travel volumes to decide whether or not to finance a project, we are facing an information challenge that we have not faced before, namely to get "investment grade" forecasted volumes. Before you can ascribe benefits and costs, you really need to know what travelers' responses are going to be to the project. Until you can get a rough handle on that, all the other issues--discount rates and everything else--are rather academic. There are plenty of consumers--the public and elected officials--for the things that we are talking about today. But, we will need to do a much more comprehensive job than we have done in the past if we are going to respond to their interests and meet our planning requirements. Panel Discussion: Dr. TAYLOR: We have heard that there is less and less willingness to put the significant resources required toward refreshing, updating, and revising data that are collected in household activity surveys. At the same time, there is much more detailed interest in your analyses. That seems somewhat contradictory. Dr. KIRBY: The dilemma is that there is a long lead time in getting and applying these data. If you want the data now, you should have collected it 10 years ago and been improving the models over that period of time. The increased funding from ISTEA and TEA-21 was a big help in improving our data collection and model development. But a whole new home interview survey is beyond our annual budget. If reauthorization of TEA-21 helps or the state agency funds such a survey, it still will be 3 years before we have results that are cleaned up, calibrated, and in the models. Currently, we are dependent on what we were able to do over the past 10 years. The idea of a standard national travel demand model disappeared 20 years ago. There has not been a strong federal presence in this area-- travel modeling has devolved into independent vendors with different software packages. As an MPO, we can choose among four or five vendors who tailor models to meet our needs. But, there is no standard, as Dr. Forkenbrock's committee concluded. [Footnote 64] If you ask how are we doing or are we doing as well as anyone else--nobody can answer these questions because nobody knows the big picture. We hope that DOT will fund TRB to prepare a synthesis of modeling best practices to see how we can address this problem. We have had 10 or 15 years of everybody doing their own modeling, and now it is very difficult to know the current state of the practice. Dr. LEWIS: There are analogous data quality problems in other areas. One type of solution is suggested by the practice of benefit-cost analysis in relation to information technology investments. Here, large organizations want to know if the benefit--a productivity gain--is going to outweigh the costs. However, the productivity data often are as awful as the counts that Dr. Kirby describes. So, what do you do about it? Increasingly, we are saying that the quality of the data is what it is. The important point is that continuously or periodically measuring productivity becomes a necessary project cost. We have a baseline and even if we are unable to measure productivity perfectly, we can measure it the same way each and every time, starting before the project is implemented. For example, the U.S. Department of Homeland Security has determined that queue lengths for visitor programs[Footnote 65] will not only remain stable but decline by using tools such as biometrics at various points of entry in and out of Canada and Mexico. We are measuring the extent to which queue lengths are improved by these interventions. The simple point is to measure it in the same way for a year before the changes are introduced and thereafter. Track performance on the basis of data for which one can be reasonably sure there is a standard of measurement. We do not have the habit of measuring the after-effects and benefits of transportation projects. However, doing so has become second nature in the information technology world because there were so many large-scale failures that companies are being forced to take risks based on whether or not benefits are likely to be realized. In a private or public toll road situation, one could imagine something similar. For example, the private or public toll authority would promise a certain degree of congestion relief, and be willing to assume the risks, in exchange for rewards should the results materialize at a level greater than forecast--that is, the authority would get to keep the revenues that were above and beyond what was expected. The point is, if we start measuring outcomes, we will insist on consistent--not perfect-- measurement. We will see whether the benefits and costs realized match our projections and more importantly, are worthwhile. While we do not see much retrospective evaluation of how well we are doing, we must be able to do it--not so much in relation to ridership and revenue, but regarding economic benefit. If a project's evaluation could be extended into the domain of economic outcomes, we might see some better data and more frequent counts. If benefit realization must be measured, money will be budgeted to do that over a project's life cycle. Dr. TAYLOR: Do we see less retrospective evaluation in transportation than other areas? A huge amount of program evaluation goes on in health care and welfare--very detailed evaluations that anticipate changes, look back, and make adjustments. Why would there be more evaluation in other government endeavors than in transportation? We also have talked mostly about projects today. But huge federal and state transportation programs also can be evaluated using some of the tools we are discussing. Why a focus more on projects than programs? Dr. PICKRELL: Perhaps there is the same answer to those questions. My sense is that there have been relatively infrequent attempts to assess whether individual projects realized their forecasted costs, utilization, or revenue--and almost no assessment of their benefits' similarity to forecasts. However, there has been something of a cottage evaluation industry at the program level. I am not sure why, except that we did not do benefit-cost analyses of major programs until relatively recently. Transit projects in the 1970s and 1980s were planned and constructed under NEPA's environmental impact assessment process. This process provided most of the information needed to estimate what should have been a project's benefits, but it was almost never used to estimate benefits in the format that we are discussing today. FHWA has a tall stack of research that attempts to estimate the highway program's benefits using a wide variety of analytic and econometric methods. For example, David Aschauer[Footnote 66] and others did work on the program's macroeconomic benefits. Although somewhat uneven, there is a wealth of research on the subject--people have not shied away from it. Dr. GOMEZ-IBANEZ: One of my best Ph.D. students wrote a dissertation on transportation and poverty. The study compared 20 families who had cars and 20 families who did not--both poor and living in suburbs--and how they managed. Despite my hope that this student would have a transportation career, the student went into poverty research because that is where all the money is. Unlike transportation, where we do not evaluate projects--people just want to declare success when the streetcar starts rolling and be done with it--this student is evaluating demonstration projects. Perhaps there is less of a sense in the antipoverty, education, or health communities that they know what works. However, there is much more willingness to evaluate these projects. In transportation, there is an ideological battle between environmentalists and smart growth supporters--each with confidence that truth is on their side and demonstration is not needed. Sidebar: "…we do not evaluate projects--people just want to declare success when the streetcar starts rolling and be done with it…"-Dr. Gomez-Ibanez. [End of sidebar] Dr. TAYLOR: Perhaps controlled experiments are done much more easily and less expensively with individuals than with metropolitan areas. Individuals are routinely selected from different groups and the eligible pool at random and assigned to different treatments. In the metropolitan areas, we would need to treat one city with a light rail system and an identical city with an expressway system. It is harder to fashion the experimental or research paradigm in our context than in theirs. Dr. Gomez-Ibanez described the analysis as being entirely different--individuals, households, and smaller neighborhoods rather than metropolitan areas. Does that reflect institutional bias in the way these things are administered? Why not analyze the travel behavior of individuals, households, and firms? It could be like analyzing changes in poverty in one city versus another. Dr. GOMEZ-IBANEZ: But the policies we are discussing are applied at a metropolitan level. You treat the entire metropolitan area when you build or do not build the transit line. Dr. PICKRELL: We were in the waning stages of a very ambitious program to design and test various innovations in providing transit services-- the Services and Methods Demonstration Program--when I first worked for the U.S. DOT. Reports produced by that program were absolute models of how to generate all the information necessary to comprehensively analyze the value of the investment, innovation, and service that had been performed. However, the analysis invariably was left undone--it was not synthesized in the format that we have been advocating. Dr. SMALL: I served on a TRB committee, chaired by Dr. Wachs, that assessed the Congestion Management and Air Quality Improvement (CMAQ) program.[Footnote 67] One of our recommendations was to systematically do post-evaluations of projects, including cost-benefit analyses, because evaluations being done were very haphazard and infrequent. In addition, because CMAQ projects are experimental and innovative projects, it is important to find out about their benefits and what works--especially at the federal level. Dr. TAYLOR: There may be another significant difference between an antipoverty program like the Temporary Assistance for Needy Families (TANF)[Footnote 68] that moves people to paid employment and a building or facility where a public official can stand and cut a ribbon. The facility serves important political purposes--people have been hired to work and build it. It is less abstract than an employment project. On some level, it has accomplished things that its promoters argued that it would. A more sophisticated evaluation of whether it returns benefits to the degree that some other investment might have seems more academic than the fact that there is concrete in the ground. What benefit is there in going back and saying that--while this may be popular locally--it is a dog project? There is much risk and potentially little reward in those analyses. Dr. FORKENBROCK: Alan Altshuler's excellent book on megaprojects [Footnote 69] exhumes a lot of old projects and points out what might have been wrong with the forecasting and the process. Dr. LEWIS: In the information technology sector, companies that are proponents of large-scale projects are forced to measure outcomes and take risks in terms of their fee in relation to those outcomes. Might we not visualize a similar incentive structure in transportation? Large engineering and architectural companies often are proponents of large transit and highway projects. If they were on the hook financially for certain economic outcomes, the information and reason that goes into project evaluation and the information about whether transit or highway capacity represents the more effective solution--at least at the corridor level--might improve. I do not know how realistic that might be in the transportation world. However, it certainly did not seem very realistic in the information technology world 10 years ago--now it is the state of the art. Sidebar: "It is not coincidental that project evaluations do not occur--they would harm the purposes of many of those who put forward the programs…the issue is the geopolitics of resource allocation, not whether benefits exceed costs for a particular project."-Dr. Wachs. [End of sidebar] Dr. WACHS: It is not coincidental that project evaluations do not occur--they would harm the purposes of many of those who put forward the programs. It comes back to the opening point that Dr. Taylor made this morning--the issue is the geopolitics of resource allocation, not whether benefits exceed costs for a particular project. I have been attending symposia, discussions, and conferences for over 40 years where this point has been made. We have not just discovered it--it has a life of its own. Our elected officials have not responded to admonitions that more transportation funds should be spent on evaluation. Their answer clearly is that the benefits of doing so are smaller than the potential risks. Dr. MEYER: I am not so sure that the reason evaluations are so rarely done is that benefits are perceived as smaller than potential risks as much as it is limited money. If I were an MPO or state agency director and someone gave us money to do before and after evaluations, I probably would do them. The problem is that there is no money to do evaluations. I do not agree that evaluations are not done because proponents are afraid to risk discovering that they did something wrong. Dr. TAYLOR: Then you have to ask yourself, why is it different in health and education? Dr. MEYER: There is a very clear answer--evaluation is very common to the social sciences' function and service orientation. Transportation is infrastructure. Although we now talk about transportation customers and services, infrastructure programs do not have a tradition of doing before and after evaluations. It just has not happened. Dr. FORKENBROCK: In some ways, health and transportation have similar problems. My academic center has both health and transportation policy programs. In health, epidemiology is sort of a dark science. It is difficult to decide whether an intervention led to changed health status in people because some smoke and some do not, some are obese and some are not. Our legislature created a transportation program to invest in highways to promote economic development. Our center tracked each project that was funded by the program in order to determine whether it had the intended effect--did it create the number of jobs and produce the value added that was expected within 3 or 4 years. We ran into the difficulty of economic cycles. Someone promised to generate 400 new jobs, but needed better access to an interstate. So, we invested a couple of million dollars and poured a road for you. However, you have only increased employment by 75 people. Did the economy go bad or the industry go through a cyclical perturbation so that things are not the same as when we made the forecast? Was the forecast done with very sanguine assumptions or did the economic cycle change? The truth probably lies somewhere in between. It is very difficult to do economic impact forecasts and determine X years later whether that promised economic impact really occurred. Changes in people's taste, economic cycles, and a hundred other things can make that very difficult. Dr. PICKRELL: Something else may be at work here. My contact with public officials who oversee construction of major transportation infrastructure projects suggests that their view is that when the ribbon is cut, the important work is done. In an extremely cynical sense, that is what makes the local political machine work. If so, the important political part of the project is finished at exactly the time that its transportation function begins. I'm not sure I believe this, but offer it as a hypothesis. People are not consciously trying to cover up presumed failures--Dr. Meyer is right. This fear does not underlie the unwillingness to look back. It is admittedly inconvenient to know that a project failed to meet criteria on which it was justified, but there always is an explanation that has a superficial plausibility about it. Dr. GOMEZ-IBANEZ: A less cynical view would be that TANF can be redesigned every year so it is worth finding out how the program works and adjusting. But then you have to ask why communities that are building a transit system are immediately proposing the next extension. Why are these communities not interested in what happened? Dr. LEWIS: They could be interested if congressional appropriations committees that put a lot of federal money into New Starts,[Footnote 70] light rail, heavy rail, and highway projects insisted on performance reports and measurements of returns. This would be a means of getting a handle on return on the federal dollar, but not limiting it to the federal share. One reason this does not happen is no one with a big hammer says that you have to do it. If we could agree that doing so would be a good thing, this could be an incentive for these committees to write language insisting on it and put money into it or insisting that certain funds be designated for it. Dr. PICKRELL: What Dr. Lewis is describing is essentially how the cooperative research programs[Footnote 71] work--they are funded by a set-aside from the program. Moreover, their perspectives tend to be broader than an individual project. Perhaps we need a companion provision that a small fraction of project level funding be dedicated to post hoc evaluation, as the CMAQ report recommended. Dr. MEYER: I would like to ask Dr. Lewis about his discussion of the importance of applying risk analysis or its principles to improve benefit-cost analysis. What does this mean in practical terms? How would it benefit the methodology? Dr. TAYLOR: I would like to ask Dr. Lewis about his comments concerning looking at values and risks so that stakeholders define what is important. Is this a slippery slope to a completely value-laden process where people's perceptions of outcomes become divorced from empirical measures? Dr. LEWIS: I do not suggest that benefit-cost analysis be a broad, free- for-all conversation. I am advocating that the benefit-cost analysis discipline be applied both on the quantitative and value sides. The discipline--the logic and everything we have learned about cause and effect and measured values from revealed and stated preference studies[Footnote 72]--should be rendered accessible and brought to the public deliberative process to increase its quality. I am not suggesting that we ask people to come up with their own subjective values of time without being constrained by that discipline. I am not suggesting this in lieu of carefully measured value of time based on a good stated preference analysis or econometric analysis. Risk analysis is being done in biomedical research, other scientific research, and to some extent in transportation. I was involved in presenting value of time issues to a group that is discussing a light rail system for their city. I told them that their city's value of time estimate, based on a stated preference analysis, was $14.50 an hour during commuting time and 40 percent of that at other times. We tried to explain the statistical meaning of an expected value and used other associated statistics to present the rest of the curve--which inevitably will be a symmetric, bell-shaped distribution because of assumptions that have gone into the analysis. People saw that other values were possible. Then we layered in another level of uncertainty from studies that came from outside the community. We produced estimates from the Journal of Economic Statistics that present values of time as low as $3 or $4 an hour during commuting hours, based on the same kinds of stated preference methods. Bottom line: these other studies seem to provide evidence that there are extremes and other possible outcomes that differ from the expected value of time--the data also appears skewed toward the low end of time values. We redrew the probability distribution with a skew to the left, showing the probability that the truth lies more on the downside of the expected value than the upside. Then, we facilitated a discussion with local MPO planners where the subjective element would enter. We asked about the reality of value of time choices among communities on this corridor and how they would modify what we had. We used elicitation protocols[Footnote 73] that have been honed rather nicely in the biomedical world of statistical research. These protocols elicited beliefs from local experts and those with experience in observing traffic flows and household choices to add to the shape of this distribution. Remember, a Bayesian probability[Footnote 74] has to start from the frequency with which something happens--just a counting exercise. Bayesian is probability. It reflects the degree of belief that an expert or someone whose beliefs are convincing is right that something will happen with a certain probability. We tried to allow for a degree of subjective probability. During this discussion, people learned about the data--a bit about the central tendency,[Footnote 75] the down sides, probabilities associated with the down side. We went through the same exercise for statistical lives, injuries, and the environmental value of a ton of carbon dioxide. In my earlier airport runway noise example, we had property distributions based on econometric studies of the property value effects of a unit increase in overhead noise. We also brought in real estate agents and taught them how to think probabilistically so they could contribute to the shape of the distribution for that community. We ended up with what we believe is a scientific, locally informed representation of the probability range for values of time and life. We can do the same thing on the quantity side. The four-step congestion relief model is how uncertain are we about the cross elasticity of transit demand[Footnote 76] with respect to its generalized cost. We try to get at the distribution and how evidence from other studies informs the shape and breadth of this distribution. We then get local people's beliefs. The value of the elicitation process is its discipline. For example, if something is 30 percent likely to happen, then it is 70 percent likely not to happen. Simple disciplines like that smoke out some of the strategic behavior rather effectively. If you break the problem into its logical parts and attack one variable at a time, you can bring the discipline to bear on how people behave. Then you put it all into a Monte Carlo type simulation[Footnote 77] and see what the results seem to suggest. The valuation question is not asking people to invent a value of time number based on their anecdotal instincts. It is looking at scientific evidence and working their beliefs into the context of that evidence. In doing so, people are learning a great deal about the science that we bring to the table. Sidebar: "…one way to improve benefit-cost analysis as a tool in transit is to actually do it. It is not being done today…"-Dr. Lewis. [End of sidebar] Finally, one way to improve benefit-cost analysis as a tool in transit is to actually do it. It is not being done today, so I would hate to see us move on without that getting into the record. FTA's New Starts is many things, but it is not benefit-cost analysis. This means that neither we nor the Appropriations Committee will ever be able to compare the rate of return or the net benefit of building a light rail system along Highway I-71 to widening it instead. It seems to me that we must be able to compare different ways of using transportation money. How to improve it? Let's do it. [End of section] Summary of Panel Responses to Audience Questions: How can the quality and value of transportation data be improved to provide better information for decision makers and benefit-cost analyses? * Dr. Kirby emphasized that the first step is to thoroughly review the existing travel modeling process to document the need for better data and to focus on which data is most important to collect. He said that analysts are likely to need different kinds of information collected in different ways, such as GPS and smart cards,[Footnote 78] particularly as old methods of data collection such as household surveys are becoming more difficult to perform. He added that some transportation programs focus all their resources on developing very sophisticated modeling techniques rather than on collecting the better data that they really need. * Dr. Forkenbrock said that there is a need for a better behavioral understanding of urban trip making. He noted that researchers like Sandra Rosenbloom[Footnote 79] have shown that women's travel behavior in cities is more complex than men's travel and involves stops along the way, while traditional four-step travel modeling generates information on trips only from one traffic analysis zone to another. He said that an understanding of the complexity of trips is needed to assess the benefits of different transportation interventions. [End of section] Appendix VI: How Can Benefit-Cost Analysis Be Most Useful In Investment Decisions? GAO Questions: * What does benefit-cost analysis contribute to transit and highway investment decisions, what are the most appropriate roles it can play, and when should it be conducted? * Should benefit-cost analysis be required in planning transit and highway investments? * What is the federal role in improving the quality and use of benefit- benefit-cost analysis? * Do retrospective analyses of the performance of transit and highway cost investments have value? Michael D. Meyer is Professor of Civil and Environmental Engineering at the Georgia Institute of Technology. [See PDF for image] [End of figure] Opening Comments by Dr. Meyer and Dr. Gomez-Ibanez: Dr. MEYER: I thought it would be interesting to share information about how some other countries use benefit-cost analysis in decision making. I recently was in Australia, New Zealand, Japan, and Canada with a team to examine transportation performance measures and their use at the national, state, and provincial levels. In Australia, we found benefit- cost analysis being a very important part of the support structure for infrastructure decisions. As my paper describes, the State of Victoria portrays benefit-cost analysis as an important part of its decision- making process and describes it as a risk-based approach. [Footnote 80] The State of Victoria defines benefit-cost analysis in a very traditional way--much the same as we do here. We found that Australian state officials monitor some performance indicators on a yearly basis. These indicators relate directly to what they call a "return on construction expenditure" (RCE) and an "achievement index." The RCE is essentially a benefit-cost ratio. In Victoria, benefit-cost ratios are prepared for all road projects. Interestingly, road officials explain projects with benefit-cost ratios of less than one as being "political projects." Australian road officials also conduct before and after studies of the benefit-cost ratios themselves. A sample of implemented projects is targeted for the collection of after data to determine what the actual benefit-cost ratio is after approximately 2 years. The ratio of the "after" benefit- cost ratio to the "analysis" benefit-cost ratio is defined as the achievement index. One of our first questions was how much time they allow for achieving a steady state in benefits. Two years seemed to be the common time frame to develop the post-implementation benefit-cost ratio, although this varied because of the different scopes of the projects (everything from a left turn lane to major highway construction). The Victoria state government has an auditing agency that audits the analysis and the management activities of all state agencies. This independent, semiautonomous group actually provides another review of the quality of the decision support function in the transportation agencies. Australia also has identified performance indicators for the different states that are used to compare one state to another. Victoria officials emphasized that their approach is important to convince Australian parliamentary decision makers that VICRoads (the state road agency) has a good technical analysis process in place to justify its investments. AUSTRoads--a national organization like our AASHTO--recommends that all Australian states take this approach. VICRoads also is headed in the direction of putting risk analysis into their cost estimates--an issue that Dr. Lewis raised earlier. The risk analysis essentially says that the longer you wait to implement a project, the greater the uncertainty associated with benefit and cost streams. By doing this, VICRoads is trying to incorporate some sense of uncertainty into the project implementation time frame. This is interesting because it is really a different model than we would usually find in the United States. Turning to GAO's questions, the first one concerns the most appropriate roles benefit-cost analysis can play in contributing to transit and highway investment decisions. Dr. Lewis talked about bringing benefit- cost to the discussion of projects and I completely agree. It is a very important piece of information for investment planning and decision- making processes. It is not the only criterion that one would look at in decision making--no one here has said that--but it gives decision makers a sense of what it is they can get for the investment they are considering. I view benefit-cost analysis as one very important part of an evaluation process and decisionmaking framework. As we said earlier, benefit-cost analysis can be very important for looking at distributional calculations, if appropriately done. It can give a sense of what distribution may mean in either geographic and/or population group terms. In some cases, this can become an incredibly important factor in decision making. Benefit-cost ratios can become a very important element of how decisions are made about projects that are on a smaller scale. When should benefit-cost analysis be conducted? Clearly, the answer to this is before you make the decision! This is a bit facetious, but really an important point from the perspective of local, regional, and state government. The evaluation process for these agencies occurs during the planning process, when the relative values of projects and strategies are considered and when benefits and costs are defined. Similar to the Australian experience, we need to do some assessment of what happens after projects are put in place. VICRoads assesses a 10 percent sample of all projects implemented each year--about 40 projects out of approximately 400 projects. This provides feedback to the project analysis process and critically examines the effectiveness of the information produced that supports the decisions. Should benefit-cost analysis be required in planning transit and highway investments? Clearly, the answer is yes. Some analysis framework that looks at benefits and costs certainly should be required. If correctly done, benefit-cost analysis is one approach that could satisfy that requirement. What is the federal role in improving the quality and use of benefit- cost analysis? We have talked about providing dollars to improve data quality, supplying technical guidance and information relating to modeling and the dissemination of information, and distributing information about best practices. This is most likely the best federal role. Do retrospective analyses of the performance of transit and highway investments have value? Absolutely. Positively yes. No question about it. Dr. Jose A. Gomez-Ibanez is Derek C. Bok Professor of Urban Planning and Public Policy at Harvard University's John F. Kennedy School of Government and Graduate School of Design. [See PDF for image] [End of figure] Dr. GOMEZ-IBANEZ: I am only going to talk about the question of whether benefit-cost analysis should be required. The implicit question that I imagined was whether Congress should require benefit-cost analysis or something like it. One thing that struck me in this session is that Dr. Pickrell and Dr. Kirby both essentially said that all the ingredients for benefit-cost analysis are there. Local agencies are required, mainly by NEPA and the CAAA, to do elaborate analyses and generate much of the information you would need. We just do not take that extra step of doing the benefit-cost analysis. One of the interesting questions is whether benefit-cost analysis should be required--since it might not be so much of a burden--and what the consequences might be. There are many places we could gather information about how such a requirement would work. We have made analytic requirements, either benefit-cost analysis or something similar to it, for projects and local planning in many contexts and many ways. The 1934 Water Act mandated that the Army Corps of Engineers (COE) execute only projects whose benefits exceeded their costs. COE now has been doing benefit-cost analysis for 70 years. The FTA's predecessor, the Urban Mass Transit Administration (UMTA) was mandated to use cost effectiveness, which was somewhat like benefit-cost analysis. Analytic requirements have also occurred in other contexts. Planning- Programming-Budgeting (PPB)[Footnote 81] was required for systematic analysis of all federal government budgets in 1960. The State of Victoria's experience with analysis is not surprising. The British have required their department of transport to do formal benefit-cost analyses of all projects for decades. They take economics much more seriously than Americans. Each Canadian province has the Crown Corporation Commission do benefit-cost analysis of Crown Corporation major investments. Sidebar: "I recall several lessons from requiring and applying analysis…At its worst, the analysis requirements ended up being like an arms race where no one was much safer than they were before--they were just spending a lot more on weapons."-Dr. Gomez-Ibanez. [End of sidebar] I recall several lessons from requiring and applying analysis. One lesson is that two can play this game. When you require analysis, you put something at stake--such as when PPB was required or UMTA was first required to perform cost effectiveness analysis. With these requirements, agencies initially needed help doing the analysis. For example, there was considerable confusion in federal departments about how to do PPB. That put the Bureau of the Budget (BOB)--later the Office of Management and Budget (OMB)--at an advantage in dealing with departments. After training these departments, OMB had to be an analytic cop as the departments got good at PPB. Federal departments learned how to cook the analysis to favor their preferred alternative, and they got more and more sophisticated about it. At its worst, the analysis requirements ended up being like an arms race where no one was much safer than they were before--they were just spending a lot more on weapons. An even worse outcome at the national level is that analyses become so technical that the public drops out. Dr. Lewis' goal of informing local decisions gets lost. You hire two experts to dispute each other, newspapers play it up as a dispute, and the public generally reacts by thinking that if two economists cannot agree on whether the benefit- cost ratio is four or one-fourth, it must just be all garbage and we will ignore it. A second corollary is that benefit-cost analysis never ends up being the determinant of decisions. Benefit-cost analysis was never the determinant of what the COE did. Despite PPB, federal departments ended up doing what they always did--the analysis did not seem to change the outcome. And UMTA's New Start analysis--its cost effectiveness index-- did not stop earmarking, which I think was its goal. Dr. TAYLOR: They might argue that it has increased earmarking. Dr. GOMEZ-IBANEZ: It may have. But actually, I am going to counterargue. If you are thinking seriously about requiring benefit- cost analysis or revising the FTA cost effectiveness analysis measure as a tool, it really would be worth your while to read about COE's benefit-cost analysis, PPB, and UMTA's cost effectiveness analysis. This history would show some success in all these cases. Success is not measured by getting a portfolio of projects that is as profitable as what VICRoads is claiming. The Tennessee-Tombigbee[Footnote 82] Waterway and other examples of projects that had post-analysis requirements certainly belie that. An analysis requirement made it harder to advance really outrageous projects. Were it not for that requirement, you would have had more Tennessee-Tombigbees. In the case of FTA's cost-effectiveness analysis, projects that have actually moved toward substantial construction funding are the ones that are not laughable on the cost-effectiveness test. In that sense, it has done a service. I would argue for making the benefit-cost analysis simple and understandable rather than too complex. This is where Dr. Lewis and I might part company a bit. I was involved in the discussions about requiring cost-effectiveness analysis in the federal transit program. Transit capital program earmarking really started later--after President Reagan was elected and then OMB Director David Stockman was zeroing out transit rail New Starts. Congress responded by appropriating the money. DOT reacted, in part, by trying not to spend the money. Congress then responded by earmarking. Congress was saying that if you are not going to spend what we have approved, we are going to tell you exactly what to spend. At this point, DOT realized it had lost control and that the end result would not be good for the program's health. In changing the debate to get projects of higher public value implemented, DOT faced the dilemma that we face as benefit-cost practitioners--being either precisely wrong or approximately right. It decided to calculate cost- effectiveness using dollars per new rider attracted--not a full benefit- cost analysis. Instead, DOT said if the purpose of New Starts is to get new rail riders, then you must calculate cost effectiveness in doing so. This approach required analytic police because everybody now was cooking the forecasts. Still, my impression was that FTA recognized that this was not the only test--it was just one of several tests. If you had an embarrassingly low or high dollar cost per new rider, it put a greater burden on you to argue that your project ought to be earmarked in Congress. In that sense, I think it worked. In this case, DOT did not pretend that this crude cost-effectiveness index or benefit-cost analysis should be the sole determinant of which projects the federal government funded. It was recognized as only part of the debate. However, it did end up shaping the argument in a way that was useful to help stop or control the most wasteful projects. Over the years, the value of that requirement eroded. It may have been changed to time measures--dollars per hour or minutes saved, in response to big cities that did not want New Starts, but wanted to improve or renovate existing service. Slowly, its power got lost. Still, it was a useful requirement. My guess is that requiring a benefit-cost analysis or something like FTA's New Starts cost-effectiveness measure would help shape the debate in ways that Dr. Lewis wants and not be a complete waste of time. However, insisting that benefit-cost analysis determine local or federal decisions is asking for a lot of trouble. Panel Discussion: Dr. WACHS: I think we can learn something relevant to Dr. Gomez- Ibanez's statement by looking at the history of NEPA. When NEPA was debated, one proposal was to require the environmental impact review to be done by an independent agency, rather than the agency preparing the project. The genius of the decision to give the agency that prepared the project this responsibility was that it infused these agencies with an environmental capacity that they would not have had if this review had been done by an independent agency. In addition, the decision embedded the environmental impact requirement in a process that forced democratic debate. The draft environmental analysis is subject to review, and must be presented at a hearing for public comments--and in turn, the comments must be responded to by the agency. I would take Dr. Gomez-Ibanez's recommendation and add that if transportation agencies had to prepare benefit-cost analyses, they would become more expert. Such a requirement probably should be embedded in a public process that subjects benefit-cost calculations to public comment and review. Dr. GOMEZ-IBANEZ: Makes sense. Dr. SMALL: One question that we did not get to in the last session was ensuring the objectivity of benefit-cost analysis. I believe that Dr. Pickrell's article about transit systems and their forecasts[Footnote 83] suggested peer review--just as a simple requirement that is not very onerous and shines a light on what is done. That seems to fit right into keeping a requirement from becoming a total game-playing exercise. Sidebar: "…these large infrastructure investment decisions are inherently political decisions. They should remain so, although I would advocate a stronger role for some systematic analysis to inform and organize the discussion that accompanies that political decision process."-Dr. Pickrell. [End of sidebar] Dr. PICKRELL: I now have a different perspective on the peer review requirement and would not make the recommendation again. It may have been naive, given the success that project sponsors have had in gaming the process by loading peer review boards with people selected from a cadre of supporters. There is a bit of log rolling or back scratching. However, peer review probably has made things marginally better--not worse. Regarding a potential requirement for benefit-cost analysis in public decisions, I would observe that these large infrastructure investment decisions are inherently political decisions. They should remain so, although I would advocate a stronger role for some systematic analysis to inform and organize the discussion that accompanies that political decision process. In the current funding environment, I believe that local officials who make these decisions simply are incapable of seeing the benefits and costs accompanying their decision alternatives from the broad perspective that is required to accomplish all the objectives we have been talking about today--avoiding multiple counts of impacts, considering distribution and downstream impacts, and thinking carefully about which impacts genuinely add to a project's benefits versus simply shifting benefits around. The current federal structure and funding for transportation infrastructure investment provides no incentive for local political officials to engage in the systematic, forthright analysis that we are recommending. Recent integrated funding legislation for transit and highway programs has helped the situation a bit by moving slightly toward a unified funding process for transportation. Superficially, it looks as if we have a unified transportation funding structure at the federal level. But there is still pretty extreme compartmentalization of the individual funding programs under the umbrella authorization legislation. Firewalls among the funding programs remain relatively strong and tend to be flexible--where they are at all--in only one direction.[Footnote 84] In this funding environment, there is simply not much incentive for local officials to engage in the kind of analysis that most of us seem to be advocating. My guess is until an infrastructure investment decision imposes an opportunity cost on the officials making it by drawing down on their ability to make competing investments, any requirement to conduct systematic analysis will be a pro forma activity in somewhat the same way that the EIS has become--checking off boxes and listing mitigation measures. Responding to public comments on an EIS can become a pro forma process as well. Inevitably, it is a process of going down the list and making sure that you have said something in response to each of the 43,000 letters that have been elicited from property owners along the northeast corridor rail line or something like that. Until the federal funding environment changes, any requirement to perform analysis will either be treated in a pro forma way, or worse, gamed in the way that Dr. Gomez-Ibanez was describing. A requirement will create the need for a policing agency. OMB seems to serve quite enthusiastically and capably in that capacity. And this does improve the analysis--there is no question. But I do not think you want the threat of having to negotiate the completeness and forthrightness of your analysis with OMB being the enforcement mechanism. I think you want to provide some incentive for project sponsors to willingly engage in the kind of process that we are describing. Sidebar: "The whole federal transportation program is currently structured precisely to avoid tradeoffs between modes and different sorts of investments."-Dr. Taylor. [End of sidebar] Dr. TAYLOR: I want to go outside the moderator's role to say that I completely agree with your point. Internalizing these opportunity costs is really key. The whole federal transportation program is currently structured precisely to avoid trade-offs between modes and different sorts of investments. There is a lot of uncertainty in that process and this creates considerable transaction costs for the people involved.[Footnote 85] As a result, the policy makers involved in federal transportation funding decisions negotiate politically on how to divide the pie. Then the federal funding for different transportation programs is established. Regions and states that compete for federal money or projects are simply engaging in a gaming process. There are zero opportunity costs involved in doing so. Trade-offs can never be internalized, despite enforcement or regulatory mechanisms, as long as there are those strict divisions of transportation funding programs. The goal is always to maximize funds that can be accessed from each funding source. A real key is involving those who are trying to divide up the pie in project level decisions--this is where the honesty is going to come in. It is a pretty fair process when different interests compete for dollars and it puts a lot more rigor in the analyses. For example, California's Transportation Development Act essentially ensures that funds in smaller counties go first to transit. If all reasonable transit needs have been met, the money can be used for streets and roads. Transit agencies argue that they need to spend this money, and county road departments challenge their analyses. This is much different from situations in which entities in the same categorical area are competing. However, federal funding programs are designed specifically to avoid just those kinds of choices--I think because of a lack of trust. Dr. GOMEZ-IBANEZ: I have heard for ages that we would be better off with consolidated transportation programs to allow trade-offs, etc. Richard Nixon tried it with categorical grants and he did not get far. I do agree that it would make the incentives very different. But it is a counsel of despair to say that, absent consolidation, there is not much we can do. This does not recognize that there are other allies for different points of view--particularly at the local level. In almost any local project, you will find people who are worried that the transit line is not going to the right section of the metropolitan area or taxpayers are going to have a white elephant on their hands. My guess is there are natural local allies on the other side to fuel debate. Even in the transportation agency that is advocating a project, there may be people who will tell a city council member that benefits will not exceed the costs of light rail for the city. There probably were COE people who knew that water systems were scarce resources and were glad to have the benefit-cost test to protect them from the worst projects being pressed on the Army. Dr. KIRBY: I agree that discipline has to come from competing interests at the local level. Absent a complete restructuring of the federal transportation program--which is unlikely--we must do better with the process we have. MPO and EIS requirements really bind us, particularly conformity requirements with regard to model structure and latest data. We struggle to meet conformity requirements because no one wants to be responsible for the whole regional plan being disapproved when it goes to DOT and federal money held up--which is the prospect if we do not do everything right. Similarly, you do not ignore EIS environmental requirements or you will be in court and your project will stop. So to some degree, those things work. Although the conformity rule is very well specified--we know what we have to do and we had better do it--other requirements are not. In particular, MPOs need much clearer, better specified descriptions of what FTA is requiring them to do. It is difficult to find a piece of paper that describes FTA's requirements for technical procedures and the requirements seem to change from time to time. This is a process that has not been fully specified and is difficult to follow in practice. There could be the need for a more prescriptive requirement from Congress through DOT. There is great potential to really improve our processes and the value of what we are doing. Knowing clearly what you must do from the start would provide discipline and head off a lot of problems. And local level competition plays a role. You are only going to get so much funding, and you do not want to put all your effort into a project 81 that is not going to make it through the approval process. Local interests recognize this and will propose working on projects that have the best chances of getting through the process. Dr. LEWIS: There are two ways to interpret congressional interest in requesting this study.[Footnote 86] Congress can be asking what can or should be done to improve the role of benefit-cost analysis in improving decision making. If that is the question, I buy into what Dr. Gomez-Ibanez and Dr. Kirby have just said. However, the question may be what can Congress do to help improve things--what Dr. Gomez-Ibanez and Dr. Kirby are saying suggests actions to introduce more local level resource trade-off analysis. One thing that Congress--particularly the appropriators--can do is to encourage both FHWA and FTA to look at more than how to spend fixed allocations. I think that appropriators want to know if a $1.1 billion New Starts budget is well spent compared to alternatives, such as highway expenditures, no further expenditures, or noncapital projects. One thing that we have not dealt with today, although Dr. Forkenbrock touched on it, is the role of benefit-cost analysis in looking at noncapital solutions. If appropriators want to encourage more comprehensive appraisal of how New Start projects and highway proposals line up against transit alternatives, I would hope that that could be a requirement--notwithstanding the fact that an enforcement modality is never the best incentive. Then I would agree that the apparatus and ingredients for doing that are in place--it is the incentive for doing it that is lacking. Summary of Panel Responses to Audience Questions: Do the current federal transportation financing and funds distribution structures suboptimize the federal dollar, since the rate of return for the federal transportation dollar keeps going down? * Dr. Lewis said that while he would not counsel using benefit-cost analysis as a wedge to restructure the entire financial and budget framework, some small steps would help, including (1) Congress legislatively declaring its interest in an economically rational allocation and information to address this allocation and (2) Congress insisting that modal agencies conduct benefit-cost analysis from a multimodal perspective. He added that motivation for efficiency can also come from outside the federal financing framework. As an example, he said that in one location, there was interest in building a light rail system because federal dollars were available, but that industrial and commercial executives lobbied to use a benefit-cost framework first to determine whether a light rail system or widening highways made more sense. * Dr. Meyer said that what is optimal to one person may not be to another. As an example, he said that building an interstate highway network benefited some states more than others, but that all states agreed to build that national network. When the interstate system was complete, many state governments wanted to deal with their own problems rather than sacrificing so that others could get the national network in place. While this may be suboptimal to some from a national perspective, others would say that states have provided their contribution and now need to spend money on their states' problems. * Dr. Forkenbrock added that the argument for states to get their money back for use on local priorities is gaining strength and that many people are arguing against having a federal program at all, bringing into question FHWA's future. * Dr. Taylor added that a recent dissertation showed that the federal highway program generally was redistributing funds from those with less financial capacity to those with more financial capacity. What practical advice would you give on structuring an ex post analysis[Footnote 87] of a transportation project when the analysis at the beginning of the project was not necessarily done for comparison purposes? The Senate reauthorization bill[Footnote 88] requires some type of New Starts ex post analysis that focuses on ridership, and FTA is trying something similar. Are there other benefits that are more appropriate to analyze? * Dr. Forkenbrock recommended looking at where ridership comes from, as rail ridership often is gained at the expense of bus ridership. * Dr. Pickrell said that he thought either substantial travel time savings or ridership increases were the primary benefits to measure. He added that ex post evaluation is better than anticipatory evaluation because in anticipatory evaluation you have to simulate the results of two possible decisions; whereas in ex post evaluation at least you know the results of one decision--that of building the project--and have to simulate only what would have happened if the project had not been built. * Dr. Small said that one need not be deterred if no benefit-cost study was done before the project was implemented, as changes during a project's construction often make it difficult to compare studies done before and after a project in any case.[Footnote 89] Would clearly specified standards for conducting benefit-cost analyses help increase the impacts of these analyses on policy decisions? * Dr. Kirby said that adding requirements to the process would improve decisions. He also said that Dr. Lewis' previous comment about a business community's interest in a benefit-cost analysis of transit lines showed that involving more people can create a constituency for good analysis, making it more likely that tough questions will be asked and that the focus will be on projects that are good for the community. He cautioned that bad decisions can result when the agency building a project works closely with the agency funding the project. A larger constituency asks tough questions and focuses on projects that are good for the community. * Dr. Meyer agreed that a constituency for analysis was important and said that adding requirements would increase the constituency for good analysis. [End of section] Appendix VII: Panelists' Closing Remarks: Dr. TAYLOR: What remains on our schedule is for me to offer a 5-minute summary of what has been 6-1/2 hours of discussion. I have 17 pages of notes and 444 points that were made. Instead, I will ask each panelist to repeat what he thought was the most important thing for people to carry away from all of this. Dr. PICKRELL: I have been encouraging you to think about how to revamp the funding structure for transportation infrastructure programs to encourage investors--public agencies--to get benefits and costs on the correct side of the ledger. If you could do that one thing, you will have accomplished more than I will have in my entire career. Dr. SMALL: One of the strengths of benefit-cost analysis is that it offers a unifying principle or consistent measure that can cut across many of the questions that we have discussed today--secondary benefits and so on. If you consistently keep in mind the key principle that the benefits we are trying to measure are people's willingness to pay for things and the costs are what people would be willing to pay to avoid, you can answer many conundrums about double counting and externalities. Dr. KIRBY: A major feature of the federal transportation program is the process you go through to use federal funds, for example, the environmental impact statement and planning process. Many people value that process very highly, and I would consider that to be one reason for the federal government to be funding transportation. If you can strengthen the benefit-cost requirement in that process through something analogous to the present conformity rule and environmental impact statement process and get more people involved in looking at these projects, you will get better projects and have a stronger rationale for federal involvement in transportation. Dr. GOMEZ-IBANEZ: The reading that most struck me was Dr. Lewis's piece about requiring benefit-cost analysis--not in the false hope that it might be the answer and determine everything, but with the idea that it would stimulate a better debate and wiser decisions. Since Dr. Kirby says that we require everything short of a benefit-cost analysis already, it seems a small step to require that of federally-funded projects. Dr. LEWIS: The main point I want to leave with you is the importance of comparing major capital projects across modes. I think we have the tools, the apparatus, and--even with huge imperfections--the models and the data. What we lack is a demand from Congress and from the broader constituency for good analysis for information about returns on investment among transit versus highway alternatives. Benefit-cost analysis makes this possible. Dr. FORKENBROCK: I agree. I would extend it to my great concern from watching MPOs, state transportation agencies, and other transportation agencies that many people who do benefit-cost analyses are not doing them very well. Perhaps one reason is that they have never really been given the opportunity to learn the rudiments about costs. There is this tremendous gap between the state of the art in benefit-cost analysis and the state of the practice. Anything that GAO can do to help MPOs, state DOTs, and other agencies to do better benefit-cost analyses would be a huge step forward. If benefit-cost analyses are not being done correctly, you run the risk of creating misinformation or--even worse- -disinformation for decision makers. That leads to bad investment decisions. Dr. MEYER: Much of our discussion has revolved around the question of how to enhance the role of benefit-cost analysis in decision making-- and we all agree that decision making is inherently a political process. I think you do this through a requirement that makes benefit- cost analysis part of the process for looking at federally supported transportation investments. This is not to say that decision makers will necessarily do what benefit-cost analysis indicates, but it brings more information to the table that will become much more useful for that decision-making process over time. Dr. WACHS: I have three points in closing. * First, benefit-cost analysis and the political nature of our decision making about large public works projects, including transportation projects, is a constant struggle to find the right balance between subjective judgments that our elected officials can make very effectively and being informed by the rational analysis that benefit- cost analysis contributes to the process. We will probably not be very effective if one of those two strains completely dominates the other. What we are trying to find is a balance. We are trying to inform the political decision making with good information. To me, that requires attention to the process in which the benefit-cost analysis is embedded as well as to the tools, techniques, and data. I would be really careful to ask questions about public hearings, about process, about the order in which analysis and debate take place and about feedback from the analysis to the political process. It is possible to have a very good benefit-cost analysis that is completely politicized, and the goal would be to have an informed political process in which the analysis is reasonably objective. That is a process design problem more than it is dealing with the subtleties of benefit-cost analysis itself. * Second, I heard that the quality of data, data collection and analysis, and tools and techniques of benefit-cost analysis are very, very important. * Third, this is a very homogenous group. It might be useful to have other constituencies address the same questions that we did-- environmentalists, shippers, state highway officials who were not terribly well represented in this group. You might find some other answers that would complement our perspectives. Dr. TAYLOR: I am hearing some consensus that the decision making process can only be improved by incorporating a federal requirement for benefit-cost analysis. However, it is important to incorporate such a requirement into a deliberative process. The debate we had about the idea of a checklist and the danger of developing requirements that become like a bureaucratic checklist is really important. A bureaucratic checklist probably is the worst alternative we could envision. Any federal requirement needs to be crafted to ensure that benefit-cost analysis informs decision making as opposed to complying with requirements. [End of section] Appendix VIII: Selected Bibliography and Related GAO Products: Related Articles Suggested by Panelists: Brian Taylor: Hill, M.C., B. Taylor, A. Weinstein, and M. Wachs, "Assessing the Need for Highways," Transportation Quarterly, Spring, 2000, pp. 93-100. Taylor, B. and K. Samples, "Jobs, Jobs, Jobs: Political Perceptions, Economic Reality, and Capital Bias in U.S. Transit Subsidy Policy," Public Works and Management, Vol. 6, April 2002, pp. 250-263. David Forkenbrock: Forkenbrock, D.J. "Some Technical Issues in Benefit-Cost Analysis of Potential Transportation Investments," July 2004. Forkenbrock, D.J. and Norman S.J. Foster, "Economic Benefits of a Corridor Highway Investment," Transportation Research-A, Vol. 24A, No. 4, 1990, pp. 303-312. Michael Meyer: T. Partridge & Associates Ltd, "Translink Transportation Evaluation Guidelines," October 11, 1999. Kenneth Small: Small, K.A., "Project Evaluation" in Essays in Transportation Economics and Policy--A Handbook in Honor of John E. Meyer (Jose Gómez-lbánez, W. B. Tye, and C. Winston, eds.), Brookings Institution Press, Washington. D.C: 1999. Brownstone D. and K.A. Small, "Valuing Time and Reliability: Assessing the Evidence from Road Pricing Demonstrations." Abstract, October 9, 2003. David Lewis: Lewis, D., "Making Cost-Benefit Analysis More Useful in Policy Making" in The Reform of Cost-Benefit Analysis by David Lewis, forthcoming from Ashgate Press. Lewis, D., "The Case for Increasing Transit Capacity to Mitigate Congestion," presented at Conference on Traffic Congestion: Issues and Opportunities, UCLA Extension Public Policy Program and Institute of Transportation Studies. Washington, D.C.: June 26-27, 2003. Don Pickrell: Mohring, H., "Maximizing, Measuring, and Not Double Counting Transportation Improvement Benefits: A Primer on Closed-and Open- Economy Cost-Benefit Analysis," Transportation Research-B, Vol. 27B, No. 6, 1993, pp. 413-424. U.S. DOT, Office of the Secretary, "Treatment of Value of Life and Injuries in Preparing Economic Evaluations," January 8, 1993 and revised January 29, 2002. U.S. DOT, Office of the Secretary, "Departmental Guidance for the Valuation of Travel Time in Economic Analysis," April 9, 1997 and revised February 11, 2003. Cohen, J.P. and C.C. Coughlin, "Congestion at Airports: The Economics of Airport Expansions," The Federal Reserve Bank of St. Louis, May/ June 2003. Related GAO Products: Federal-Aid Highways: FHWA Needs a Comprehensive Approach to Improving Project Oversight, GAO-05-173 (Washington, D.C.: Jan. 31, 2005). Highway and Transit Investments: Options for Improving Information on Projects' Benefits and Costs and Increasing Accountability for Results, GAO-05-172 (Washington, D.C.: Jan. 24, 2005). Federal-Aid Highways: Trends, Effect on State Spending, and Options for Future Program Design, GAO-04-802 (Washington, D.C.: Aug. 31, 2004). Surface Transportation: Many Factors affect Investment Decisions, GAO- 04-744 (Washington, D.C.: June 30, 2004). Mass Transit: FTA Needs to Better Define and Assess Imp[act of Certain Policies on New Starts Program, GAO-04-748 (Washington, D.C.: June 25, 2004). Freight Transportation: Strategies Needed to Address Planning and Financing Limitations, GAO-04-165 (Washington, D.C.: Dec. 19, 2003). [End of section] Appendix IX: GAO Contacts and Acknowledgements: GAO Contacts: * Katherine Siggerud (202) 512-2834; * Nikki Clowers (202) 512-4010. Acknowledgments In addition to those named above, Jay Cherlow, Libby Halperin, Sara Ann Moessbauer, Andrew Von Ah, and Alwynne Wilbur made key contributions to this report. [End of section] Endnotes: [1] National Association of State Budget Officers. 2003 State Expenditure Report, 2004 (Washington, D.C.) [2] U.S. Department of Transportation, 2002 Status of the Nation's Highways, Bridges, and Transit: Conditions and Performance, Report to Congress (Washington, D.C.: 2002). Estimates are in 2000 dollars. [3] ISTEA (P.L. 102-240) authorized federal transportation programs for highways, highway safety, and transit for fiscal years 1992 to 1997. [4] TEA-21 (P.L. 105-178) authorized federal surface transportation programs for highways, highway safety, and transit for fiscal years 1998 to 2003. The act has since been extended. [5] GAO, Highway and Transit Investments: Options for Improving Information on Projects' Benefits and Costs and Increasing Accountability for Results, GAO-05-172 (Washington, D.C.: Jan. 24, 2005). [6] GAO, Surface Transportation: Many Factors Affect Investment Decisions, GAO-04-744 (Washington, D.C.: June 30, 2004). [7] For more information about the use of benefit-cost analysis in transportation investment decision making, see GAO-05-172. [8] NEPA (P.L. 91-190) declares that it is national policy to use all practicable means to create and maintain conditions in which man and nature can exist in productive harmony, among other purposes. It requires federal agencies to integrate environmental values into their decisions by considering the environmental impacts of their proposed actions and reasonable alternatives to those actions. [9] Earmarking refers to dedicating appropriations for a particular purpose. [10] HOV lanes promote ridesharing by providing dedicated lanes on a highway for buses, vanpools, and carpools. Bus Rapid Transit is designed to provide major improvements in the speed, reliability, and quality of bus service through barrier-separated busways, high- occupancy vehicle lanes, or reserved buses or other enhancements on arterial streets. For more information about bus rapid transit, see GAO, Mass Transit: Bus Rapid Transit Shows Promise, GAO-01-984 (Washington, D.C.: Sept. 17, 2001). [11] ROI is one of several approaches that decision makers use to evaluate the investment potential of projects or actions. ROI is a ratio that compares the net benefits of a project to its total costs. [12] Professor Sen was awarded the Nobel Prize in Economic Sciences in 1998 for his contributions to welfare economics. See Amartya Sen, "The Possibility of Social Choice," Lecture Delivered in Stockholm, Sweden, on December 8, 1998. [13] Welfare economics studies how efficiently an economy distributes income and the consequences that are associated with it. It is concerned with the welfare of individuals--rather than groups or societies. [14] David J. Forkenbrock and Norman S. J. Foster, "Economic Benefits of A Corridor Highway Investment," Transportation Research-A. No. 4, (1990), 303-312. [15] David Lewis, "Making Cost-Benefit Analysis More Useful in Policy Making." Paper (June 22, 2004) drawn from the manuscript of Dr. Lewis's forthcoming book from Ashgate Press. [16] Smart Growth seeks to accommodate development by focusing on environmentally sensitive land development with the goals of minimizing dependence on auto transportation, reducing air pollution, and making infrastructure investments more efficient. [17] NEPA requires federal agencies to integrate environmental values into their decision making by considering the environmental impacts of their proposed actions and reasonable alternatives to those actions. To meet this requirement, agencies prepare a detailed statement known as an EIS for all major federal actions that could have significant effects on the environment. [18] Agglomeration refers to the tendency of firms to cluster close to each other, or to residences, presumably due to advantages this gives them in production. [19] Externalities are direct spillover effects on third parties that result from production and/or consumption of goods and services for which no appropriate compensation is paid. [20] Dr. Small noted that he owed this answer to a discussion with Anthony Downs and Katharine Bradbury. Downs is a Senior Fellow at the Brookings Institution in Washington, D.C. He is the author or co-author of numerous books, including "Still Stuck in Traffic: Coping with Peak- Hours Traffic Congestion," Brookings Institution Press. Washington, D. C., 2004. Katharine Bradbury is a Vice President and Economist at the Federal Reserve Bank of Boston. Her research focuses on the regional economy and state and local public finance and income inequality. [21] Dr. Forkenbrock mentioned a study by a number of Midwestern states: Transportation Economics & Management Systems, Inc., in association with HNTB Corporation, Midwest Regional Rail System Executive Report, September 2004. Prepared for the Illinois, Indiana, Iowa, Michigan, Minnesota, Missouri, and Wisconsin Departments of Transportation, the Nebraska Department of Roads, and the Ohio Rail Development Commission. [22] Dr. Pickrell recommended the following paper: Herbert Mohring and Harold F. Williamson, Jr., "Scale and 'Industrial Reorganization' Economies of Transport Improvements," Journal of Transport Economics and Policy 3. September 1969, pp. 251-271. [23] GDP is the total value of all goods and services produced within a territory during a specified period. [24] Guidance from the U.S. Department of Transportation/Office of the Secretary, "Treatment of Value of Life and Injuries in Preparing Economic Evaluations," January 8, 1993; Revised on January 29, 2002. [25] Indirect impacts include changes in land use and development, changes in decisions to locate homes and businesses, etc., and changes in operations that businesses make to take advantage of improved transportation system speed and reliability. These impacts lead to increased property values, increased productivity, employment, and economic growth. There also can be indirect costs, such as reduced land values for regions that might lose economic activity that is diverted to an area where transportation is improved. [26] Multiplier refers to the cumulatively reinforcing interaction between consumption and production that amplifies changes in investment, government spending, or exports. Multipliers are called estimators of the "ripple effect" in an area. [27] REMI Policy Insight is the economic forecasting and policy analysis model of Regional Economic Models, Inc. [28] Disutility of time refers to the value that people place on reducing time spent in a particular activity. To the extent that travel time is less pleasant than time spent in other ways, people may be willing to pay to reduce their travel time. Therefore, travel time can be said to have disutility. See Jay R. Cherlow, "Measuring Values of Travel Time Savings," Journal of Consumer Research Vol. 7 (March 1981). [29] Ian Savage, "An Empirical Investigation into the Effect of Psychological Perceptions on the Willingness-to-Pay to Reduce Risk," Journal of Risk and Uncertainty 6(1): pages 75-90, 1993. [30] Opportunity cost is the value of the best alternative given up when making a choice. [32] Probabilistic analysis allows for the range of all possible outcomes rather than the single most likely outcome. [33] Bayesian logic applies to decision making and inferential statistics that deal with probability inference--using the knowledge of prior events to predict future events. The way to quantify a situation with an uncertain outcome is to determine its probability. [34] Positive economics deals with objective, relatively testable statements that focus on descriptions that do not reflect obvious value judgments and predictions about economic relationships. Normative economics deals with subjective statements based on opinion about "what ought to be." [35] A microsimulation model is a tool used to evaluate the impact of changes on a system. For example, such an approach to travel demand forecasting might integrate household activities, land use distributions, regional demographics, and the transportation network to estimate the impacts of converting a highway intersection into a cloverleaf. [36] The Four-Step Urban Transportation Planning Process integrates various aspects of travel behavior (trip or mode choices) with information on land use patterns and the transportation network. The four steps include: trip generation, trip distribution, mode choice, and assignment to specific parts of the transportation network. [37] Frank Ackerman and Lisa Heinzerling, Priceless: On Knowing the Price of Everything and the Value of Nothing. The New Press, 2004. [38] Joseph Berechman, "Transportation Economic Aspects of Roman Highway Development: The Case of Via Appia." Transportation Research Part A, Vol. 37 (2003), pp. 453-478. [39] Distributional issues refer to who gets what and how much they get of benefits or items, such as money, land, transportation facilities, services, etc. [40] HOT lanes are limited access, barrier-separated highway lanes that provide free or reduced cost access to qualifying high-occupancy vehicles and also provide access to other paying vehicles that do not meet passenger occupancy requirements. [41] D. Brownstone and K.A. Small, "Valuing Time and Reliability: Assessing the Evidence from Road Pricing Demonstrations," Working Paper. Oct. 2003. [42] John Rawls, A Theory of Justice, 1971, (Revised edition, 1999). Belknap Press. [43] The Americans with Disabilities Act of 1990 (P.L. 101-336) prohibits discrimination of various sorts against persons with physical or mental handicaps. It emphasizes employment and outlaws most physical barriers. [44] A Pareto improvement is based on the notion that an action improves efficiency if it is possible for one person to benefit without anyone else being harmed. A Pareto improvement is possible if the economy has idle resources or market failures that can be corrected without hurting others. [45] Disbenefits also can be described as disadvantages. [46] Induced demand is the phenomenon that more of a good/service is consumed after the supply increases. [47] A discursive benefit-cost exercise is the conduct of a benefit- cost analysis within a formal public process that engages stakeholders in the analysis process. [48] The Federal Highway Administration uses the HERS model to estimate future investment requirements of the nation's highway system for Congress. The HERS-ST model is a version of the HERS model that allows states to simulate future highway conditions and performance levels and identifies deficiencies using engineering principles. It then simulates the selection of improvement projects, applying economic criteria to estimate the most cost-beneficial mix of improvements for system-wide implementation. [49] David Forkenbrock, "Improving Benefit-Cost Analysis" (Washington, D.C.: 2004. Photocopy). [50] Section 1105 of ISTEA contained corridor provisions and identified 21 corridors of significance. This identification allowed states to give funding priority to these corridors, provided federal funding to specific projects on these corridors, and directed other benefits to these corridors. When TEA-21 was enacted, there were 29 corridors, which increased to 44 by 2002. [51] Discounting is the process of comparing current and future values that finds the present worth of a future amount of money. [52] MPOs are regional transportation policy bodies made up of representatives from various government and other organizations. The Federal-Aid Highway Act of 1970 required that such agencies be developed in areas with populations of more than 50,000 to carry out cooperative planning at the metropolitan level. MPOs are responsible for planning, programming, and coordinating federal highway and transit investments in urbanized areas. [53] The Kaldor-Hicks criterion holds that for a change in policy to be viewed as beneficial, the gainers should be able to compensate the losers and still be better off. The criterion does not require that compensation actually be paid. [54] Distributional effects also are described in OMB Circular No. A- 94, Office of Management and Budget--Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs. (Washington D.C.; 1992). [55] Parameter values are assumed or estimated values of key components of an analysis (such as a statistical life saved). These values are used with observations on key explanatory variables (such as the number of lives saved) to estimate the value of the variable being studied (such as benefits). [56] In general, the economists' theory of the second best says that the application of policy rules that are designed to enhance net economic welfare--such as benefit-cost analysis for public investment decisions--might not enhance economic welfare if elsewhere in the economy there are distortions from economic efficiency. In this context, the second best issue is that the demand for travel on a new road would be much less if users had to pay the full cost that their additional travel causes than if, as is true under current road pricing policies, users pay only part of the cost. Hence, the benefit-cost ratio of an investment decision could be quite different if the distortions introduced by the fact that individuals and businesses do not take into account the full benefits and costs of making their decisions are removed. [57] American Association of State Highway and Transportation Officials, A Manual of User Benefit Analysis for Highways, 2003. [58] Sensitivity analyses assess how sensitive outcomes are to changes in assumptions. The assumptions that deserve the most attention should depend largely on the major benefits and costs and areas of greatest uncertainty in the program or process that is analyzed. [59] Highway cost allocation studies are used to evaluate the highway- related costs attributable to different vehicle classes and the user fees they pay. Comparing user fees and cost responsibility indicates the relative equity of highway user fees. [60] The CAAA were passed in 1990 (P.L. 101-549). The CAAA's conformity requirements specify that MPOs and the U.S. DOT determine that transportation plans and programs in areas that do not meet federal air quality standards set by the National Ambient Air Quality Standards move toward reducing pollutant emissions to meet these standards. [61] MOBILE6 Vehicle Emission Modeling Software is a model for predicting gram per mile emission of hydrocarbons, carbon monoxide, nitrogen oxides, carbon dioxide, particulate matter, and other toxics. [63] GPS is a system of satellites, computers, and receivers that is able to determine the latitude and longitude of a receiver on earth. [64] TRB's Committee for Review of Travel Demand Modeling conducted by the Metropolitan Washington Council of Governments (Sept. 8, 2003/ first report). [65] The purpose of the United States Visitor and Immigrant Status Indicator Technology Program is to improve border management at ports of entry by capturing more complete arrive/departure data for those who require visas to enter the United States. [66] David A. Aschauer is the Elmer W. Campbell Professor at Bates College. His primary research field is macroeconomics. He is currently conducting research on the relationship between fiscal and monetary policy in the U.S. as well as the relationships between real interest rates and real exchange rates in an open economy setting. [67] TRB Special Report 264, The Congestion Mitigation and Air Quality Improvement Program: Assessing 10 Years of Experience, April 2002. Washington, D.C 68] TANF is a block grant program to help move recipients into work. The Office of Family Assistance in the U.S. Department of Health and Human Services administers this program. [69] Alan Altshuler and David Luberoff, Mega-Projects: The Changing Politics of Urban Public Investment, Lincoln Institute of Land Policy, 2003. [70] FTA's New Starts program helps pay for designing and constructing certain rail, bus, and trolley projects. [71] The National Cooperative Highway Research Program (NCHRP) is administered by TRB and sponsored by the member departments (individual state departments of transportation) of AASHTO in cooperation with FHWA. Support is voluntary and funds are drawn from the states' Federal- Aid Highway apportionment of State Planning and Research funds. The Transit Cooperative Research Program (TCRP) functions under the direction of the Federal Transit Administration, the National Academies (acting through the Transportation Research Board) and a nonprofit research/education organization established by the American Public Transportation Association. [72] Revealed preference studies use data from real consumer behavior to obtain information about demand, values, etc. Stated preference studies employ data from a particular kind of consumer survey. [73] Elicitation is the process of obtaining knowledge from experts or persons to be used in decision making or producing a design. [74] In the Bayesian approach, the probability of an event is a person's degree of belief that the event will occur, given all the relevant information currently known to that person. Thus, the probability is a function of both the event and the state of information. [75] Central tendency is the average outcome of any given phenomenon (travel time, etc.) as distinct from the range of other outcomes--both above and below the average--that occur. [76] Cross elasticity of demand is the percentage of change in quantity demanded in response to a 1 percent change in the price of another good. [77] A Monte Carlo simulation is a problem-solving and investigation technique used to approximate the probability of certain outcomes by running multiple trial runs (simulations) using random variables. [78] A smart card is a plastic card with a built-in microprocessor and memory that is used for identification or financial transactions. [79] Sandra Rosenbloom is Professor of Planning and Director of the Roy P. Drachman Institute for Land and Regional Development Studies at the University of Arizona. Her work has centered on transportation problems of poor people, working women with children, and the elderly. [80] Michael Meyer, "Paper Distributed to Expert Panel" (Washington, D.C., 2005. Photocopy). [81] PPB, which was grounded in systems analysis, was the process of defining objectives and designing alternative systems to achieve them. [82] The Tennessee-Tombigbee Waterway is a canal located in northeast Mississippi and west central Alabama. [83] Pickrell, Donald H., "A Desire Named Streetcar: Fantasy and Fact in Rail Transit Planning," Journal of the American Planning Association, 1992, pp. 158-76. [84] TEA-21 established firewalls, or new budget categories, that ensured that highway user fee revenues would be used for transportation programs. [85] Transaction costs are the full costs of making an exchange. [86] See GAO, Highway and Transit Investments: Options for Improving Information on Projects' Benefits and Costs and Increasing Accountability for Results, GAO-05-172 (Washington, D.C.: Jan. 24, 2005) for information about the request. [87] Evaluations can be conducted ex ante (before the intervention is initiated or outcomes have been produced) or ex post (measures the outcomes produced by the interventions to date). [88] S. 1072, 108th Cong.(2004). This bill would have authorized funds for federal-aid highways, highway safety programs, and transit programs and for other purposes. [89] As an example, Dr. Small said that several authors pulled together three different studies of the Coquihalla Highway in British Columbia- -one study was done before it opened, one was done part way through construction, and one was done when it was near completion. The numbers are phenomenally different, but there is not an obvious bias in this case. Many things changed along the way, including the scope of the project. [End of section] GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: