This is the accessible text file for GAO report number GAO-04-822 
entitled 'Information Technology: FAA Has Many Investment Management 
Capabilities in Place, but More Oversight of Operational Systems Is 
Needed' which was released on September 20, 2004.

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

GAO Highlights: 

Highlights of GAO-04-822, a report to congressional requesters

Report to Congressional Requesters: 

August 2004: 

INFORMATION TECHNOLOGY: 

FAA Has Many Investment Management Capabilities in Place, but More 
Oversight of Operational Systems Is Needed: 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-822]: 

Why GAO Did This Study: 

The Federal Aviation Administration’s (FAA) mission is to promote the 
safe, orderly, and expeditious flow of air traffic in the United States 
airspace system, commonly referred to as the National Airspace System 
(NAS). To maintain its ability to effectively carry out this mission 
FAA embarked, in 1981,on a multi-billion dollar effort to modernize 
its aging air traffic control (ATC) system, the principle technology 
component of the NAS. Yet the NAS modernization has continued to be 
plagued by cost increases, schedule delays, and performance shortfalls. 
To gain insight into how FAA is meeting its management challenges, 
congressional requesters asked GAO to evaluate FAA’s processes for 
making IT investment management decisions. The objectives of this 
review included (1) evaluating FAA’s capabilities for managing its IT 
investments and (2) determining what plans, if any, the agency might 
have for improving these capabilities.

What GAO Found: 

Judged against the criteria of GAO’s framework for information 
technology investment management (ITIM), which measures the maturity 
of an organization’s investment management processes, FAA has 
established about 80 percent of the basic selection and control 
practices that it needs to manage its mission-critical investments 
(see table below). For example, business lines actively monitor 
projects throughout their life cycles. However, the agency’s senior IT 
investment board does not regularly review investments that are in the 
“in-service management,” or operational, phase of their life cycles, 
and this creates a weakness in FAA’s ability to oversee more than $1 
billion of its IT investments. In addition, the agency has not yet 
established the key practices that would allow it to manage all of its 
investments as one portfolio—an integrated set of competing options. 
Until FAA has established the practices that would enable it to 
effectively manage its annual IT budget of about $2.5 billion, agency 
executives lack assurance that they are selecting and managing the mix 
of investments that best meets the agency’s needs and priorities.

The agency has initiated efforts to improve its investment management 
processes, but it has not yet developed and implemented a 
comprehensive plan—supported by management—to guide all of its 
improvement efforts. Such a plan is crucial in helping FAA to 
coordinate and prioritize its improvement efforts and sustain its 
commitment to the efforts it already has under way. Without such a 
plan—and controls for implementing it—FAA will be unlikely to develop 
a mature investment management capability. 

Summary of Results for Foundational Critical Processes and Key 
Practices: 

[See PDF for table]

Source: GAO.

[End of table]

What GAO Recommends: 

To strengthen FAA’s investment management capability, GAO recommends 
that FAA develop and implement a plan to address the weaknesses 
identified in this report. In commenting on a draft of this report, 
the Department of Transportation commented that the report was 
balanced and fair, showing where FAA has many capabilities in place 
and identifying areas that need improvement.

www.gao.gov/cgi-bin/getrpt?GAO-04-822.

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact David Powner, 202-512-
9286, pownerd@gao.gov or Lester Diamond, 202-512-7957, diamondl@gao.gov.

[End of section]

Contents: 

Letter: 

Results in Brief: 

Background: 

FAA Has Established an IT Management Structure to Manage Its NAS 
Investments: 

FAA Has Initiated Efforts to Improve Its Investment Management Process: 

DOT is Taking Steps to Integrate Oversight of FAA's IT Investments: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments: 

Appendixes: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Investment Management Process Used by Some Organizational 
Units to Manage Non-NAS Investments: 

Tables: 

Table 1: NAS and Non-NAS IT Investments: 

Table 2: Stage 2 Critical Processes--Building the Investment 
Foundation: 

Table 3: Summary of Results for Stage 2 Critical Processes and Key 
Practices for NAS Investments: 

Table 4: Instituting the Investment Board: 

Table 5: Meeting Business Needs: 

Table 6: Selecting an Investment: 

Table 7: Providing Investment Oversight: 

Table 8: Capturing Investment Information: 

Table 9: Stage 3 Critical Processes--Developing a Complete Investment 
Portfolio: 

Table 10: Status of Stage 3 Critical Processes: 

Figures: 

Figure 1: FAA's Life Cycle Management Process: 

Figure 2: Detailed Breakdown of FAA's Life Cycle Management Process: 

Figure 3: The Five ITIM Stages of Maturity with Critical Processes: 

Abbreviations: 

ABA: Financial Services unit: 

AHR: Human Resource Management unit: 

AIO: Information Services unit: 

AMS: Acquisition Management System: 

APB: acquisition program baseline: 

ARA: Research and Acquisition unit: 

ARC: Region and Center Operations unit: 

ATC: air traffic control: 

ATO: Air Traffic Organization: 

ATS: Air Traffic Services unit: 

AVR: Regulation and Certification unit: 

CIO: Chief Information Officer: 

DOT: Department of Transportation: 

ECG: En Route Communications Gateway: 

F&E: facilities and equipment (predeployment stage of system life 
cycle): 

FAA: Federal Aviation Administration: 

FTI: FAA Telecommunications Infrastructure: 

IAT: Investment Analysis Team: 

IT: information technology: 

ITEB: Information Technology Executive Board: 

ITIM: Information Technology Investment Management framework: 

ITIPS: Information Technology Investment Portfolio System: 

JRC: Joint Resources Council: 

NAS: National Airspace System: 

NSIP: NAS Support Integration Process: 

OMB: Office of Management and Budget: 

OPS: operations (postdeployment stage of system life cycle): 

PIR: postimplementation review: 

VCSU: VSCS Control Subsystem Upgrade: 

VSCS: Voice Switching and Control System: 

Letter August 20, 2004: 

The Honorable Tom Davis: 
Chairman: 
Committee on Government Reform: 
U.S. House of Representatives: 

The Honorable Adam H. Putnam: 
Chairman: 
Subcommittee on Technology, Information Policy, Intergovernmental 
Relations and the Census: 
Committee on Government Reform: 
U.S. House of Representatives: 

The Federal Aviation Administration's (FAA) mission is to promote the 
safe, orderly, and expeditious flow of air traffic in the United States 
airspace system, commonly referred to as the National Airspace System 
(NAS). To maintain its ability to effectively carry out this mission 
FAA embarked, in 1981, on a multibillion dollar effort to modernize its 
aging air traffic control (ATC) system, the principle technology 
component of the NAS. Over the past 2 decades, individual FAA 
modernization projects have experienced cost overruns, schedule delays, 
and performance shortfalls of large proportions. Because of the size, 
complexity, cost, and problem-plagued past of FAA's modernization 
program, we have designated it a high-risk information technology 
investment since 1995.[Footnote 1]

This report is one in a series of reports responding to your request to 
evaluate FAA's efforts to address the information technology (IT) 
management challenges it faces as it continues to modernize the ATC 
system. It focuses on FAA's processes for making IT investment 
management decisions and uses our Information Technology Investment 
Management (ITIM) framework,[Footnote 2] which was released at a 
hearing of the subcommittee on March 3, 2004. The framework provides a 
method for evaluating and assessing how well an agency is selecting and 
managing its IT resources. As agreed, our objectives were to 
(1) evaluate FAA's capabilities for managing its IT investments, 
(2) determine what plans the agency might have for improving these 
capabilities, and (3) describe the Department of Transportation's (DOT) 
oversight of FAA's investments and investment management process. To 
address these objectives we analyzed documents and interviewed agency 
officials to (1) validate and update FAA's self-assessments of the key 
practices in the framework, (2) evaluate FAA's plans for improving its 
capabilities, and (3) describe the department's oversight role. We 
performed our work from October 2003, through July 2004, in accordance 
with generally accepted government auditing standards. Appendix I 
contains further details on our objectives, scope, and methodology.

Results in Brief: 

FAA has established most--about 80 percent--of the basic practices 
needed to manage its mission critical investments, including many of 
the foundational practices for selecting and controlling IT 
investments. These key practices provide additional assurance that the 
investments selected will meet organizational needs and will be 
completed on time and within budget. The practices also will enable the 
agency to manage its IT investments as a portfolio, or integrated set 
of competing options.

Even with these many capabilities in place, weaknesses remain in 
several areas. Specifically, FAA: 

* does not involve its senior IT investment board in regular reviews of 
investments that have entered the in-service management phase, that is, 
those systems that have completed development and become operational;

* does not have standard practices for managing its mission-support and 
administrative investments;

* has not developed a process where the senior IT investment board 
regularly reviews the full portfolio of investments; and: 

* has not implemented postimplementation reviews of its major 
investments to validate that they are providing the expected benefits 
after they become operational.

FAA has begun to act to resolve the weaknesses described above, but 
until FAA establishes the practices it needs to effectively manage its 
IT investments, executives cannot be assured that they are selecting 
and managing the mix of investments that best meets the agency's needs 
and priorities. Establishing the capabilities needed to effectively 
manage investments requires the development and implementation of a 
plan, supported by management, that defines and prioritizes 
improvements to the investment process. While FAA has initiated a 
series of efforts to improve its investment management processes, it 
does not have such a plan. Without this plan--and controls for 
implementing it--it is unlikely that the agency will effectively 
establish mature investment management capabilities.

The Department of Transportation has recently initiated several efforts 
that can serve to provide better departmental oversight of FAA 
investments. For example, DOT is finalizing capital investment guidance 
for all of its operating administrations to follow in implementing 
their investment management processes, and it has initiated a process 
for reviewing the fiscal year 2006 budget justifications for major 
programs, including those of FAA. The department has also identified 
about a dozen programs it plans to monitor on a regular basis and has 
asked FAA to report cost, schedule, and performance data on some of its 
programs quarterly.

To further strengthen FAA's investment management capability, we are 
recommending that the agency develop and implement a plan aimed at 
addressing the weaknesses identified in this report.

In commenting on a draft of this report, the Department of 
Transportation's Director of Audit Relations stated that GAO did a good 
job of keeping the report balanced and fair, showing where FAA has many 
capabilities in place and identifying areas that need improvement. The 
agency also provided a technical comment, which we have incorporated 
into the report.

Background: 

FAA's Mission and Organizational Structure: 

As an agency of the Department of Transportation, FAA's mission is to 
promote the safe, orderly, and expeditious flow of air traffic in the 
national airspace. To fulfill its mission requires the extensive use of 
technology. The achievement of the agency's mission is also dependent 
in large part on the skills and expertise of its workforce. Its 
workforce of nearly 50,000 people provides aviation services that 
include air traffic control; maintenance of air traffic control 
equipment; and certification of aircraft, airline operations, and 
pilots.

FAA is organized into several staff support offices (examples include 
the Office of Information Services and the Office of Human Resource 
Management) and five lines of business, which include Airports, 
Regulation and Certification, Commercial Space Transportation, the 
Office of Security and Hazardous Materials, and the newly formed Air 
Traffic Organization (ATO). The ATO was formed on February 8, 2004, to 
better provide safe, secure, and cost-effective air traffic services 
now and into the future. The Air Traffic Services and the Research and 
Acquisitions units, which had been primarily responsible for managing 
air traffic services within FAA, were combined into one performance-
based organization to create ATO. ATO is led by FAA's Chief Operating 
Officer and consists of 10 service units.[Footnote 3]

FAA's Use of IT: 

FAA relies extensively on information technology to carry out its NAS 
operations. It constantly depends on the adequacy and reliability of 
the nation's ATC system, which comprises a vast network of radars; 
automated data processing, navigation, and communications equipment; 
and ATC facilities.[Footnote 4] Through this system, FAA provides 
services such as controlling takeoffs and landings and managing the 
flow of traffic between airports. For example, the Integrated Terminal 
Weather System is employed to allow maximum use of airport runways in 
all kinds of weather through a variety of weather sensors. The Wide 
Area Augmentation System is used to provide vertically guided landing 
to aircraft at thousands of airports and airstrips where there is 
currently no vertically guided landing capability.

FAA also relies on IT to carry out its mission-support and 
administrative operations (non-NAS operations). For example, FAA uses 
IT to support accident and incident investigations, security 
inspections, and personnel and payroll functions.

With an IT budget of about $2.5 billion for fiscal year 2004, FAA 
accounts for over 90 percent of the Department of Transportation's IT 
budget. The amount of investments in both NAS and non-NAS IT is shown 
in the table 1 below.

Table 1: NAS and Non-NAS IT Investments: 

Type of investment: NAS; 
Funding of IT investments: Facilities and Equipment (F&E) (development 
through 2 years of operations); 
Total IT investment in fiscal year 2004 in billions of dollars: $1.464.

Type of investment: NAS; 
Funding of IT investments: Operations (OPS) (through the rest of the 
life cycle); 
Total IT investment in fiscal year 2004 in billions of dollars: $0.834.

Type of investment: Non-NAS; 
Funding of IT investments: Operations; 
Total IT investment in fiscal year 2004 in billions of dollars: 
$0.350-0.500[A].

Source: FAA.

[A] According to FAA, these numbers will be verified via baselining by 
the end of Fiscal Year 2004 to reflect non-NAS IT assets and their 
costs.

[End of table]

Prior Reviews Identified Weaknesses in the Agency's IT Investment 
Management Process: 

In 1995, we designated FAA's modernization of its air traffic control 
system, the principle technology component of the NAS, as a high-risk 
area because of the size and complexity of the program and FAA's many 
failures in meeting projects' cost, schedule, and performance goals. In 
our latest High-Risk Series, issued in January 2003,[Footnote 5] we 
addressed the critical need for FAA to continue to improve its 
investment management practices--the management processes the agency 
uses to select, control and evaluate the benefits realized from its IT 
spending--because the agency would be spending nearly $16 billion more 
through FY 2007, after having already spent $35 billion since 1981. 
Other reports have also noted weaknesses in FAA's IT investment 
management processes and have made a number of recommendations to 
address this area.[Footnote 6] For instance, last year we reported that 
while FAA had improved its processes, several issues remained 
unresolved. We noted, for example, that the agency had not yet 
implemented processes for evaluating projects after implementing them, 
in order to identify lessons learned and improve the investment 
management process.

FAA's Current Approach to Investment Management: 

FAA's process for managing an IT investment varies depending on the 
type of investment--NAS systems in development through the second year 
of operation (F&E), NAS systems in operation after the second year 
(OPS), and non-NAS systems each follow different processes. NAS 
investments are managed through a standardized process, the FAA 
Acquisition Management System (AMS), and non-NAS investments are 
managed through a number of different processes.

Process for Managing NAS Investments: 

In April 1996, FAA implemented its AMS in response to legislation that 
directed the agency to develop a new acquisition management 
system.[Footnote 7] Because of FAA's contention that some of its 
modernization problems were caused by federal acquisition regulations, 
the Congress enacted legislation in November 1995 that exempted the 
agency from most federal procurement laws and regulations and directed 
FAA to develop and implement a new acquisition management system that 
would address the unique needs of the agency. AMS was intended to 
reduce the time and cost for fielding new products and services by 
introducing (1) a new investment management system that spans the 
entire life cycle of an acquisition, (2) a new procurement system that 
provides flexibility in selecting and managing contractors, and 
(3) organizational and human capital reforms that support the new 
investment and procurement systems.

AMS provides high-level acquisition policy and guidance for selecting 
and controlling FAA's NAS investments through all phases of the 
acquisition life cycle, which is organized into a series of phases and 
decision points that include (1) mission analysis, (2) investment 
analysis, (3) solution implementation, and (4) in-service management. 
To select investments, FAA has established two processes--mission 
analysis and investment analysis--which together constitute a set of 
policies, procedures, and guidance that enhance the agency's ability to 
screen projects that are submitted for funding. Also, through these two 
processes FAA is to assess and rank each project based on its relative 
costs, benefits, risks, and contribution to FAA's mission, and a 
senior, corporate-level decision-making group selects projects for 
funding. After a project has been selected, FAA officials are required 
to formally establish the life cycle cost, schedule, benefits, and 
performance baselines that are used to monitor the project's status 
throughout the remaining phases of the acquisition management life 
cycle. See figure 1 for a graphic depiction of FAA's life cycle 
management process.

Figure 1: FAA's Life Cycle Management Process: 

[See PDF for image] 

Note: During the front end of the life cycle, research and system 
analysis activities are undertaken to discover applications of new 
technology for FAA's present services, explore new opportunities for 
service delivery, solve problems within current operations, and define 
requirements.

[End of figure] 

Several groups are involved in managing FAA's NAS investments; they 
perform functions from analysis of mission needs and alternative 
investments through system development, implementation, operation, 
and, ultimately, disposal. The roles and responsibilities of each group 
are described below: 

Joint Resources Council (JRC)--This board makes corporate-level 
resource and investment decisions and establishes investment programs. 
Members include Associate Administrators representing FAA's lines of 
business, the FAA Acquisition Executive,[Footnote 8] the Chief 
Financial Officer, the Chief Information Officer (CIO), and the 
Assistant Administrators for System Safety, for Policy, Planning and 
International Aviation, and for Region and Center Operations. The board 
is supported by the JRC Secretariat Team, a group that facilitates the 
board's processes by maintaining the meeting calendar and guidance 
documents, developing records of decisions, and providing advisory and 
liaison support to programs.

Systems Engineering/Operational Analysis Team--This team performs 
affordability assessments for newly proposed investments and prepares 
recommendations for the reprogramming of funds from lower priority 
programs. It also prepares annual budget submissions for approval by 
the JRC. This team is composed of representatives from each line of 
business and from other functional disciplines and is chaired by the 
Director, System Architecture and Investment Analysis.

Investment Analysis Team (IAT)--This team is assembled for a relatively 
short period for each specific investment being considered, to conduct 
the detailed analysis of alternatives that will lead to selecting and 
recommending a preferred acquisition solution. The team draws experts 
from the integrated product teams,[Footnote 9] the organizational unit 
with the need, the investment analysis staff,[Footnote 10] and other 
organizations.

Corporate Mission Analysis Organization--Performs agency-level mission 
analysis and coordinates service area analysis, an activity that is 
conducted during mission analysis to (1) identify capability shortfalls 
for or in conjunction with service organizations,[Footnote 11] (2) 
ensure alignment with agency strategic goals, and (3) eliminate 
redundant activity, duplicate benefits, service gaps, and service 
overlaps. It also develops and maintains standards and tools for 
conducting service area analysis, and it assists service organizations 
in establishing a service area analysis capability.

In addition to identifying the roles and responsibilities of the groups 
involved in the management process, AMS provides guidance on the 
documents and decisions that result from each of the life cycle phases. 
For example, through the mission analysis phase, FAA identifies 
critical needs that the agency must meet for improving the safety, 
capacity, efficiency, and effectiveness of the NAS. Approval of a 
mission need statement by the JRC signifies that the agency agrees that 
the need is critical enough to proceed to the next phase--investment 
analysis. During the investment analysis phase, the IAT is to analyze 
and recommend a solution that best satisfies FAA's performance goals 
and customer service needs. This team is then to rank each proposed 
project based on a number of factors, including how well it meets 
mission needs compared to other projects and whether it has a favorable 
cost-benefit ratio. As part of the JRC selection process, the life 
cycle cost, schedule, benefits, and performance baselines are 
established in a formal document called the acquisition program 
baseline (APB), which is designed to be used by program offices to 
monitor a project's status in achieving those baselines throughout the 
remaining phases of the acquisition management life cycle.

The solution implementation phase begins when the JRC approves and 
funds a project, establishes its acquisition program baseline, and 
authorizes the service organizations to implement and manage the 
project over its life cycle. After the project has been implemented and 
is in operation (FAA's in-service management phase), the service 
organizations monitor and assess operational performance. Also during 
this phase, the project is monitored to determine whether the current 
capability satisfies the demand for services or whether another 
solution offers the potential for improving safety or effectiveness or 
for significantly lowering costs. If the current capability is lacking, 
FAA initiates a process whereby the mission need would be revalidated 
and the investment analysis process begun again, possibly leading to a 
new investment decision. Figure 2 provides detail on the phases of 
FAA's IT investment management process and decision points. The 
highlighted decision points represent those for which the JRC must make 
an approval decision before a project can move forward.

Senior executives have stated that with the reorganization of the ATO 
in February 2004, discussions have been held about realigning the 
investment management process to make the heads of the service units 
responsible and accountable for managing programs' capital investments 
and operating costs from inception to retirement. In the past, the 
business units have been organized to manage either capital investments 
or operating costs, but not both. These discussions have not yet led to 
specific changes in FAA's investment management processes and 
responsibilities.

Figure 2: Detailed Breakdown of FAA's Life Cycle Management Process: 

[See PDF for image] 

[A] Corporate requirements organization takes the lead in planning for 
the concept and requirements definition phase of the life cycle 
management process.

[B] In a February 2004 memorandum, FAA's Chief Operating Officer 
assumed the in-service decision authority and stated that he would 
delegate this responsibility to the vice presidents of the service 
organizations, unless the JRC retained the in-service decision 
authority. If the JRC retains this authority, it determines the in-
service decision authority at the time of the final investment 
decision.

[End of figure] 

Process for Managing Non-NAS Investments: 

While the AMS was intended to apply to all FAA investment programs, it 
has not been implemented for non-NAS investments. Each of the agency's 
business line and staff offices that manage non-NAS 
investments[Footnote 12] has implemented its own processes for managing 
these investments. Examples of these various non-NAS investment 
processes include the following: 

* Regarding an investment management board structure, the Financial 
Services staff office has an informal board consisting of the Chief 
Financial Officer, Deputy Chief Financial Officer, and heads of offices 
within Financial Services. The Financial Services life cycle process 
guide directs the board's operations. In the Regulation and 
Certification unit, the senior management team makes investment 
management decisions with input from the Chief Information management 
team. This unit is developing an IT investment management processes 
guide, which is expected to be completed by the end of the fiscal year.

* When selecting investments, the Human Resource Management unit uses 
its established annual budget formulation process, while the Region and 
Center Operations unit is moving toward a new process whereby in order 
to be selected investments need to demonstrate, at a minimum, that they 
(1) are compliant with FAA's architecture, (2) have a business sponsor, 
(3) have a solid business case, and (4) can be funded.

* In controlling investments, Information Services has developed 
processes to monitor contract expenditures, and unit managers regularly 
perform financial management reviews of the programs under their 
purview, but there is no structured process for oversight of projects' 
performance against expectations. In the Human Resource Management 
unit, division managers hold quarterly reviews to assess projects' 
progress in meeting cost and schedule expectations and aligning with 
strategic goals.

Descriptions of the processes used by each of the units responsible for 
managing non-NAS investments can be found in appendix II.

In January 2004, the FAA Administrator established the Information 
Technology Executive Board (ITEB) to "strengthen FAA's ability to use 
IT as an agencywide strategic asset" and "guide fundamental changes in 
the governance of IT assets." Its charter calls for the ITEB to assume 
responsibility for making investment decisions about non-NAS IT 
investments. However, the ITEB has not yet implemented this aspect of 
its charter. Therefore, at the current time there is no single board or 
investment management process for non-NAS investments that would be 
analogous to the JRC board and AMS process that are used for NAS 
investments.

ITIM Maturity Framework: 

The ITIM framework is a maturity model composed of five progressive 
stages of maturity that an agency can achieve in its investment 
management capabilities.[Footnote 13] It was developed on the basis of 
our research into the IT investment management practices of leading 
private-and public-sector organizations. The framework identifies 
critical processes for making successful IT investments, organized into 
the five increasingly mature stages. These maturity stages are 
cumulative; that is, in order to attain a higher stage of maturity, the 
agency must have institutionalized all of the requirements for all of 
the lower stages, in addition to those for the higher stage.

The ITIM can be used both to assess the maturity of an agency's 
investment management processes and as a tool for organizational 
improvement. The overriding purpose of the framework is to encourage 
investment processes that increase business value and mission 
performance, reduce risk, and increase accountability and transparency 
in the decision process. We have used the framework in several of our 
evaluations,[Footnote 14] and a number of agencies have adopted it. 
These agencies have used ITIM for purposes ranging from self-assessment 
to redesign of their IT investment management processes.

ITIM's five maturity stages represent steps toward achieving stable and 
mature processes for managing IT investments. Each stage builds on the 
lower stages; the successful attainment of each stage leads to 
improvement in the organization's ability to manage its investments. 
With the exception of the first stage, each maturity stage is composed 
of "critical processes" that must be implemented and institutionalized 
in order for the organization to achieve that stage. These critical 
processes are further broken down into key practices that describe the 
types of activities that an organization should be performing to 
successfully implement each critical process. An organization may be 
performing key practices from more than one maturity stage at the same 
time. This is not unusual, but efforts to improve investment management 
capabilities should focus on becoming compliant with lower-stage 
practices before addressing higher-stage practices.

Stage 2 of the ITIM framework encompasses building a sound investment 
management process by establishing basic capabilities for selecting new 
IT projects. It also involves developing the capability to control 
projects so that they finish predictably within established cost and 
schedule expectations and the capability to identify potential 
exposures to risk and put in place strategies to mitigate that risk. 
The basic selection processes established in Stage 2 lays the 
foundation for more mature selection capabilities in Stage 3.

Stage 3 requires that an organization continually assess both proposed 
and ongoing projects as parts of a complete investment portfolio--an 
integrated and competing set of investment options. It focuses on 
establishing a consistent, well-defined perspective on the IT 
investment portfolio and maintaining mature, integrated selection (and 
reselection), control, and evaluation processes, which are to be 
evaluated during postimplementation reviews (PIR). This portfolio 
perspective allows decision makers to consider the interaction among 
investments and the contributions to organizational mission goals and 
strategies that could be made by alternative portfolio selections, 
rather than relying exclusively on the balance between the costs and 
benefits of individual investments.

Stages 4 and 5 require the use of evaluation techniques to continuously 
improve both the investment portfolio and investment processes in order 
to better achieve strategic outcomes. At Stage 4 maturity an 
organization has the capacity to conduct IT succession activities and 
therefore can plan and implement the deselection of obsolete, high-
risk, or low-value IT investments. An organization with Stage 5 
maturity conducts proactive monitoring for breakthrough information 
technologies that will enable it to change and improve its business 
performance. Organizations implementing Stages 2 and 3 have in place 
the selection, control, and evaluation processes that are required by 
the Clinger-Cohen Act. Stages 4 and 5 define key attributes that are 
associated with the most capable organizations.

Figure 3 shows the five maturity stages and the critical processes 
associated with each.

Figure 3: The Five ITIM Stages of Maturity with Critical Processes: 

[See PDF for image] 

[End of figure] 

As defined by the model, each critical process consists of "key 
practices" that must be executed to implement the critical process.

FAA Has Established Many Key Practices for Managing NAS Investments but 
Lacks Oversight of Operational Systems: 

In order to have the capabilities to effectively manage IT investments, 
an agency should, at a minimum, (1) build an investment foundation by 
putting basic, project-level control and selection practices in place 
(Stage 2 capabilities) and (2) manage its projects as a portfolio of 
investments, treating them as an integrated package of competing 
investment options and pursuing those that best meet the strategic 
goals, objectives, and mission of the agency; and it should also 
conduct PIRs to maintain mature, integrated selection, control, and 
evaluation processes (Stage 3 capabilities). In addition, an agency 
would be well served by implementing capabilities for improving its 
investment process through performance evaluations of its portfolio and 
succession management of current investments (Stage 4 capabilities). In 
order to develop the capabilities to effectively manage its 
investments, FAA would, at minimum, need to implement Stage 2 
capabilities for both its NAS and non-NAS investments and Stage 3 
capabilities for its portfolio of investments.

FAA's investment management capabilities vary depending on whether an 
investment is considered to be NAS or non-NAS. Specifically: 

* For NAS investments, FAA has executed 30 of the 38 Stage 2 key 
practices that are required to establish a foundation for investment 
management maturity. For these investments, the agency has in place a 
strong set of processes to support investment management, although the 
JRC does not regularly review investments that have passed into the in-
service management phase (i.e., operational systems).

* For its non-NAS investments, the agency has not yet adequately 
implemented a single management line of responsibility and the standard 
processes needed to manage in a consistent manner. Although some 
structured processes exist within individual business units, this lack 
of consistency undermines the agency's maturity.

* In Stage 3, the lack of regular JRC oversight of operational systems 
and the absence of a structured approach to managing non-NAS 
investments prevent FAA from managing its investments as a portfolio 
that includes all major NAS and non-NAS investments. In addition, the 
agency is not conducting PIRs on its major investments.

* FAA has not executed any of the Stage 4 key practices for managing 
the succession of its information systems, although the agency has 
begun to address this weakness by defining procedures for retiring 
investments in the AMS.

When FAA implements all of the key practices associated with building 
the investment foundation and managing its investments as a portfolio, 
the agency will have greater assurance that it has selected the mix of 
investments that best supports its strategic goals and that it will be 
able to manage the investments to successful completion.

FAA Has Established Much of the Foundation Needed to Manage Its NAS 
Investments: 

At the ITIM Stage 2 level of maturity, an organization has attained 
repeatable, successful IT project-level investment control processes 
and basic selection processes. Through these processes, the 
organization can identify expectation gaps early and take appropriate 
steps to address them. According to ITIM, critical processes at Stage 2 
include (1) defining IT investment board[Footnote 15] operations, 
(2) identifying the business needs for each IT investment, 
(3) developing a basic process for selecting new IT proposals and 
reselecting ongoing investments, (4) developing project-level 
investment control processes, and (5) collecting information about 
existing investments. Table 2 describes the purpose of each of the 
Stage 2 critical processes.

Table 2: Stage 2 Critical Processes--Building the Investment 
Foundation: 

Critical process: Instituting the investment board; 
Purpose: To define and establish an appropriate IT investment 
management structure and the processes for selecting, controlling, and 
evaluating IT investments.

Critical process: Meeting business needs; 
Purpose: To ensure that IT projects and systems support the 
organization's business needs and meets users' needs.

Critical process: Selecting an investment; 
Purpose: To ensure that a well-defined and disciplined process is used 
to select new IT proposals and reselect ongoing investments.

Critical process: Providing investment oversight; 
Purpose: To review the progress of IT projects and systems, using pre-
defined criteria and checkpoints, in meeting cost, schedule, risk, and 
benefit expectations and to take corrective action when these 
expectations are not being met.

Critical process: Capturing investment information; 
Purpose: To make available to decision makers information to evaluate 
the impacts and opportunities created by proposed (or continuing) IT 
investments.

Source: GAO.

[End of table]

To its credit, FAA has put in place about 80 percent of the key 
practices associated with managing its NAS investments through the 
Stage 2 critical processes. The agency has satisfied all of the key 
practices associated with capturing investment information and most of 
those associated with instituting the investment board, meeting 
business needs, selecting an investment, and providing investment 
oversight. Most of the weaknesses in these critical processes relate 
to NAS investments in the in-service management phase. Table 3 
summarizes the status of FAA's critical processes for Stage 2, showing 
how many key practices FAA has executed in managing its NAS 
investments.

Table 3: Summary of Results for Stage 2 Critical Processes and Key 
Practices for NAS Investments: 

Critical process: Instituting the investment board; 
Key practices executed: 7; 
Total required by critical process: 8; 
Percentage of key practices executed: 88%.

Critical process: Meeting business needs; 
Key practices executed: 6; 
Total required by critical process: 7; 
Percentage of key practices executed: 86%.

Critical process: Selecting an investment; 
Key practices executed: 7; 
Total required by critical process: 10; 
Percentage of key practices executed: 70%.

Critical process: Providing investment oversight; 
Key practices executed: 4; 
Total required by critical process: 7; 
Percentage of key practices executed: 57%.

Critical process: Capturing investment information; 
Key practices executed: 6; 
Total required by critical process: 6; 
Percentage of key practices executed: 100%.

Critical process: Total; 
Key practices executed: 30; 
Total required by critical process: 38; 
Percentage of key practices executed: 79%. 

Source: GAO.

[End of table]

FAA Has Established an IT Management Structure to Manage Its NAS 
Investments: 

The establishment of decision-making bodies or boards is a key 
component of the IT investment management process. At the Stage 2 level 
of maturity, organizations define one or more boards, provide resources 
to support their operations, and appoint members who have expertise in 
both operational and technical aspects of proposed investments. The 
boards operate according to a written IT investment process guide that 
is tailored to the organization's unique characteristics, thus ensuring 
that consistent and effective management practices are implemented 
across the organization. Once board members are selected, the 
organization ensures that they are knowledgeable about policies and 
procedures for managing investments. Organizations at the Stage 2 level 
of maturity also take steps to ensure that executives and line managers 
support and carry out the decisions of the IT investment board. 
According to ITIM, an IT investment management process guide should be 
a key authoritative document that the organization uses to initiate and 
manage IT investment processes and should provide a comprehensive 
foundation for the policies and procedures that are developed for all 
of the other related processes. (The complete list of key practices is 
provided in table 4.)

FAA has executed 7 of the 8 key practices for this critical process. 
For example, in 1996, Congress directed FAA to develop a new 
acquisition management system as part of a broad mandate for 
acquisition reform at the agency.[Footnote 16] In response, FAA 
implemented AMS in April 1996. AMS establishes policy and guidance for 
all aspects of the agency's acquisition life cycle and documents the 
investment management process used for NAS investments. The agency 
established the JRC as its corporate-level investment board for the NAS 
investments. The JRC makes select and control decisions, including 
corporate decisions on mission needs, acquisition investments, and 
acquisition program baseline changes; it also reviews and recommends 
approval of the agency's F&E budget submission.

The board is adequately resourced to support its operations. The JRC 
Secretariat Team supports the board in such ways as developing and 
updating guidance, scheduling meetings, and preparing and executing the 
JRC readiness process. In addition, the Mission Analysis Steering 
Group[Footnote 17] is responsible for assisting the board in 
prioritizing mission needs, while the Systems Engineering/Operational 
Analysis Team is to assist in addressing budget issues among 
investments. The JRC consists of senior officials from both business 
and IT areas, including the Chief Information Officer and the associate 
administrators representing FAA lines of business. These members are to 
exhibit the core competencies required by FAA in selecting executives 
and in assessing executive training needs. In addition, the agency 
offers a 3-day AMS overview course for all employees, including JRC 
members. Although the board as an entity does not oversee the 
development and maintenance of AMS, it is involved through FAA's 
Acquisition System Advisory Group, which evaluates all proposed changes 
to AMS. To ensure that the board's decisions are carried out, an 
acquisition program baseline document is approved at the JRC final 
investment decision point; this document identifies the capabilities, 
benefits, costs, and schedule for the approved investment, which are 
monitored by FAA through its variance reporting process.

Despite these strengths, FAA has not yet clearly defined the 
relationship between the JRC and the newly formed ITEB. Although the 
ITEB was established by the Administrator to function as the central 
authority responsible for assuring that FAA IT investments are based on 
sound business practices, FAA has not yet clearly delineated the 
specific roles the ITEB is to play and the relationship it will have 
with the JRC. This task has been assigned to the ITEB as a longer-range 
initiative.

Table 4 shows the rating for each key practice required to implement 
the critical process for instituting the investment board at the Stage 
2 level of maturity. Each of the "Executed" ratings shown below 
represents instances where, based on the evidence provided by FAA 
officials, we concluded that the specific key practices were executed 
by the organization.

Table 4: Instituting the Investment Board: 

Type of practice: Organizational commitments; 
Key Practice: 1. An enterprisewide IT investment board composed of 
senior executives from IT and business units is responsible for 
defining and implementing the organization's IT investment governance 
process; 
Rating: Executed; 
Summary of evidence: The JRC, FAA's corporate-level investment board 
for the NAS investments, is responsible for defining and implementing 
the agency's IT investment governance process. It consists of the 
agency's most senior executives, including the CIO, Chief Financial 
Officer, FAA's Acquisition Executive, and Associate Administrators 
from its lines of business.

Type of practice: Organizational commitments; 
Key Practice: 2. The organization has a documented IT investment 
process directing each investment board's operations; 
Rating: Executed; 
Summary of evidence: FAA's AMS sets forth acquisition policy and 
processes for the JRC. Also, the board has established its own 
guidance to implement AMS core JRC policy.

Type of practice: Prerequisites; 
Key Practice: 1. Adequate resources, including people, funding, and 
tools, are provided for supporting the operations of each IT investment 
board; 
Rating: Executed; 
Summary of evidence: Adequate resources are provided to support the 
board's operations. The JRC Secretariat Team provides such operations 
support as developing and updating guidance, scheduling meetings, and 
preparing and distributing records of decisions. Two other groups 
support the JRC decision-making process. The Mission Analysis Steering 
Group assists the board in ranking mission needs, while the Systems 
Engineering/Operational Analysis Team provides assistance by performing 
affordability assessments for the JRC when it is considering 
alternatives during investment analysis.

Type of practice: Prerequisites; 
Key Practice: 2. The board members understand the organization's IT 
investment management policies and procedures and the tools and 
techniques used in the board's decision-making process; 
Rating: Executed; 
Summary of evidence: JRC members are senior managers representing all 
agency lines of business. They include the CIO, Chief Financial 
Officer, and Associate Administrators representing FAA lines of 
business such as Air Traffic Services. Core executive competencies 
based on FAA's Executive Success Profile are used in selecting 
executives and in assessing executive training needs. A 3-day AMS 
overview course is also available.

Type of practice: Prerequisites; 
Key Practice: 3. Each board's span of authority and responsibility is 
defined to minimize overlaps or gaps among the boards; 
Rating: Not executed; 
Summary of evidence: The JRC is FAA's corporate-level investment board 
for making decisions related to NAS investments. In January 2004, the 
ITEB was established to oversee the governance of the agency's IT 
assets. However, the ITEB has yet to take significant action on the 
charge in its charter to clearly delineate the roles it is to play and 
its relationship with the JRC.

Type of practice: Activities; 
Key Practice: 1. The enterprisewide investment board has oversight 
responsibilities for the development and maintenance of the 
organization's documented IT investment process; 
Rating: Executed; 
Summary of evidence: Although the JRC does not directly oversee the 
development and maintenance of the FAA's documented investment process, 
it is involved in this process through FAA's Acquisition System 
Advisory Group, which is a corporate crossfunctional body that 
evaluates all proposed changes to AMS. Membership consists of 
representatives from each line of business as well as the JRC 
Secretariat Team. Policy changes that are endorsed by the Group are 
presented, via the FAA Acquisition Executive, who is on the JRC, to 
the Administrator for approval.

Type of practice: Activities; 
Key Practice: 2. Each investment board operates in accordance with its 
assigned authority and responsibility; 
Rating: Executed; 
Summary of evidence: The JRC is operating in accordance with its 
assigned authority and responsibility as FAA's corporate-level 
investment board for making decisions related to NAS investments. The 
charter for the ITEB specifically indicates the ITEB's 
responsibilities, including for making decisions for non-NAS IT 
acquisitions.

Type of practice: Activities; 
Key Practice: 3. The organization has established management controls 
for ensuring that investment boards' decisions are carried out; 
Rating: Executed; 
Summary of evidence: FAA has controls for ensuring that the JRC's 
investment decisions are carried out as approved. At the JRC final 
investment decision point, an acquisition program baseline document is 
finalized and approved, which represents the mutual agreement between 
the JRC, the provider organization, and the user organization 
concerning the expected capability, benefits, costs, and schedule for 
the investment program. It also establishes performance metrics for 
assessing the program's success. 

Source: GAO.

[End of table]

FAA Has a Process for Ensuring That its Investments Support Business 
Needs and Meet Users' Needs: 

Defining business needs for each IT project helps to ensure that 
projects and systems support the organization's business needs and meet 
users' needs. This critical process ensures that a link exists between 
the organization's business objectives and its IT management strategy. 
According to ITIM, effectively meeting business needs requires, among 
other things, (1) documenting business needs with stated goals and 
objectives, (2) identifying specific users and other beneficiaries of 
IT projects and systems, (3) providing adequate resources to ensure 
that projects and systems support the organization's business needs and 
meet users' needs, and (4) periodically evaluating the alignment of IT 
projects and systems with the organization's strategic goals and 
objectives. (The complete list of key practices is provided in table 
5.)

FAA has in place 6 of the 7 key practices for meeting business needs. 
The agency's AMS and mission analysis guidance calls for business needs 
for both proposed and ongoing IT projects and systems to be identified 
in the mission need statement developed during the mission analysis 
phase. FAA also has detailed procedures for developing this document 
that call for identifying business needs. Resources for ensuring that 
IT projects and systems support the organization's business needs and 
meet users' needs include service organizations, the Corporate Mission 
Analysis Organization, the Mission Analysis Steering Group, and 
detailed procedures and associated templates for developing mission 
need statements. FAA's specific business mission, with stated goals and 
objectives, is defined in the Federal Aviation Administration Flight 
Plan for fiscal years 2004 through 2008.

Further, FAA defines and documents business needs for both proposed and 
ongoing IT projects and identifies users and other beneficiaries during 
its mission analysis activities. In addition, the AMS policy calls for 
users to participate in project management throughout the FAA life 
cycle management process. For the three projects we reviewed,[Footnote 
18] we verified that business needs and specific users and other 
beneficiaries were identified and documented in mission needs 
statements as well as in other documents. In addition, users are 
involved in project management throughout the life cycle of the 
projects. For example, according to project officials, En Route 
Communications Gateway (ECG) users participate in project meetings, 
weekly integrated product team status meetings, and monthly En Route 
domain national deployment teleconferences. FAA Telecommunications 
Infrastructure's (FTI) end users are heavily involved in the 
"operational test" period, which determines whether the equipment can 
be safely implemented in NAS. VSCS Control Subsystem Upgrade[Footnote 
19] users are involved in the project's life cycle via a Web site 
through which they review and comment on project documentation.

Despite these strengths, the JRC has no process for evaluating the 
organizational alignment of NAS systems through most of their in-
service management phase (and non-NAS investments, which are described 
separately in this report). While the JRC does evaluate the alignment 
of projects and systems with organizational goals throughout the 
systems' development and 2 years into their operations as part of the 
annual budget formulation process, it does not use any consistent 
process to review projects and systems after that point in their life 
cycles. For NAS systems in the in-service management phase, these 
activities are carried out within the business unit that owns the 
system, but the JRC does not regularly oversee these processes and may 
go for several years without reviewing a system's alignment with 
organizational goals. In-service NAS systems only return to the JRC if 
they are judged to require additional funds for correction. Until FAA 
establishes a process for periodic evaluation of systems throughout the 
in-service management phase and takes corrective actions when 
misalignment occurs, the agency will not be able to ensure that these 
projects, totaling about $1.3 billion per year, are still continuing to 
maintain alignment with the FAA's strategic plans and its business 
goals and objectives.

Table 5 shows the rating for each key practice required to implement 
the critical process for meeting business needs at the Stage 2 level of 
maturity and summarizes the evidence that supports these ratings.

Table 5: Meeting Business Needs:

Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and 
procedures for ensuring IT projects or systems that support the 
organization's ongoing and future business needs;
Rating: Executed;
Summary of evidence: AMS and mission analysis guidance contain 
documented policies and procedures for identifying the IT projects or 
systems that support the organization's ongoing and future business 
needs.

Type of practice: Prerequisites;
Key practice: 2. The organization has a documented business mission 
with stated goals and objectives;
Rating: Executed;
Summary of evidence: The Federal Aviation Administration Flight Plan 
(Strategic Plan) for fiscal years 2004 through 2008 defines the 
agency's mission goals and objectives.

Type of practice: Prerequisites;
Key practice: 3. Adequate resources, including people, funding, and 
tools, are provided for ensuring that IT projects and systems support 
the organization's business needs and meet users' needs;
Rating: Executed;
Summary of evidence: FAA has adequate resources for ensuring that its 
IT projects and systems support the organization's business needs and 
meet users' needs. They include service organizations, the Mission 
Analysis Steering Group, and the Corporate Mission Analysis 
Organization. FAA also has detailed procedures and associated templates 
for developing mission need statements.

Type of practice: Activities;
Key practice: The organization defines and documents business needs for 
both proposed and ongoing IT projects and systems;
Rating: Executed;
Summary of evidence: AMS policy calls for business needs for both 
proposed and ongoing IT projects and systems to be specified in the 
mission need statement. We verified that business needs were defined 
and documented in mission need statements for the three projects we 
reviewed.

Type of practice: Activities;
Key practice: The organization identifies specific users and other 
beneficiaries of IT projects and systems;
Rating: Executed;
Summary of evidence: FAA policy and procedures call for specific users 
and other beneficiaries of IT projects and systems to be identified. We 
verified that specific users and other beneficiaries were identified 
for the three projects we reviewed.

Type of practice: Activities;
Key practice: 3. Users participate in project management throughout an 
IT project's or system's life cycle;
Rating: Executed;
Summary of evidence: FAA policies and procedures call for users to 
participate in project management throughout an IT project's or 
system's life cycle. We verified that users participated in project 
management throughout the life cycle of the three projects we reviewed.

Type of practice: Activities;
Key practice: 4. The investment board periodically evaluates the 
alignment of its IT projects and systems with the organization's 
strategic goals and objectives and takes corrective actions when 
misalignment occurs;

The investment board periodically evaluates the alignment of its IT 
projects and systems with the organization's strategic goals and 
objectives and takes corrective actions when misalignment occurs;
Rating: Not executed;
Summary of evidence: The JRC evaluates the alignment of systems through 
development and 2 years into operations with the organization's 
strategic goals and objectives--through the annual budget formulation 
process--and takes corrective actions when misalignment occurs. 
However, there is no process for the JRC to periodically evaluate the 
alignment of investments later in their life cycles.

Source: GAO.

[End of table]

FAA Has a Disciplined Process for Selecting New IT Proposals but Lacks 
a Similar Process for Reselecting Ongoing Investments:

Selecting new IT proposals and reselecting ongoing investments requires 
a well-defined and disciplined process to provide the agency's 
investment board, business units, and developers with a common 
understanding of the process and the cost, benefit, schedule, and risk 
criteria that will be used both to select new projects and to reselect 
ongoing projects for continued funding. According to ITIM, this 
critical process requires, among other things, (1) making funding 
decisions for new proposals according to an established process; (2) 
providing adequate resources for investment selection activities; (3) 
using a defined selection process to select new investments and 
reselect ongoing investments; (4) establishing criteria for analyzing, 
prioritizing, and selecting new IT investments and for reselecting 
ongoing investments; and (5) creating a process for ensuring that the 
criteria change as organizational objectives change. (The complete list 
of key practices is provided in table 6.):

FAA has executed 7 of the 10 key practices associated with selecting an 
investment. For example, the AMS establishes two processes--mission 
analysis and investment analysis--that together constitute a set of 
policies and procedures, as well as guidance that is designed to 
enhance the agency's ability to select investments. In addition, FAA 
has policies and procedures for its annual F&E budget formulation 
process to reselect ongoing IT projects. Also, FAA's AMS sets forth 
policies and procedures for reselecting ongoing IT investments by 
identifying their capability shortfalls and addressing them as new 
investments.

The AMS also integrates funding with the process of selecting an 
investment by requiring the Systems Engineering/Operational Analysis 
Team to perform affordability assessments for new proposed investment 
programs; it may recommend funding reallocations from lower priority 
programs when an alternative solution cannot be funded within FAA 
planning and budgeting baselines. This team also supports the JRC to 
ensure that the executives' funding decisions are aligned with 
selection decisions during the investment analysis activities.

Resources for proposal selection activities include the program 
director, the Integrated Product Team, and the Investment Analysis 
Team, as well as detailed procedures and a template that have been 
defined for developing investment analysis reports. The investment 
analysis reports identify the evaluation criteria used, the 
alternatives analyzed, and the ranking of each alternative so that the 
JRC can select the best overall solution identified in the mission need 
statement. The criteria that were established during the initial 
investment analysis phase are used by the Investment Analysis Team to 
rank each proposed project on the basis of how well it meets the 
agency's mission needs compared with other projects.

FAA uses the processes defined in the AMS for selecting new IT 
investments. In addition, it uses two processes to reselect ongoing IT 
investments. Specifically, the FAA uses its annual budget formulation 
process for projects in development or in the first 2 years of 
operations. It also uses the AMS process when a system's capability 
shortfall is identified, and it treats the correction of the shortfall 
as a new investment. The managers of the three projects we reviewed 
confirmed that their projects were selected using the AMS process. One 
project's officials stated that this included market, alternatives, 
investment, and affordability analyses. The program managers also 
stated that the annual F&E budget formulation process is used to 
reselect their projects. These project officials also noted that if a 
project is scheduled for a hardware replacement, a reselection is done. 
The AMS process is followed to explore new alternatives and make sure 
the replacement is in the best interest of the government.

Despite these strengths, FAA has not developed similarly strong 
processes for NAS investments more than 2 years into their operations-
-those NAS systems that are in the in-service management phase. For 
example, while FAA's F&E budget formulation process establishes 
criteria for analyzing, prioritizing, and reselecting IT investments 
for systems in development or up until 2 years into operations, neither 
of the two processes used to reselect IT investments has established 
criteria for investments beyond 2 years into operations. In addition, 
while FAA uses its annual budget formulation process to reselect 
projects that are part of the F&E budget, the agency does not have an 
analogous reselection process as part of its operations budget 
formulation. Until FAA establishes consistent criteria for reselecting 
all of its IT investments, it will not be adequately assured that it is 
consistently and objectively continuing to fund ongoing projects that 
still meet the needs and priorities of the agency in a cost-effective 
and risk-insured manner.

Table 6 shows the rating for each key practice required to implement 
the critical process for selecting an investment at the Stage 2 level 
of maturity and summarizes the evidence that supports these ratings.

Table 6: Selecting an Investment:

Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and 
procedures for selecting new IT proposals;
Rating: Executed;
Summary of evidence: FAA's AMS policy, mission analysis, and investment 
analysis guidance has documented policies and procedures for selecting 
new IT proposals.

Type of practice: Organizational commitments;
Key practice: 2. The organization has documented policies and 
procedures for reselecting ongoing IT investments;
Rating: Executed;
Summary of evidence: FAA has documented policies and procedures for its 
annual F&E budget formulation process, which is used to reselect 
ongoing IT projects. In addition, FAA's AMS policy has documented 
policies and procedures for reselecting ongoing IT investments 
throughout the FAA's acquisition life cycle.

Type of practice: Organizational commitments;
Key practice: 3. The organization has documented policies and 
procedures for integrating funding with the process of selecting an 
investment;
Rating: Executed;
Summary of evidence: FAA's AMS and investment analysis guidance have 
documented policies and procedures for integrating funding with the 
process of selecting an investment.

Type of practice: Prerequisites;
Key practice: 1. Adequate resources, including people, funding, and 
tools, are provided for identifying and selecting IT projects and 
systems;
Rating: Executed;
Summary of evidence: Adequate resources are provided for identifying 
and selecting IT projects and systems. They include the program 
director, Integrated Product Teams, and the Investment Analysis Team. 
FAA also has detailed procedures and associated templates for 
developing investment analysis reports.

Type of practice: Prerequisites;
Key practice: 2. Criteria for analyzing, prioritizing, and selecting 
new IT investment opportunities have been established;
Rating: Executed;
Summary of evidence: The Investment Analysis Team has established 
criteria for analyzing, prioritizing, and selecting new IT investment 
opportunities. The investment analysis report, which is submitted to 
the JRC, identifies the evaluation criteria, the alternatives analyzed, 
and the ranking for each alternative.

Type of practice: Prerequisites;
Key practice: 3. Criteria for analyzing, prioritizing, and 
reselecting[A] IT investment opportunities have been established;
Rating: Not executed;
Summary of evidence: While FAA's F&E budget formulation process has 
established criteria for analyzing, prioritizing, and reselecting IT 
investments that are part of that budget, neither of the two processes 
used to reselect IT investment opportunities has established criteria 
for investments beyond 2 years into operations.

Type of practice: Prerequisites;
Key practice: 4. A mechanism exists to ensure that the criteria 
continue to reflect organizational objectives;
Rating: Not executed;
Summary of evidence: While FAA ensures that the criteria continue to 
reflect organizational objectives for selecting new IT investments, 
there are no consistent criteria used by the JRC to reselect 
investments more than 2 years into operations.

Type of practice: Activities;
Key practice: 1. The organization uses its defined selection process, 
including predefined selection criteria, to select new IT investments;
Rating: Executed;
Summary of evidence: FAA uses the policies and procedures defined in 
the AMS, mission analysis, and investment analysis guidance to select 
new IT investments. We verified that the three projects we reviewed 
were selected using the mission analysis and investment analysis 
activities defined in the AMS, mission analysis and the investment 
analysis guidance.

Type of practice: Activities;
Key practice: 2. The organization uses the defined selection process, 
including predefined selection criteria, to reselect[A] ongoing IT 
investments;
Rating: Not executed;
Summary of evidence: FAA has a process defined in the AMS to reselect 
ongoing IT investments. It also uses its annual budget formulation 
process to reselect projects that are part of the F&E budget. According 
to the project managers of the three projects we reviewed, this budget 
process is used to reselect their projects. However, FAA does not 
consistently use these defined processes to reselect IT investments 
more than 2 years into operations.

Type of practice: Activities;
Key practice: 3. Executives' funding decisions are aligned with 
selection decisions;
Rating: Executed;
Summary of evidence: The Systems Engineering/Operational Analysis Team, 
which is composed of representatives from FAA's service organizations, 
supports the JRC in making funding decisions that are aligned with 
selection decisions as part of FAA's investment analysis activities.

Source: GAO.

[A] According to the GAO ITIM framework, reselecting is the periodic 
reconsideration of an investment's continuing value to the organization 
and the decision to continue funding. It is a recurring process that 
continues for as long as a project is receiving funding.

[End of table]

FAA Does Not Have a Process for Effectively Overseeing Investments in 
All Phases of Their Life Cycles:

An organization should provide effective oversight for its IT projects 
throughout all phases of their life cycles. Its investment board should 
maintain adequate oversight and observe each project's performance and 
progress toward predefined cost and schedule expectations as well as 
each project's anticipated benefits and risk exposure. The investment 
board should also employ early warning systems that enable it to take 
corrective action at the first sign of cost, schedule, or performance 
slippages. This board has ultimate responsibility for the activities 
within this critical process. According to ITIM, effective project 
oversight requires, among other things, (1) having written policies and 
procedures for management oversight; (2) developing and maintaining an 
approved management plan for each IT project; (3) making up-to-date 
cost and schedule data for each project available to the oversight 
boards; (4) having regular reviews by each investment board of each 
project's performance against stated expectations; and (5) ensuring 
that corrective actions for each underperforming project are 
documented, agreed to, implemented, and tracked until the desired 
outcome is achieved. (The complete list of key practices is provided in 
table 7.):

FAA has in place 4 of the 7 key practices associated with effective 
project oversight. The agency has developed written policies and 
procedures for management oversight of its investments. These include 
(1) AMS; (2) the integrated program plan, which is the detailed 
planning document for all aspects of a program's implementation, 
including program control; and (3) the Integrated Baseline 
Establishment and Management Process document for reporting variances 
from the performance expectations approved by the JRC in the 
acquisition program baseline.

We verified that cost, schedule, benefit, and risk expectations were 
documented in the acquisition program baseline and that the integrated 
program plan contained details for project execution for En Route 
Communications Gateway and FAA Telecommunications Infrastructure. For 
the VSCS Control Subsystem Upgrade, performance expectations and 
details on project execution were both captured in the integrated 
program plan.[Footnote 20] In addition, the JRC Secretariat Team 
maintains a tracking system for action items that are assigned during a 
project's acquisition reviews, including the action to be taken, the 
responsible FAA organization, and whether the underlying problem has 
been resolved.

FAA has not established processes that bring investments before the JRC 
for oversight on a regular basis. There is a process for reporting 
variances from the performance expectations that were approved by the 
JRC in the investment's acquisition program baseline. However, although 
this process is carried out as part of the F&E budget formulation for 
IT investments in development or less than 2 years into operations, it 
is not being carried out for investments that are part of the 
operations budget. Investments that are meeting performance 
expectations may not return to the JRC for several years. FAA also 
conducts acquisition reviews as a means for program offices to report 
to agency executives on the status of investments compared to program 
baselines. However, since program offices may select which investments 
they wish to bring forward for review, many investments may never come 
forward. Until FAA develops (1) procedures for reporting on an 
investment throughout its entire acquisition life cycle and (2) 
mechanisms for ensuring that all investments are reviewed regularly, 
the agency is placing itself at risk that underperforming investments 
will not be reported to the JRC in order for it to take appropriate 
actions.

Table 7 shows the rating for each key practice that is required to 
implement the critical process for project oversight at the Stage 2 
level of maturity and summarizes the evidence that supports these 
ratings.

Table 7: Providing Investment Oversight:

Type of practice: Organizational commitment;
Key practice: 1. The organization has documented policies and 
procedures for management oversight of IT projects and systems;
Rating: Executed;
Summary of evidence: FAA has developed written policies and procedures 
for management oversight of IT projects and systems. These include AMS, 
the integrated program plan, and the Integrated Baseline Establishment 
and Management Process document for reporting variances from the 
performance expectations approved in the acquisition program baseline 
for an investment program.

Type of practice: Prerequisites;
Key practice: 1. Adequate resources, including people, funding, and 
tools, are provided for IT project oversight;
Rating: Executed;
Summary of evidence: FAA has adequate resources for providing IT 
project oversight. The agency has staff for compiling monthly variance 
reports submitted by the investment program areas, preparing quarterly 
baseline variance reports for the JRC, and preparing semi-annual 
baseline variance report for the FAA Administrator. An automated system 
is used to facilitate the maintenance of information for these reports.

Type of practice: Prerequisites;
Key practice: 2. IT projects and systems, including those in steady 
state (operations and maintenance), maintain approved project 
management plans that include expected cost and schedule milestones and 
measurable benefit and risk expectations;
Rating: Executed;
Summary of evidence: AMS policy calls for an acquisition program 
baseline (APB) document and an integrated program plan to be available 
at the JRC final investment decision point. The APB document serves as 
the AMS cost/schedule/ technical performance/benefits/risks control 
document. The integrated program plan specifies how the APB baselines 
will be controlled and details the management, contracting, and 
technical actions and activities to be performed in executing the 
acquisition. We verified that cost, schedule, benefit, risk, and 
performance expectations were documented in the APBs for ECG and FTI. 
For VCSU, these expectations were documented in an integrated program 
plan.

Type of practice: Activities;
Key practice: 1. Data on actual performance (including cost, schedule, 
benefit, and risk performance) are provided to the appropriate IT 
investment board;
Rating: Not executed;
Summary of evidence: FAA has established a process for reporting 
variances to the JRC from the performance expectations that have been 
approved by the JRC in the APB for an investment. This process is 
carried out for IT investments that are part of the F&E budget, but it 
is not being carried out for investments that are managed as part of 
the operations budget.

Type of practice: Activities;
Key practice: 2. Using verified data, each investment board regularly 
reviews the performance of IT projects and systems against stated 
expectations;
Rating: Not executed;
Summary of evidence: FAA does not have a process that provides an 
opportunity for the JRC to regularly review investment performance. It 
has a process for conducting acquisition reviews where program offices 
provide status information to agency executives on the progress of 
investments against their acquisition program baselines. However, the 
individual program offices choose which investments they want to 
discuss at these reviews. Also, although the process for reporting 
variances from the performance expectations approved by the JRC in the 
acquisition program baseline is carried out for IT investments that are 
part of the F&E budget, this process is not being carried out for 
investment programs that are part of the operations budget, and it only 
results in investments with variances to be reviewed.

Type of practice: Activities;
Key practice: 3. For each underperforming IT project or system, 
appropriate actions are taken to correct or terminate the project or 
system in accordance with defined criteria and the documented policies 
and procedures for management oversight;
Rating: Not executed;
Summary of evidence: During acquisition reviews, action items are 
identified for investment programs discussed, an organization assigned 
responsibility to carry them out, and the items are tracked until the 
appropriate action is taken, at which time they are closed out. 
Similarly, variance reports are prepared quarterly for the JRC 
identifying investments with a 10 percent or greater variance from the 
established acquisition program baseline. An investment program is to 
remain on the quarterly variance report until successful corrective 
action is taken. However, FAA has no mechanism that provides assurance 
that every program has an acquisition review regularly, since it is 
left to the individual program offices to decide which programs they 
want discussed at the reviews. Also, although variance reports are 
prepared for IT investments that are part of the F&E budget, reports 
are not prepared for investments that are part of the operations 
budget.

Type of practice: Activities;
Key practice: 4. The investment board regularly tracks the 
implementation of corrective actions for each underperforming project 
until the actions are completed;
Rating: Executed;
Summary of evidence: The JRC's Secretariat Team maintains a tracking 
system for action items assigned during a project's acquisition 
reviews. This system identifies the action to be taken, what FAA 
organization is to perform it, and whether it is open or closed.

Source: GAO.

[End of table]

FAA Has a Structured Process for Capturing Investment Information and 
Using It to Support Investment Management:

To make good IT investment decisions, an organization must be able to 
acquire pertinent information about each investment and store that 
information in a retrievable format. During this critical process an 
organization identifies its IT assets and creates a comprehensive 
repository of investment information. This repository provides 
information to investment decision makers to help them evaluate the 
impacts and opportunities that would be created by proposed or 
continuing investments. It can provide insights and trends about major 
IT cost and management drivers. The repository can take many forms and 
does not have to be centrally located, but the collection method should 
identify each IT investment and its associated components. This 
critical process may be satisfied by the information contained in the 
organization's current enterprise architecture, augmented by 
additional information--such as financial information and information 
on risk and benefits--that the investment board may require to ensure 
that informed decisions are being made. According to ITIM, effectively 
managing this repository requires, among other things, (1) developing 
written policies and procedures for identifying and collecting the 
information, (2) assigning responsibility for ensuring that the 
information being collected meets the needs of the investment 
management process, (3) identifying IT projects and systems and 
collecting relevant information to support decisions about them, and 
(4) making the information easily accessible to decision makers and 
others. (The complete list of key practices is provided in table 8.):

FAA's AMS guidance identifies specific information that is needed in 
the investment management process, including information for its 
investment analysis phase. FAA maintains a number of repositories of 
relevant information, including its Simplified Program Information 
Reporting & Evaluation database, which reports variances in cost, 
schedule, performance, or benefits from an investment's approved 
acquisition program baseline. The information that is collected is made 
available to the JRC in several documents, including program plans and 
the acquisition program baseline document. The JRC Secretariat Team 
ensures that the investment board has all the relevant information it 
needs for its decision-making process.

Table 8 shows the rating for each key practice required to implement 
the critical process for capturing investment information at the Stage 
2 level of maturity and summarizes the evidence that supports these 
ratings.

Table 8: Capturing Investment Information:

Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and 
procedures for identifying and collecting information about IT projects 
and systems to support the investment management process;
Rating: Executed;
Summary of evidence: FAA has developed policies and procedures for 
identifying and collecting information to support the investment 
management process. For example, AMS guidance indicates what 
information is needed for the investment analysis phase of FAA's 
investment management process.

Type of practice: Organizational commitments;
Key practice: 2. An official is assigned responsibility for ensuring 
that the information collected during project and systems 
identification meets the needs of the investment management process;
Rating: Executed;
Summary of evidence: AMS guidance specifies which officials are 
responsible for approving the completion of reports containing 
information prepared for the investment management process. The JRC 
Secretariat Team ensures through its readiness process that all of the 
necessary information is available to the JRC for its decision making.

Type of practice: Prerequisite;
Key practice: 1. Adequate resources, including people, funding, and 
tools, are provided for identifying IT projects and systems and 
collecting relevant investment information about them;
Rating: Executed;
Summary of evidence: FAA has adequate resources for meeting this key 
practice. Several teams, including a mission analysis team and an 
investment analysis team, collect the relevant investment information 
needed by the JRC to make its decisions on which investments to 
approve. The JRC Secretariat Team ensures that the JRC has all the 
relevant information for its decision making.

Type of practice: Activities;
Key practice: 1. The organization's IT projects and systems are 
identified, and specific information is collected to support decisions 
about them;
Rating: Executed;
Summary of evidence: FAA maintains data relevant to the investment 
management process in several sources, including its financial 
management system, its Simplified Program Information Reporting & 
Evaluation tool, and its Capital Investment Plan. Also, AMS guidance 
identifies information to be collected for the investment management 
process--including for the investment analysis phase--to aid the JRC in 
its final investment decisions.

Type of practice: Activities;
Key practice: 2. The information that has been collected is easily 
accessible and understandable to decision makers and others;
Rating: Executed;
Summary of evidence: Information collected for the JRC decision-making 
process is compiled in documents such as detailed program plans, 
acquisition program baselines, and investment analysis reports. The JRC 
Secretariat Team ensures that the JRC has all the relevant information 
for its decision making through its readiness process.

Type of practice: Activities;
Key practice: 3. The information repository is used by investment 
decision makers and others to support investment management;
Rating: Executed;
Summary of evidence: The JRC Secretariat Team, through its JRC 
readiness process, collects information for the JRC to use in its 
decision-making process. The JRC also receives an investment analysis 
report that contains all of the information that has been gathered 
during investment analysis activities.

Source: GAO.

[End of table]

FAA Does Not Have Structured Processes to Manage Its Non-NAS 
Investments: 

FAA does not have a single set of processes for making consistent basic 
selection and control decisions for its non-NAS investments (Stage 2 
capabilities). As previously discussed in the background section of 
this report, several business units within FAA make decisions about 
non-NAS investments. We reviewed the investment management processes of 
seven of these units--Information Services, Region and Center 
Operations, Regulation and Certification, Financial Services, Research 
and Acquisition, Air Traffic Services, and Human Resource Management. 
Appendix II describes the investment management processes we found in 
these units. The extent to which these processes comply with the ITIM 
framework for Stage 2 varies considerably by business unit, and FAA 
currently does not specify non-NAS investment management processes in a 
coordinated manner. Since the ITIM framework calls for a consistent 
investment management process, we assessed FAA's non-NAS investment 
management capability at an aggregate level. That is, we assessed FAA's 
capability to manage its non-NAS investments, not the capability of 
each individual business unit. Even though individual business units 
may have some of these processes in place, FAA as a whole has not yet 
defined: 

* an investment management structure that allows the agency to 
consistently manage its non-NAS investments,

* a uniform process for ensuring that non-NAS investments are linked to 
business needs and meet users' needs,

* a process for selecting new IT proposals and reselecting ongoing 
investments,

* a single process for reviewing the progress of investments and taking 
corrective action when performance expectations are not being met, or: 

* a comprehensive inventory of project and system information to 
support investment decisions.

According to FAA officials, the agency has not defined a coherent 
investment management structure and a set of processes for non-NAS 
investments in the past because many of these investments have not had 
the agencywide impact of the NAS investments. However, because there is 
now recognition that a disciplined approach to managing non-NAS 
investments could help control FAA's IT assets and costs in general, 
efforts are currently under way to address this weakness. As previously 
discussed, an IT Executive Board (ITEB) has been chartered with 
responsibility for, among other things, making decisions about non-NAS 
IT investments, but it has not yet taken action on developing a 
standard process. Until FAA fully establishes the consistent practices 
it needs to make basic project selection and control decisions, 
executives will be hampered in their ability to effectively manage non-
NAS investments and ultimately to find the opportunities to achieve the 
cost savings they are seeking.

FAA Lacks Key Capabilities Needed to Manage All IT Investments as a 
Portfolio and Does Not Conduct Postimplementation Reviews: 

During Stage 3, the investment board enhances the IT investment 
management process by developing a complete investment portfolio and 
carrying out PIRs. An IT investment portfolio is an integrated, 
agencywide collection of investments that are assessed and managed 
collectively on the basis of common criteria. Managing investments 
within the context of such a portfolio is a conscious, continuous, and 
proactive approach to expending limited resources on an organization's 
competing initiatives in light of the relative benefits expected from 
these investments. Taking an agencywide perspective enables an 
organization to consider its investments comprehensively, so that 
collectively the investments optimally address the organization's 
missions, strategic goals, and objectives. Managing IT investments with 
a portfolio approach also allows an organization to determine 
priorities and make decisions about which projects to fund, and 
continue to fund, based on analyses of the relative organizational 
value and risks of all projects, including projects that are proposed, 
under development, and in operation. For an organization to reap the 
full benefits of the portfolio process, it should collect all of its 
investments into an enterprise-level portfolio that is overseen by its 
senior investment board. Although investments may initially be selected 
into subordinate portfolios--based on, for example, lines of business 
or life cycle stages--and managed by subordinate investment boards, 
they should ultimately be aggregated into this enterprise-level 
portfolio.

The purpose of a PIR is to evaluate an investment after its development 
has been completed (i.e., after its transition from the implementation 
phase to the in-service management phase) in order to validate actual 
investment results. This review is conducted to (1) examine differences 
between estimated and actual investment costs and benefits and their 
possible ramifications for unplanned funding needs in the future and 
(2) extract "lessons learned" about the investment selection and 
control processes that can be used as the basis for management 
improvements. Similarly, PIRs should be conducted for investment 
projects that were terminated before completion, to help to readily 
identify potential management and process improvements.

According to ITIM, critical processes performed by Stage 3 
organizations include (1) defining the portfolio criteria, 
(2) creating the portfolio, (3) evaluating the portfolio, and 
(4) conducting PIRs. Table 9 shows the purpose of each critical process 
in Stage 3.

Table 9: Stage 3 Critical Processes--Developing a Complete Investment 
Portfolio: 

Critical process: Defining the portfolio criteria; 
Purpose: To ensure that the organization develops and maintains IT 
portfolio selection criteria that support its mission, organizational 
strategies, and business priorities.

Critical process: Creating the portfolio; 
Purpose: To ensure that IT investments are analyzed according to the 
organization's portfolio selection criteria and that an optimal IT 
investment portfolio with manageable risks and returns is selected and 
funded.

Critical process: Evaluating the portfolio; 
Purpose: To review the performance of the organization's investment 
portfolio(s) at agreed-upon intervals and to adjust the allocation of 
resources among investments as necessary.

Critical process: Conducting postimplementation reviews; 
Purpose: To compare the results of recently implemented investments 
with the expectations that were set for them and to develop a set of 
lessons learned from these reviews.

Source: GAO.

[End of table]

FAA has executed only 1 of the 27 key practices associated with Stage 3 
critical processes: it has a process for distributing portfolio 
criteria to project management personnel and other stakeholders. The 
remaining 26 key practices were not executed--primarily because FAA 
does not involve the JRC in the regular oversight of non-NAS 
investments or in NAS investments during their in-service management 
phase, weaknesses that we noted in our assessment of Stage 2 
requirements. Since Stage 3 requires an enterprisewide perspective, the 
lack of oversight of these classes of investments precludes the 
successful completion of most Stage 3 critical processes. In addition, 
Stage 3 requires an enterprisewide perspective that FAA has not 
adopted, which would enable the JRC to oversee all major IT 
investments, regardless of life cycle phase or business unit. Although 
it can be appropriate for FAA to manage its NAS, in-service NAS, and 
non-NAS investments as separate subordinate portfolios--depending on 
the successful execution of all Stage 2 key practices--its enterprise-
level portfolio should contain all major IT investments regardless of 
life cycle stage or business line. In building this enterprise-level 
portfolio, the JRC can choose whether to include specific investments 
based on predetermined criteria, as described by the ITIM framework. 
Until FAA fully implements the critical processes associated with 
managing its investments as a complete portfolio, it will not have the 
data or enterprisewide perspective it needs to make informed decisions 
about all of its major IT investments.

In addition, FAA has not executed the six key practices for conducting 
PIRs. In June 2004, in response to a recommendation contained in our 
1999 report[Footnote 21] that FAA initiate PIRs for projects or 
programs within 3 to 12 months of deployment or termination, the NAS 
Configuration Management and Evaluation Staff developed a proposed 
approach to PIRs, but this approach was not implemented. In November 
2003, the life cycle management policy team proposed a change to the 
AMS that would require conducting these reviews, but there has been no 
action on the proposal. Although the JRC has recently reaffirmed its 
commitment to implement PIRs, there is no policy and no established 
process to carry them out. If PIRs are not conducted on a routine 
basis, then FAA will not be able to effectively evaluate the results of 
its IT investments; this will affect the agency's ability to determine 
whether to continue, modify, or terminate an IT investment in order to 
meet its stated mission objectives.

Table 10 summarizes the status of FAA's critical processes for Stage 3, 
showing how many associated key practices it has executed.

Table 10: Status of Stage 3 Critical Processes: 

Critical process: Defining the portfolio criteria; 
Key practices executed: 1; 
Total required by critical process: 7; 
Percentage of key practices executed: 14.

Critical process: Creating the portfolio; 
Key practices executed: 0; 
Total required by critical process: 7; 
Percentage of key practices executed: 0.

Critical process: Evaluating the portfolio; 
Key practices executed: 0; 
Total required by critical process: 7; 
Percentage of key practices executed: 0.

Critical process: Conducting postimplementation reviews; 
Key practices executed: 0; 
Total required by critical process: 6; 
Percentage of key practices executed: 0.

Totals; 
Key practices executed: 1; 
Total required by critical process: 27; 
Percentage of key practices executed: 4. 

Source: GAO.

[End of table]

FAA Has Not Established a Process for Managing the Succession of Its 
Information Systems: 

Once an agency has attained Stage 3 maturity, it evaluates its IT 
investment processes and portfolios to identify opportunities for 
improvement (Stage 4 capabilities). This entails (1) improving the 
portfolio's performance and (2) managing systems and technology 
succession. We did not assess FAA's capability for improving the 
portfolio's performance, because it did not claim to be executing any 
of the relevant key practices in its self-assessment.

According to ITIM, regarding system and technology succession 
management includes (1) defining policies and procedures for managing 
the IT succession process, (2) assigning responsibility for the IT 
succession process, (3) developing criteria for identifying IT 
investments that may meet succession status, and (4) periodically 
analyzing IT investments to determine whether they are ready for 
succession. This critical process enables an organization to recognize 
low-value or high-cost IT investments and augments the routine 
replacement of systems at the end of their useful lives. It also 
promotes the development of a forward-looking, solution-oriented view 
of IT investments that anticipates future resource requirements and 
allows the organization to plan appropriately. This process differs 
from the reselection activity in Stages 2 and 3 in that it focuses on 
anticipating and planning for the retirement of legacy systems and on 
meeting remaining requirements with other, perhaps new, systems. In 
addition, succession management takes place at the end of a system's 
life cycle.

FAA has not executed any of the nine key practices required to 
implement this critical process. Although the agency has defined 
procedures in AMS for retiring investments, it still needs to describe 
how to regularly review systems that are in operations in order to 
identify candidates for retirement. According to FAA, decisions on 
succession are made by the service organizations. However, no 
individual or group has been assigned responsibility for managing the 
succession process from an enterprise perspective, which would allow 
the FAA to better anticipate and plan for future resource requirements. 
Without an institutionalized process for succession management, the FAA 
may not be able to identify those IT investments that are eligible for 
succession in enough time to minimize the effect of the transition on 
their successors. In addition, by establishing an effective succession 
management process, the agency can identify systems for retirement, 
freeing resources for other, superior, investments.

FAA Has Initiated Efforts to Improve Its Investment Management Process: 

We have previously reported that to effectively implement IT investment 
management processes, organizations need to be guided by a plan that 
(1) is based on an assessment of strengths and weaknesses; 
(2) specifies measurable goals, objectives, and milestones; 
(3) specifies needed resources; (4) assigns clear responsibility and 
accountability for accomplishing tasks; and (5) is approved by senior 
management.

FAA has begun to take steps to resolve some of the weaknesses 
identified in this report. For example, at a June 10, 2004, meeting, 
the JRC decided to incorporate budget justification documents 
(Exhibit 300s), which are currently prepared for the Office of 
Management and Budget (OMB) as part of the President's Budget 
formulation process, into the AMS process for managing NAS investments. 
The Exhibit 300 will become the board's decision-making document, and 
essential information from existing AMS-required documents--the 
investment management report, the acquisition strategy paper, the 
integrated program plan, and the requirements documents--will be 
incorporated into the Exhibit 300. The JRC also recently decided to 
implement PIRs in order to track metrics during program implementation. 
Finally, at that same meeting, the board decided to collectively 
determine, at the meeting where the F&E budget is approved, which F&E 
and OPS programs should be brought forward for review the following 
year. This decision serves to bring certain investments in the 
in-service management phase under the JRC's direct purview, although it 
does not specify that consistent criteria be established, as the ITIM 
framework requires.

FAA has also begun to initiate steps to bring more clarity to the 
ITEB's responsibilities, although the specifics have yet to be defined. 
In its charter, the ITEB is charged with making investment decisions 
about non-NAS IT investments. This action would begin to bring all of 
the non-NAS investments under a single authority. The charter suggests 
that the ITEB choose among three options: (1) to send major non-NAS 
investment decisions to the JRC, (2) to make the decision itself, given 
an acceptable review process similar to the JRC processes, or (3) have 
the CIO, Chief Financial Officer, and owning assistant/associate 
administrator make the decision jointly. This description of the ITEB's 
roles and responsibilities further alludes to the senior board's 
evolving responsibility toward major non-NAS IT investments, although 
it falls short of laying out specific criteria for selecting which 
investments should be sent forward to the JRC. The ITEB has been given 
responsibility for four short-term initiatives as well, including 
establishing an agencywide cost control program for non-NAS 
expenditures and ensuring that all OMB Exhibit 300s receive a passing 
grade for the 2006 budget year. The ITEB has been charged with the 
long-term initiative of clearly delineating the roles it plays and its 
relationship with the more senior board. The successful completion of 
this initiative is likely to satisfy the single key practice that FAA 
has not yet executed in the Instituting the Investment Board critical 
process of the ITIM.

The Chief Operating Officer's recent reorganization of the ATO is 
intended to make the heads of the service units responsible for IT 
projects from their inception through the in-service management phase. 
This new organization is designed to support his expressed intentions 
to increase accountability for systems in operation in order to manage 
costs more effectively. According to the Chief Operating Officer, FAA 
recognizes that good processes are needed for both NAS and non-NAS to 
improve the way the agency manages its investments.

While FAA has initiated these improvement efforts, it has not linked 
them together in a plan with the characteristics listed above that 
would help coordinate and guide the efforts. Until FAA develops a plan 
that would allow for the systematic prioritization, sequencing, and 
evaluation of improvement efforts, the agency risks not being able to 
effectively establish mature investment management processes.

DOT is Taking Steps to Integrate Oversight of FAA's IT Investments: 

DOT has recently initiated several efforts that can serve to provide 
better departmental oversight of FAA investments. This fiscal year DOT 
and FAA reached an agreement by which DOT reviews FAA's Exhibit 300s as 
part of the department's annual budget process, in which all 
departmental components participate. Under this agreement, DOT conducts 
a review of all FAA Exhibit 300s starting in June of each budget year 
and culminating in the review of all Exhibit 300s by the Department 
Investment Review Board in late August, prior to the submission of the 
budget to OMB in September. As part of this agreement, DOT has outlined 
a process and schedule for reviewing the fiscal year 2006 budget 
justifications for major FAA programs and is monitoring FAA's progress 
in meeting this schedule. In addition, the department has identified 
about a dozen programs that it plans to monitor regularly and has begun 
reviewing these programs through its senior investment management 
decision-making board, on which the FAA Administrator is a voting 
member. DOT has also requested that FAA set reasonable expectations for 
cost, schedule, and performance for its major projects and that it then 
report quarterly on variances to those expectations. FAA submitted its 
first quarterly report as of June 2004. These regular reports are 
intended to help DOT maintain oversight of FAA's processes and ensure 
that they are appropriate and consistent with OMB's requirements. 
Furthermore, the department is currently planning to issue an 
investment management guide that specifies minimum expectations that 
its operating administrations (including FAA) are to follow in managing 
their investments. According to DOT officials, FAA has been complying 
with the department's requests for information to facilitate its 
oversight process.

Department officials are attributing their increased oversight--and 
cooperation from FAA--to the fact that the department has recently 
reinstituted its own investment management processes. In addition, DOT 
officials said that FAA now understands the role the department can 
play in helping it to obtain the funding it needs for its programs.

Conclusions: 

FAA has established most of the project selection and control 
capabilities needed to manage its NAS investments. This should help 
provide the executive-level decision-making and oversight capabilities 
required to establish accountability and guide major IT investments 
through most of their life cycles. However, weaknesses remain. For 
example, although business units are involved in the regular review of 
investments throughout their life cycles, the JRC may not review the 
performance of operations systems for several years unless they require 
significant additional funds. Also, FAA has yet to define and implement 
the practices it needs to select and control its non-NAS investments. 
Ultimately, because the JRC does not regularly review NAS systems 
during the in-service management phase and does not regularly review 
the non-NAS systems in general, significant portions of FAA's 
approximately $2.5 billion investment in IT go without top-level 
executive oversight and are not viewed as part of an enterprisewide 
portfolio. FAA has taken some initial steps to implement PIRs, but it 
has not yet established a process to carry them out.

The agency has begun to take some steps to develop improvements to 
address some of these weaknesses, such as establishing an Information 
Technology Executive Board with relevant responsibilities. In addition, 
the JRC has begun integrating some budgeting and oversight processes, 
and the Chief Operating Officer has begun to articulate a vision that 
includes additional accountability for investments in operations. But 
FAA has not developed a comprehensive plan to guide all improvement 
efforts. Such a plan would help coordinate and prioritize improvement 
efforts and help sustain commitment to the efforts under way. The 
increasing collaboration between FAA and DOT further contributes to the 
likelihood that the management of FAA's investments will improve as 
FAA's Exhibit 300s have the benefit of department-level review and the 
departmental investment review board conducts periodic reviews of 
selected projects.

Recommendations for Executive Action: 

To strengthen FAA's investment management capability and address the 
weaknesses discussed in this report, we recommend that the Secretary of 
the Department of Transportation direct the FAA Administrator to 
develop and implement a plan for improving FAA's IT investment 
management processes. The plan should address the weaknesses described 
in this report, beginning with those we identified in our Stage 2 
analysis and continuing with those we identified in our Stage 3. The 
plan should also draw together ongoing efforts as well as instituting 
new initiatives where called for. The plan should, at a minimum, 
provide for accomplishing the following: 

In Stage 2: 

* Define procedures for aligning the JRC and the newly established 
ITEB.

* Establish a process for the JRC to periodically reevaluate the 
alignment of projects in the in-service management phase with strategic 
goals and objectives.

* Establish a process for the JRC to regularly review the performance 
of IT systems throughout their life cycles and take corrective actions 
when expected performance is not being met.

* Define and implement an IT investment management structure, including 
an investment management board and a disciplined process for managing 
all non-NAS investments.

In Stage 3: 

* Define and implement processes for managing major investments as part 
of an enterprise-level portfolio, including NAS F&E investments, NAS 
investments in the in-service management phase, and non-NAS 
investments.

* Define and implement processes for carrying out PIRs on investments 
as they enter the in-service management stage.

In developing the plan, the FAA Administrator should ensure that it 
(1) specifies measurable goals, objectives, and milestones; 
(2) specifies needed resources; (3) assigns clear responsibility and 
accountability for accomplishing tasks; and (4) is approved by senior 
management. In implementing the plan, the FAA Administrator should 
ensure that the needed resources are provided to carry out the plan and 
that progress is measured and reported periodically to the Secretary of 
Transportation.

Agency Comments: 

In commenting on a draft of this report, DOT's Director of Audit 
Relations stated via e-mail that DOT appreciated the opportunity to 
review and offer comment on our report and that GAO had done a good job 
keeping the report balanced and fair, showing where FAA has many 
capabilities in place and identifying areas that need improvement. The 
Director also provided a technical comment, which we have incorporated 
into the report.

As agreed with your offices, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days 
from the date of this letter. At that time, we will send copies to 
other interested congressional committees, the Director of the Office 
of Management and Budget, the Secretary of Transportation, FAA's 
Administrator and CIO, and other interested parties. We also will make 
copies available to others upon request. In addition, the report will 
be available at no charge on the GAO Web site at 
[Hyperlink, http://www.gao.gov].

Should you or your offices have questions on matters discussed in this 
report, please contact me at (202) 512-9286 or Lester P. Diamond, 
Assistant Director, at (202) 512-7957. We can also be reached by e-mail 
at [Hyperlink, pownerd@gao.gov], or [Hyperlink, diamondl@gao.gov], 
respectively. Key contributors to this report were William G. Barrick, 
Niti Bery, Joanne Fiorino, Michael Giannone, Sabine R. Paul, and Nik 
Rapelje.

Signed by: 

David A. Powner: 
Director, IT Management Issues: 

[End of section]

Appendixes: 

Appendix I: Objectives, Scope, and Methodology: 

The objectives of our review were to (1) evaluate FAA's capabilities 
for managing its IT investments, (2) determine what plans the agency 
might have for improving these capabilities, and (3) describe how DOT 
oversees FAA's investments and investment process. Because FAA told us 
that it managed its NAS and non-NAS investments differently, we 
performed separate assessments for the practices to evaluate FAA's 
capabilities for managing IT investments.

To address the first objective, for the NAS investments we reviewed the 
results of the agency's self-assessment of Stages 2, 3, and 4 practices 
using GAO's ITIM framework[Footnote 22] and validated and updated the 
results of the self-assessment through document reviews and interviews 
with officials. We reviewed written policies, procedures, and guidance 
and other documentation providing evidence of executed practices, 
including FAA's Acquisition Management System guidance, mission 
analysis and investment analysis guidance, and memorandums. We also 
reviewed JRC guidance and records of decision, acquisition review 
guidance and meeting minutes, and variance reporting procedures and 
reports. We did not assess FAA's progress in establishing the 
capabilities found in one of the two Stage 4 critical processes, 
entitled Improving the Portfolio's Performance, or in any of the Stage 
5 critical processes, because FAA acknowledged that it had not executed 
any of the key practices in these critical processes. For the non-NAS 
investments, we reviewed the results of FAA's self-assessments of 
Stage 2 practices using GAO's ITIM framework and conducted interviews 
to clarify and update the results. We did not perform a detailed 
assessment of these practices because they most likely will be 
superseded by a new process (when it is defined) for managing non-NAS 
investments, and non-NAS investments are of lower cost and impact to 
FAA.

As part of our analysis, we selected three IT projects as case studies 
to verify that the critical processes and key practices were being 
applied. We selected projects that (1) supported different FAA 
functional areas, (2) were in different life cycle phases, and 
(3) required different levels of funding. The three projects are 
described below: 

* FAA Telecommunications Infrastructure (FTI)--FTI is a performance-
based telecommunications services contract for voice, video, and data 
point-to-point support for telecommunications for the National Airspace 
System and its support system. It contributes to both the separation of 
aircraft (the mission-support network) and other FAA uses (the 
operational network, e.g., e-mail and phone). FTI will replace the 
current telecom system. FTI will eliminate the need for other 
subnetworks, of which there are currently eight or nine, and therefore 
eliminate the management overhead associated with operating so many 
networks. The integration of multiple networks and subnetworks will 
provide a single source and single vehicle for telecom. FTI is in the 
Technical Operations unit and has estimated life cycle costs of 
$2 billion. The contract for FTI was awarded in June 2002.

* En Route Communications Gateway (ECG)--ECG is a mission critical 
gateway, or interface, for data from radar sites to Air Route Traffic 
Control Centers. ECG will serve as a single domain communications 
gateway and will provide the path for exchanging flight plan data from 
outside sources and transfer data among systems. ECG provides a 
commercial-off-the-shelf nondevelopmental item digital gateway using a 
modern, open and extensible platform consisting of modular scalable 
hardware components. ECG will incorporate interface capability to 
support legacy and future systems and will provide the capability to 
transition to modern network communications and access more 
surveillance sources. The flexibility provided by the ECG system 
architecture will facilitate the evolution of the En Route domain 
modernization. ECG will replace the Peripheral Adapter Module 
Replacement Item system and provide a modern domain gateway that will 
support the current and future En Route infrastructure. ECG is in the 
En Route & Oceanic Service group and has estimated life cycle costs of 
$442.5 million through September 2015.

* Voice Switching and Control System (VSCS)--In our review of the VSCS 
program, we focused our review on one of VSCS's subcomponents, the VSCS 
Control Subsystem Upgrade (VCSU). The VCSU program, part of the 
Technical Operations Communications service group, is designed to 
maintain overall supportability of VSCS[Footnote 23] by replacing the 
hardware for the existing control subsystem, associated VSCS 
operational and application software, required software licenses, and 
supporting software and hardware documentation. Deliverables for the 
VCSU program include all hardware, spare parts, software, software 
licenses, system baseline documentation, training, and other technical 
documentation necessary to support the product at 21 locations. 
According to FAA, the VCSU program has a funding baseline of over $59 
million and is in the operations and maintenance phase.

For these projects, we reviewed project management documentation, such 
as mission needs statements, acquisition program baselines, and 
integrated program plans. We also interviewed the project managers for 
these projects.

We compared the evidence collected from our document reviews and 
interviews to the key practices in ITIM. We rated the key practices as 
"executed" on the basis of whether the agency demonstrated (by 
providing evidence of performance) that it had met the criteria of the 
key practice. A key practice was rated as "not executed" when we found 
insufficient evidence of a practice during the review or when we 
determined that there were significant weaknesses in FAA's execution of 
the key practice.

To address our second objective, we obtained and evaluated documents 
showing what management actions had been taken and what initiatives had 
been planned by the agency. This documentation included JRC records of 
decisions, the agency's capital investment guidance, and the recently 
formed ITEB charter and meeting minutes. We also interviewed the Chief 
Information Officer, other members of the JRC, and the Chief Operating 
Officer to determine what efforts FAA had undertaken to improve IT 
investment management processes.

To address our third objective, we reviewed documentation on DOT's 
process for reviewing FAA's budget proposals and capital planning and 
investment control reviews. We also conducted interviews with both FAA 
and DOT officials, including DOT's CIO and Director for Capital 
Planning and Investment Control to determine DOT's oversight role in 
FAA's investments and investment management processes.

We conducted our work at FAA Headquarters in Washington, D.C., from 
October 2003 through July 2004, in accordance with generally accepted 
government auditing standards.

[End of section]

Appendix II: Investment Management Process Used by Some Organizational 
Units to Manage Non-NAS Investments: 

Financial Services (ABA):

Instituting the investment board; ABA has an investment board that 
conducts periodic and monthly program reviews for all IT programs to 
determine whether a program will be approved as an IT investment. A 
life cycle process guide is now in place to direct the activities of 
the investment board along with providing oversight of IT projects 
within ABA.

Meeting business needs; The business needs of a project within ABA, 
along with the dates for achieving them, need to be aligned with the 
strategic goals established in the FAA Flight Plan. Projects or systems 
that are no longer aligned with the Flight Plan will be decommissioned; 
A project management plan identifies, among other things, the system's 
users, customers, and types of services to be provided.

Selecting an investment; Selecting and reselecting an IT investment 
within ABA involves both the executive management team and ABA's CIO 
team. The executive management team reviews the business needs of the 
investment and compares them against the ABA's IT budget, while ABA's 
CIO team is involved with the selecting and reselecting processes by 
analyzing the technical costs associated with the IT investment and 
comparing those technical costs against the ABA's IT budget.

Providing investment oversight; ABA uses its life cycle process guide 
to help manage its $25 million IT budget, which consists of 22 or 23 
financial systems, 5 or 6 of them considered major programs under OMB's 
definition of a major IT investment. A requirement of the life cycle 
process guide is for every critical system in ABA to have a detailed 
project management plan that addresses performance measures such as 
cost, schedule, benefits, and risks; The day-to-day progress of IT 
projects is tracked against critical milestones that have been already 
established through weekly summary reviews with IT staff. For major IT 
projects, biweekly meetings are conducted that address any concerns 
with meeting the performance measures.

Capturing the investment information; ABA captures its IT asset 
information using its Information Technology Investment Portfolio 
System (ITIPS), which is available to all ABA management and system 
support personnel. The information in ITIPS is used to manage projects 
that are in production as well as ensuring that the life cycle 
activities are in alignment with FAA's mission statements.

Research and Acquisitions (ARA):

Instituting the investment board; ARA uses its Operations Resource 
Management Team guide to select, control, and evaluate ARA IT 
investments. The team composed of representatives from ARA service 
units. ARA investments are controlled and tracked through quarterly 
reviews. These reviews look at the cost, schedule, and overall 
performance of the investment.

Meeting business needs; The business needs for ARA investments need to 
be mapped back to the Flight Plan. A monthly status review report is 
prepared in order to ensure that the business needs are tracking back 
to the Flight Plan.

Selecting an investment; ARA does not have any well-defined selection 
criteria since each program uses its own configuration management plan. 
ARA Ops build process guides the establishment of new projects.

Providing investment oversight; A project plan does exist, along with 
established expenditures, which the program managers submit to the ARA 
CIO on a monthly basis. These monthly status reports occur between the 
CIO and the program managers to decide if an investment's resources, 
such as funding, need to be reallocated. Once the CIO and program 
manager decide that it is necessary for an investment's resources to be 
reallocated, the CIO will discuss the need further with the Deputy 
Associate Administrator for ARA, who ultimately will determine whether 
a program will receive additional resources, such as funding; With 
respect to the level of interaction that ARA has had with the JRC in 
the past, only one program from ARA, NextGen, has gone before the JRC. 
According to the ARA CIO, in order for a program to go to the JRC, 
there must be justification made to the council that the program is 
fully operational and is considered to be a benefit and a priority to 
FAA. The ARA Deputy Associate Administrator will determine if a program 
should go before the JRC for approval and funding.

Capturing investment information; The configuration control board uses 
a database to capture asset inventory data about the systems that are 
owned by the ARA CIO. According to the ARA CIO, in order for IT assets 
to be effectively managed in ARA, there needs to be vision from AIO 
about what programs to invest in over the next 5 years.

Air Traffic Services (ATS):

Instituting the investment board; The Information Resource Management 
Executive Board is responsible for selecting, controlling, and 
evaluating ATS IT investments.

Meeting business needs; Not all services within ATS have defined their 
business needs. Even though ATS has the NAS Support Integration Process 
(NSIP) data repository available for capturing IT asset information, 
including business needs, and for defining system users, there is no 
consistency in terms of the records being complete because there are 
systems within ATS that have not registered with NSIP.

Selecting an investment; The ATS CIO manages the selection process, 
which begins with the NSIP registration criteria.

Providing investment oversight; Each business unit within ATS has its 
own project management plan and procedures. The day-to-day tracking of 
projects as well as the monitoring of whether corrective actions are 
being executed is also the responsibility of the individual business 
units. Even though the individual business units are tasked with this 
level of responsibility, the ATS CIO does play an oversight role by 
setting the criteria and policies for the investments to be made for 
the projects.

Capturing investment information; ATS uses the NSIP meta data 
repository to collect any changes to the IT projects and systems by 
providing a full declaration of the project or system. This includes 
providing information to help ATS avoid unwanted costs due to systems 
having redundant functionality and determining whether a system's or a 
project's functions match the stated mission goals for ATS. NSIP also 
handles the technical rollover for ATS systems or projects.

Information Services (AIO):

Instituting the investment board; AIO's investment management process 
can be characterized as iterative and well managed, but undocumented. 
The AIO Business Plan and IT Strategy are used to ensure that when 
funds are appropriated and allocated that they map back to the Flight 
Plan. Investments are controlled or tracked by the Deputy CIO on a 
monthly basis to get an indication as to where the program is in the 
process against the expenditures that have been already established. 
Weekly meetings are held with the unit's CIO to discuss any issues 
regarding AIO's investment management process.

Meeting business needs; AIO does not have any written policies or 
procedures for identifying business needs for its IT projects. Only one 
of its major projects, NAS Adaptation Service and Environment, has 
documented its requirements, which includes specific users.

Selecting an investment; AIO uses an undocumented process for reviewing 
new IT proposals to reach an agreement on selection.

Providing investment oversight; There are no AIO-wide policies or 
procedures for managing projects or investment oversight. The 
Information Technology Executive Board (ITEB)[A] has been formed to 
provide a governing structure for non-NAS programs. One of the targets 
for ITEB is to look at cost control and cross-cutting IT initiatives by 
involving the heads of the lines of business. The ITEB is also going to 
be involved with improving the scores on the Exhibit 300 business cases 
for OMB.

Capturing the investment information; AIO uses ITIPS to track its asset 
inventory and IT investments. The Deputy CIO of AIO is responsible for 
ensuring that the inventory located in ITIPS meets the needs of AIO's 
investment management process. According to AIO, the information within 
ITIPS is updated at least twice a year.

Human Resource Management (AHR):

Instituting the investment board; AHR does not have an investment 
board. Instead, AHR's senior management[B] is responsible for 
selecting, controlling, and evaluating all IT investments by using 
established agency acquisition policies and procedures to conduct 
investment management decisions.

Meeting business needs; Business needs and specific users for each 
project are identified within the project plan and are aligned with the 
AHR Strategic Plan, the FAA Flight Plan, and the AIO Plan. AHR is also 
aligning its business needs to the ITEB plans. Business needs are re-
evaluated on a quarterly basis to ensure that a project is aligned with 
FAA's strategic goals and objectives.

Selecting an investment; AHR senior management uses its prioritization 
process to evaluate and select investments for funding. The office and 
center directors determine their requirements and then a budget request 
is submitted for proposal funding. AHR receives an allowance amount 
from the budget office. The first priority is to handle personnel 
payments. The remaining balance is then redistributed to the business 
divisions. The "building blocks" process starts at this point. This is 
when base funding is reviewed to decide if a current investment needs 
continued funding by asking questions about the importance of 
continuing the funding of a particular project by looking at the 
project activity and what the impact will be if this project is no 
longer funded. Each division will submit a list of prioritized projects 
with costs to the directorate. This list may exceed the budget level. 
The directorate will reprioritize the original list.

Providing investment oversight; AHR has a Human Resource Management 
Automation Plan that contains procedures for approving IT projects, and 
describes the policies and procedures that AHR uses for project 
management. Despite having project management policies and procedures, 
not all projects within AHR have a formal project plan. The size and 
scope of the project are two factors that help determine whether a 
project has a formal project plan. AHR Division Managers ensure that 
projects are on time by performing quarterly reviews that assess a 
project's cost and schedule. AHR uses a color scheme (red, green, and 
yellow) to indicate the schedule status of major milestones.

Capturing the investment information; AHR uses the ITIPS as its 
inventory for making investment management decisions. AHR projects are 
listed in ITIPS, along with business cases.

Regions and Center Operations (ARC):

Instituting the investment board; The IT Configuration Management Board 
is ARC's investment review board. The board's charter has recently been 
redone to provide more traceability back to the Flight Plan. The board 
functions include evaluating potential IT investment options for ARC, 
making recommendations on IT investment, establishing ARC-wide IT 
standards, and developing and maintaining investment policies and 
procedures. The board is led by the unit's CIO and includes four IT 
managers from the regional offices and aeronautical center and two 
members from the ARC Management Team. The ARC Management Team makes the 
final-selection decisions. The IT investment management decisions are 
then incorporated into the ARC Business Plan. The ARC unit is also 
involved with cross-organizational investment decisions for FAA through 
its membership on the FAA CIO Council.

Meeting business needs; Business needs are identified through entries 
made in ITIPS, along with documentation from Exhibit 300s and Exhibit 
53s.

Selecting an investment; ARC does not have its selection criteria 
documented. To evaluate and select IT investments, the ARC IT 
Configuration Management Board considers such things as benefits to ARC 
across the regions, expected return on investment, technical 
feasibility, and risk. The ARC business plan and the Flight Plan are 
the documents that address these priorities.

Providing investment oversight; ARC does not have policies or 
procedures for project management. Instead, ARC uses a weekly 
teleconference to address expectations and progress of ARC-wide IT 
initiatives at the IT manager level across ARC. According to the ARC 
CIO, a second teleconference has been added to discuss portfolio 
management--schedule, budget, training, and deployment along with 
whether the project will be integrated with other lines of business.

Capturing the investment information; ARC uses ITIPS as its 
standardized repository for collecting asset information that will be 
useful for ARC's IT investment management decisions by providing 
information about what types of systems and functions are available and 
how they are supporting a specific business issue.

Regulation and Certification (AVR):

Instituting the investment board; Similar to an IT investment board, 
AVR has a two-tiered management structure that is composed of the AVR 
management team and the CIO management team. The AVR management team 
includes the Associate Administrator and the Service Directors who make 
the final decisions based upon recommendations and input from the CIO 
management team and its business partners from each of the service 
units. According to AVR, its IT investment process guide is still under 
development and will be completed at the end of Fiscal Year 2004.

Meeting business needs; Each line of business within AVR identifies and 
documents its business needs including project requirements and 
specific users. Once the business needs have been identified, the IT 
Management and Resources section prioritizes them for funding.

Selecting an investment; Programs in AVR are reviewed quarterly. For 
major projects, meetings are designed to look at project milestones to 
see if they are being met. These meetings are carried out biweekly and 
presented to the AVR management team.

Providing investment oversight; The AVR CIO management team is 
responsible for monitoring projects and reporting to the AVR Management 
team. Biweekly meetings are held for major projects within AVR.

Capturing the investment information; AVR's system inventory is a part 
of its enterprise architecture. The system inventory is being used 
primarily in developing the Exhibit 300s. The performance of IT 
projects in AVR is monitored daily, based upon each project's 
individual plan, using project management tools such as MS Project. 
According to AVR, not all projects have a project plan in place, but 
AVR is trying to make it a requirement. 

Source: GAO, based on information from FAA.

[A] ITEB is a board that can provide a governing structure so that 
information technology is used as an agency wide strategic asset.

[B] Composed of Assistant Administrator; two Deputy Assistant 
Administrators; three Office Directors; the Director, Center for 
Management and Development; and the AHR Business Officer.

[End of table]  

(310456): 

FOOTNOTES

[1] See GAO, High-Risk Series: An Overview, GAO/HR-95-1 (Washington, 
D.C.: February 1995); GAO, High-Risk Series: Information Management and 
Technology, GAO/HR-97-9 (Washington, D.C.: February 1997); GAO, High-
Risk Series: An Update, GAO/HR-99-1 (Washington, D.C.: January 1999); 
GAO, High-Risk Series: An Update, GAO-01-263 (Washington, D.C.: January 
2001); and GAO, High-Risk Series: An Update, GAO-03-119 (Washington, 
D.C.: January 2003).

[2] GAO, Information Technology Investment Management: A Framework for 
Assessing and Improving Process Maturity, GAO-04-394G (Washington, 
D.C.: March 2004).

[3] The 10 service units that make up the ATO include Safety, 
Communications, Operations Planning, Finance, Acquisition & Business 
Services, En Route and Oceanic Services, Terminal Services, Flight 
Services, System Operations Services, and Technical Operations 
Services.

[4] FAA uses three types of facilities to control traffic: airport 
towers, terminal radar approach control facilities, and en route 
centers. Airport towers direct traffic on the ground, before landing, 
and after takeoff within 5 nautical miles from the airport and about 
3,000 feet above the airport. Terminal radar approach control 
facilities sequence and separate aircraft as they approach and leave 
airports, beginning about 5 nautical miles and ending about 50 nautical 
miles from the airport and generally up to 10,000 feet above the 
ground. Air route traffic control centers, called en route centers, 
control planes in transit and during approaches to some airports, 
generally controlling air space that extends above 18,000 feet for 
commercial aircraft.

[5] GAO-03-119.

[6] GAO, Air Traffic Control: FAA's Modernization Investment Management 
Approach Could Be Strengthened, GAO/RCED/AIMD-99-88 (Washington, D.C.: 
Apr. 30, 1999); GAO, Air Traffic Control: FAA's Modernization Efforts-
-Past, Present, and Future, GAO-04-227T (Washington, D.C.: Oct. 30, 
2003).

[7] 49 U.S.C. 40110(d).

[8] May be delegated to an associate administrator.

[9] These teams may operate as entities or be organized into 
subintegrated product teams or product teams to develop, procure, and 
deliver products and services for users or customers. They are 
responsible for the acquisition of new or improved capability for 
services and products throughout their life cycles and for developing 
cost and schedule baselines for candidate solutions during investment 
analysis. 

[10] The investment analysis staff assists and oversees the work of all 
the investment analysis teams, is responsible for all investment 
analyses, and is responsible for developing the tools, techniques, and 
databases to ensure quality performance of investment analysis on 
behalf of the JRC.

[11] A service organization is any organization within FAA that 
delivers a service, whether it is a business unit, project office, 
program directorate, or integrated product team, or whether it is 
engaged in air traffic services, security, regulation, certification, 
operations, commercial space transportation, or airport development. 
These organizations are responsible and accountable for managing 
service delivery throughout the life cycle. Investment decisions are 
made to support service delivery. Specifically, after the investment 
decision has been made, the service organization assumes responsibility 
for the investment program, implements the selected solution, and 
manages the product throughout the in-service management phase of its 
life cycle.

[12] Non-NAS business units include Information Services (AIO), Region 
and Center Operations (ARC), Regulation and Certification (AVR), 
Financial Services (ABA), Research and Acquisition (ARA), Air Traffic 
Services (ATS), and Human Resource Management (AHR). 

[13] GAO-04-394G.

[14] GAO, Information Technology: DLA Needs to Strengthen Its 
Investment Management Capability, GAO-02-314 (Washington, D.C.: Mar. 
15, 2002); GAO, United States Postal Service: Opportunities to 
Strengthen IT Investment Management Capabilities, GAO-03-3 
(Washington, D.C.: Oct. 15, 2002); GAO, Information Technology: 
Departmental Leadership Crucial to Success of Investment Reforms at 
Interior, GAO-03-1028 (Washington, D.C.: Sept. 12, 2003); and GAO, 
Bureau of Land Management: Plan Needed to Sustain Progress in 
Establishing IT Investment Management Capabilities, GAO-03-1025 
(Washington, D.C.: Sept. 12, 2003).

[15] An IT investment board is a decision-making body, made up of 
senior program, financial, and information managers, that is 
responsible for making decisions about IT projects and systems based on 
comparisons and trade-offs among competing projects, with an emphasis 
on meeting mission goals.

[16] 49 U.S.C. 40110(d).

[17] An advisory group, composed of representatives from each line of 
business, that establishes guidelines for conducting mission analysis 
and developing mission need statements as well as resolving agencywide 
mission analysis issues. 

[18] We reviewed the FAA Telecommunications Infrastructure, En Route 
Communications Gateway, and Voice Switching and Control System (VSCS) 
Control Subsystem Upgrade (VCSU) projects. The projects are described 
in appendix I.

[19] VCSU is a subcomponent of the VSCS project. We decided to review 
VCSU because its investment management process was carried out using 
FAA's AMS, whereas the VSCS project was funded before the AMS became 
part of the FAA's investment management process.

[20] According to FAA, no acquisition program baseline was prepared for 
VCSU.

[21] GAO/RCED/AIMD-99-88.

[22] GAO-04-394G.

[23] VSCS is FAA's highly distributed, computer-controlled 
communications and control system for U.S. air traffic management that 
allows air traffic controllers to establish all air-to-ground and 
ground-to-ground communications with pilots and other air traffic 
controllers.

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading.

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548: 

To order by Phone: 



Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: