This is the accessible text file for GAO report number GAO-05-242 
entitled 'NASA's Space Vision: Business Case for Prometheus 1 Needed to 
Ensure Requirements Match Available Resources' which was released on 
March 23, 2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters:

United States Government Accountability Office:

GAO:

February 2005:

NASA's Space Vision:

Business Case for Prometheus 1 Needed to Ensure Requirements Match 
Available Resources:

GAO-05-242:

GAO Highlights:

Highlights of GAO-05-242, a report to congressional requesters

Why GAO Did This Study:

In 2003, the National Aeronautics and Space Administration (NASA) 
initiated the Prometheus 1 project to explore the outer reaches of the 
Solar System. The Prometheus 1 spacecraft is being designed to harness 
nuclear energy that will increase available electrical power from about 
1,000 watts to over 100,000 watts and enable the use of electric 
propulsion thrusters. 

Historically, NASA has had difficulty implementing some initiatives. 
NASA’s failure to adequately define requirements and quantify the 
resources needed to meet those requirements has resulted in some 
projects costing more, taking longer, and achieving less than 
originally planned. Prometheus 1 will need to compete for NASA 
resources with other space missions—including efforts to return the 
shuttle safely to flight and complete the International Space Station. 

GAO was asked to determine (1) whether NASA is establishing initial 
justification for its investment in the Prometheus 1 project and (2) 
how the agency plans to ensure that critical technologies will be 
sufficiently mature at key milestones. 

What GAO Found:

NASA is in the process of establishing initial justification for its 
investment in the Prometheus 1 project but faces challenges 
establishing preliminary requirements and developing accurate cost 
estimates. Decision makers will not get their first comprehensive 
picture of the project’s requirements and the resources needed to meet 
those requirements until the preliminary mission and systems review, 
scheduled for summer 2005. Defining the project’s requirements and 
developing life-cycle cost estimates by then could be challenging, 
given the short time frames. The fidelity of this information should 
improve by the preliminary design review scheduled for 2008. At that 
time, NASA has the opportunity to use these more refined requirements 
and cost estimates to establish a sound business case for its 
investment in the Prometheus 1 project. According to Prometheus 1 
project management, a flat funding profile is inadequate to ramp up for 
the planned 2015 launch of Prometheus 1, the project’s first spacecraft 
to its original destination of Jupiter’s Icy Moons. By matching 
requirements to resources a sound business case would allow NASA to 
determine whether trade-offs in the design of the spacecraft or the 
agency’s expectations are needed to avoid outstripping available 
resources. Significant program cost and schedule increases in past 
programs can be traced to not matching requirements with resources at 
preliminary design review. 

While development of the Prometheus 1 technologies is under way, each 
will require extensive advancement before they are mature enough to 
support reliable cost estimates. NASA is preparing technology 
development plans that include measurable criteria to ensure the 
Prometheus 1 technologies are on track for meeting NASA’s maturity 
requirements through the end of the preliminary design phase. 

GAO’s best practices work has shown, however, that establishing a 
formal business case based on a knowledge-based approach that includes 
matching requirements and available resources—which include technical 
and engineering knowledge, time, and funding—and controls to ensure 
that sufficient knowledge has been attained at critical junctures 
within the product development process is an essential part of any 
product development justification. NASA’s current policy does not 
require projects to develop knowledge-based business cases that match 
requirements to available resources and include controls to ensure that 
sufficient knowledge has been attained. Therefore, the agency had not 
planned to develop such a business case for Prometheus 1. 

Since GAO provided our draft report to NASA for comment, the agency 
released its fiscal year 2006 budget request that includes changes to 
Prometheus 1. If properly implemented, these changes could be positive 
steps in addressing the findings and recommendations in this report. 

What GAO Recommends:

GAO is making recommendations aimed at ensuring that NASA prepares a 
sound business case for Prometheus 1. NASA concurred with our 
recommendations. 

www.gao.gov/cgi-bin/getrpt?GAO-05-242. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Allen Li at (202) 512-
3600 or Lia@gao.gov. 

[End of section]

Contents:

Letter:

Results in Brief:

Background:

Initial Justification for Prometheus 1 Project Could Form the Basis of 
a Sound Business Case:

NASA's Plans to Ensure Mature Technologies Rely on Prototype 
Demonstrations:

Conclusions:

Recommendations for Executive Action:

Agency Comments:

Scope and Methodology:

Appendix I: Technology Readiness Levels Definitions:

Appendix II: Prometheus 1 Critical Technologies:

Nuclear Reactor:

Power Conversion:

Heat Rejection:

Nuclear Electric Propulsion:

Radiation Hardening:

High Power Communications:

Autonomous Rendezvous and Docking:

Appendix III: Comments from the National Aeronautics and Space 
Administration:

Appendix IV: GAO Contact and Staff Acknowledgments:

GAO Contact:

Acknowledgments:

GAO Related Products:

Figures:

Figure 1: Prometheus 1 Spacecraft:

Figure 2: NASA Prometheus 1 Fiscal Year 2005 Budget Request Profile:

Figure 3: Prometheus 1 Milestone Reviews:

Figure 4: Technology Maturity Levels for Product Development:

Abbreviations:

AR&D: autonomous rendezvous and docking:

JIMO: Jupiter Icy Moons Orbiter:

JPL: Jet Propulsion Laboratory:

MCT: maturity criteria tables:

NASA: National Aeronautics and Space Administration:

PDR: preliminary design review:

PMSR: preliminary mission and systems review:

SLI: Space Launch Initiative:

TRL: technology readiness levels:

United States Government Accountability Office:

Washington, DC 20548:

February 28, 2005:

The Honorable Daniel K. Inouye: 
Co-Chairman: 
Committee on Commerce, Science, and Transportation: 
United States Senate:

The Honorable John McCain: 
United States Senate:

In 2003, the National Aeronautics and Space Administration (NASA) 
initiated the Prometheus 1 project to explore the outer reaches of the 
Solar System in the hopes of finding answers to some of humankind's 
most profound questions about life and its origins. The Prometheus 1 
spacecraft is being designed to harness nuclear energy that will 
efficiently increase available electrical power from about 1,000 watts 
to over 100,000 watts and enable the use of electric propulsion 
thrusters. The availability of nuclear power at this magnitude will 
fundamentally change NASA's capability to explore deep space. 

While NASA has been successful in missions such as Mars Pathfinder and 
Exploration rovers, the agency has had difficulty implementing a number 
of other costly initiatives because it was overly optimistic in what 
could be achieved within available resources. NASA's failure to 
adequately define requirements and quantify the resources needed to 
meet those requirements has resulted in some projects costing more, 
taking longer, and achieving less than originally planned. Prometheus 1 
will compete for NASA resources with other space missions--including, 
in the near term, efforts to return the shuttle safely to flight and 
completing the International Space Station. 

Cognizant of the outlook for a constrained federal budget for the 
foreseeable future and NASA's difficulty in implementing some major 
programs within projected resources, you asked us to review the 
Prometheus 1 project to determine (1) whether NASA is establishing 
initial justification for its investment in the Prometheus 1 project 
and (2) how the agency plans to ensure that critical technologies will 
be sufficiently mature at key milestones. 

To address our objectives, we obtained and reviewed Prometheus 1 plans, 
schedules, risk assessments, budget documentation, technology maturity 
assessments, and technology development plans. We conducted further 
qualitative and quantitative analyses of these documents and compared 
them to criteria established in NASA and Jet Propulsion Laboratory 
(JPL) policies governing development programs and in GAO's best 
practices body of work (see GAO Related Products). Our work was 
conducted between April 2004 and January 2005 in accordance with 
generally accepted government auditing standards. 

Since our draft report was provided to NASA for comment, a significant 
change has been made to the Prometheus 1 project. NASA's fiscal year 
2006 budget request includes changes to the Prometheus 1 project that 
directly address the findings and recommendations of this report. These 
changes are briefly outlined in the Agency Comments section of this 
report. 

Results in Brief:

NASA is in the process of establishing initial justification for its 
investment in the Prometheus 1 project but faces challenges preparing 
preliminary requirements and cost estimates. NASA wants to have a 
defined set of preliminary system requirements and an initial estimate 
of the life-cycle cost for Prometheus 1 by summer 2005--when the 
project enters into the preliminary design phase. While the project 
office has drafted cost guidelines and a technical baseline for use in 
developing an initial life-cycle cost estimate, due to the early nature 
of the project, the development of this estimate is just under way. 
Before Prometheus 1 can move forward into the preliminary design phase, 
NASA must make funding decisions. At this time, however, the level of 
funding NASA needs to execute the project is not fully defined. NASA 
requested $438 million for fiscal year 2005, but according to project 
officials, the "flat line" estimate for the next few years would need 
to be increased to reflect project needs of a mission to Jupiter's Icy 
Moons. A business case providing an understanding of the potential 
return on such investment would be helpful to decision makers for 
determining whether continued investment--currently estimated by the 
Congressional Budget Office at about $10 billio[Footnote 1]n--is 
warranted. Developing a sound business case that includes matching 
requirements with expected resources, would enable NASA to make early 
trade-offs either in the preliminary design of the spacecraft or in the 
agency's expectations to avoid outstripping available resources. While 
NASA will have information available to match preliminary requirements 
to expected resources, the fidelity of this information is not expected 
to be completely defined until the preliminary design review (PDR), 
currently scheduled for 2008. 

NASA plans to ensure critical technologies are mature by demonstrating 
subsystem prototypes before the end of the preliminary design phase. 
There are several technologies, however, whose maturity will influence 
NASA's ability to develop initial requirements and reliable resource 
estimates. The Prometheus 1 design conceived for the mission to 
Jupiter's Icy Moons relied on advancements in several breakthrough 
technologies, including nuclear electric power and propulsion, high 
power communications, radiation-hardened electronics, and autonomous 
rendezvous and docking (AR&D). NASA is preparing technology development 
plans that include measurable criteria to ensure each technology is on 
track for meeting the maturity requirements through the end of the 
preliminary design phase. While development of these technologies is 
under way, each will require extensive advancement before they are 
mature enough to provide the revolutionary capabilities of the 
Prometheus 1 spacecraft. 

Our past work on the best practices of product developers in government 
and industry has found that development of a sound business case based 
on matching requirements to resources is a key factor in successfully 
addressing such challenges. Despite the fact that the Prometheus 1 
project is in the very early stages of development, the use of a sound 
business case that includes well-defined requirements, realistic cost 
estimates, and mature technology is an essential part of any product 
development investment justification. NASA's current policy does not 
require projects to develop formal knowledge-based business cases that 
match requirements to available resources and include controls to 
ensure that sufficient knowledge has been attained. NASA had not 
planned to develop such a business case. Subsequent to our draft being 
provided to NASA for comment, the agency announced in its fiscal year 
2006 budget request that it was conducting an analysis of alternatives 
to identify a new mission with reduced technical, schedule, and 
operational risk. As NASA will be establishing a new justification for 
the Prometheus 1 project, we are recommending that the NASA 
Administrator establish a sound business case for the Prometheus 1 
project wherein resources are matched to requirements and controls are 
in place to ensure that sufficient knowledge has been attained at 
critical phases of the product development process. 

Background:

The Prometheus 1 project is part of NASA's Prometheus Nuclear Systems 
and Technology program[Footnote 2] to develop nuclear power 
technologies capable of providing power and propulsion for a new 
generation of missions. The Prometheus 1 spacecraft is being designed 
to use nuclear power and electric propulsion technologies to explore 
the outer reaches of the solar system. The Jupiter Icy Moons Orbiter 
(JIMO) mission--a 4 to 6-year study of three of Jupiter's moons: 
Callisto, Europa, and Ganymede--was the original destination identified 
by NASA.[Footnote 3] The JIMO mission's overarching science objectives 
were to (1) investigate the origin and evolution of the three moons; 
(2) scout their potential for sustaining life; and (3) determine the 
current rate of movement of surface ice and the rates at which the 
moons are weathered. With an unprecedented level of power, Prometheus 
1, the first in a potential series of spacecraft, is expected to 
support the use of high capability science instruments and high power 
communications systems to provide scientists with an a unprecedented 
amount of scientific information. Figure 1 depicts the notional 
Prometheus 1 spacecraft. 

Figure 1: Prometheus 1 Spacecraft:

[See PDF for image]

[End of figure]

NASA contracted with the Jet Propulsion Laboratory (JPL) to manage the 
Prometheus 1 project and to manage development of the science mission 
payload. In turn, JPL awarded a $400-million contract for the initial 
development of the Prometheus 1 spacecraft to Northrop Grumman Space 
Technology in September 2004. NASA is collaborating with the Department 
of Energy's Office of Naval Reactors to develop and handle all issues 
related to the spacecraft's nuclear reactor. 

The Prometheus 1 project will have to compete for funding with other 
NASA programs. In January 2004, the President charged NASA with 
implementing a new strategy for space exploration--which includes the 
Prometheus 1 project--while simultaneously returning the shuttle to 
flight status and completing the International Space Station. NASA laid 
out its plan for implementing the strategy in its fiscal year 2005 
budget request. In essence, NASA's implementation plan holds 
aeronautics, science, and other activities at near constant levels and 
transitions funding levels currently dedicated to the Space Station and 
shuttle programs to the space exploration strategy as the Space Station 
and shuttle programs phase out. This plan was predicated upon NASA's 
annual funding level receiving increases to about $18 billion a year by 
fiscal year 2008 and then remaining near that level, except for 
inflation, through at least 2020. 

Best Practices Reveal Elements of a Sound Business Case for Product 
Development:

In the last several years, we have undertaken a best practices body of 
work on how leading developers in industry and government use a 
knowledge-based approach to develop products that reduces risks and 
increases the likelihood of successful outcomes. Development of a sound 
business case based on this best practices model enables decision 
makers to be reasonably certain about their products at critical 
junctures during development and helps them make informed investment 
decisions. 

Our best practice work has shown that developing a sound business case 
based on matching requirements to resources is essential to 
implementing a knowledge-based approach. A sound business case includes 
the following elements:

* well-defined requirements,

* preliminary design,

* realistic cost estimates, and:

* mature technology. 

A knowledge-based business case also involves the use of controls or 
exit criteria to ensure that the required knowledge has been attained 
at each critical juncture. It ensures that managers will (1) conduct 
activities to capture relevant product development knowledge, (2) 
provide evidence that knowledge was captured, and (3) hold decision 
reviews to determine that appropriate knowledge was captured to allow a 
move to the next phase. If the knowledge attained at each juncture does 
not confirm the business case on which the effort was originally 
justified, the program does not go forward. 

Use of this approach has enabled leading organizations to deliver high 
quality products on time and within budget. Product development efforts 
that have not followed a knowledge-based business case approach can be 
frequently characterized by poor cost, schedule, and performance 
outcomes. 

Although NASA does not require projects to develop a formal business 
case based on matching requirements to resources, JPL project 
implementation policy,[Footnote 4] which establishes JPL's 
institutional structure for implementation and management of JPL flight 
projects in accordance with NASA policies, does require projects to 
develop documentation that includes elements essential to a sound 
business case. For example, before entering the preliminary design 
phase, JPL projects are required to develop preliminary requirements, a 
conceptual design, realistic cost estimates, and technology development 
plans. JPL projects are required to update and improve the fidelity of 
information in these documents by PDR. The information in these 
documents could provide NASA decision makers with the information 
necessary to support sound business case decisions based on matching 
requirements to resources at preliminary mission and systems review 
(PMSR) and PDR. 

In September 2004, the Congressional Budget Office reported that if 
NASA's costs for implementing the strategy were similar to prior 
analogous NASA programs--such as Apollo, Viking, and Mars Exploration 
Rover--NASA's funding needs could increase by 15 to 23 percent--or $40 
billion to $61 billion--over the 16-year estimate. The Congressional 
Budget Office concluded that if funding were held constant, NASA would 
likely have to either eliminate mission content or delay schedules. 

Initial Justification for Prometheus 1 Project Could Form the Basis of 
a Sound Business Case:

NASA is still in the process of preparing initial justification for the 
Prometheus 1 project to enter the preliminary design phase. 
Consequently, at this time the level of funding NASA needs to execute 
the project is not fully defined. According to project officials, 
however, funding levels would need to be increased to support the 
planned launch of Prometheus 1 to Jupiter's Icy Moons. While NASA plans 
to have defined preliminary system requirements and an initial estimate 
of the life-cycle cost for Prometheus 1 by summer 2005--when the 
project enters the preliminary design phase--the agency faces 
significant challenges in doing so. 

Project Management Believes Current Funding Is Insufficient to Support 
Launch of Prometheus 1:

According to Prometheus 1 project management, current funding is 
inadequate to support a 2015 launch of Prometheus 1 as initially 
planned. Following small funding increases from fiscal years 2005 
through 2007, the budget profile[Footnote 5] becomes relatively flat 
through fiscal year 2009 (see fig. 2). Project officials believe that 
the current profile would need to be increased beginning in fiscal year 
2007 to reflect project needs of a Jupiter Icy Moons mission. Decision 
makers will not get their first comprehensive picture of the project's 
requirements and the resources needed to meet those requirements--the 
first basis for funding decisions--until PMSR scheduled for summer 
2005. While the fiscal year 2006 request includes an updated Prometheus 
1 funding profile, a funding profile based on life-cycle cost 
estimates--which NASA plans to have when it enters the preliminary 
design phase--will not be included until NASA's fiscal year 2007 
request. 

Figure 2: NASA Prometheus 1 Fiscal Year 2005 Budget Request Profile:

[See PDF for image]

[End of figure]

Short Time Frames and History Foretell Difficulty in Defining 
Requirements and Developing Life-Cycle Cost Estimates:

The Prometheus 1 project office is required to develop preliminary 
requirements by PMSR. Defining the project's requirements and 
developing life-cycle cost estimates by then could be challenging, 
given the short time frames and NASA's past difficulties developing 
requirements and estimates. While it is not unusual for a project at 
this stage in acquisition to still be defining requirements, several 
factors could make it difficult for NASA to develop preliminary 
requirements by PMSR. The contractor, Northrop Grumman, was only 
recently selected, and according to project officials, input from both 
the contractor and Office of Naval Reactors is needed to finalize the 
preliminary ground, space, and launch systems requirements mandatory 
for PMSR. In addition, NASA continues to refine its requirements. For 
example, Prometheus 1 project management increased requirements for 
reactor lifetime, reactor power, and propellant tank capacity to ensure 
that the Prometheus 1 spacecraft and reactor designs could be used to 
support follow-on missions. Currently, project managers are working 
with broad NASA requirements for deep space exploration and more 
refined project requirements specific to the Prometheus 1 ground, 
space, and launch systems. 

NASA is also required to have an initial life-cycle cost estimate for 
Prometheus 1 at PMSR.[Footnote 6] However, because the estimate is 
based on a conceptual design, preliminary system requirements, and 
detailed technology development plans that are not yet complete, it 
will be difficult for NASA to develop an estimate in the short time 
available by PMSR. 

The project office is working with Northrop Grumman to merge and 
finalize the conceptual design. Once the conceptual design is 
finalized, the project office will update the work breakdown 
structure[Footnote 7] and develop a "grass roots" estimate of the 
spacecraft cost. However, project officials do not expect to receive 
cost estimates from the Office of Naval Reactors and Northrop Grumman, 
which are also needed to develop the estimate, until the end of 
February 2005. The JPL Costing Office will prepare a separate cost 
estimate based on its experiences with prior programs, and both JPL and 
NASA will contract for additional independent cost estimates. 

Adding to these complexities, NASA has historically had difficulty 
establishing life-cycle cost estimates. In May 2004, we reported that 
NASA's basic cost-estimating processes--an important tool for managing 
programs--lack the discipline needed to ensure that program estimates 
are reasonable.[Footnote 8] Specifically, we found that 10 NASA 
programs that we reviewed in detail did not meet all of our cost- 
estimating criteria--based on criteria developed by Carnegie Mellon 
University's Software Engineering Institute. Moreover, none of the 10 
programs fully met certain key criteria--including clearly defining the 
program's life cycle to establish program commitment and manage program 
costs, as required by NASA. In addition, only three programs provided a 
breakdown of the work to be performed. Without this knowledge, we 
reported that the programs' estimated costs may be understated and 
thereby subject to underfunding and cost overruns, putting programs at 
risk of being reduced in scope or requiring additional funding to meet 
their objectives. In this report we recommended that NASA take a number 
of actions to improve its cost -estimating practices. NASA concurred 
noting that our recommendations validated and reinforced the importance 
of activities underway at NASA. 

Business Case Allows Match of Needs to Resources and Facilitates 
Informed Decision Making:

By PDR--which occurs at end of the preliminary design phase and is 
scheduled for 2008--the fidelity of the information is expected to 
improve and could allow NASA to develop a business case that would 
match requirements with resources and provide decision makers with the 
information needed to determine whether continued investment in the 
project is warranted. However, in the past NASA has had difficulties 
developing the realistic requirements and cost estimates needed to 
develop a sound business case. 

To help ensure program requirements do not outstrip resources, leading 
commercial firms obtain the right knowledge about a new product's 
technology, design, and production at the right time. We have issued a 
series of reports[Footnote 9] on the success these firms have had in 
estimating the time and money to develop new and more sophisticated 
products--the kinds of results that NASA seeks. Our best practice work 
has shown that developing business cases based on matching requirements 
to resources before program start leads to more predictable program 
outcomes--that is, programs are more likely to be successfully 
completed within cost and schedule estimates and deliver anticipated 
system performance. 

Figure 3: Prometheus 1 Milestone Reviews:

[See PDF for image]

[End of figure]

A sound business case includes the following elements--well-defined 
requirements, a preliminary design, realistic cost estimates, and 
mature technology. While NASA does not require projects to develop a 
formal business case based on matching requirements to resources, JPL 
policy, which implements NASA policy, does require projects to develop 
documentation that could support formulation of a sound business case. 
Before a JPL project enters the preliminary design phase, JPL project 
implementation policy requires that the project develop preliminary 
requirements, a conceptual design, realistic cost estimates, and 
technology development plans. This policy also requires that the 
fidelity of information in these documents improve by PDR. 

The requirements and resource estimates NASA is developing for PMSR 
could form the basis for an initial business case based on matching 
Prometheus 1 requirements to available resources. However, Prometheus 1 
project management plans to continue directing requirements changes to 
accommodate follow-on missions. While our work shows that the 
preliminary design phase is the appropriate place to conduct systems 
engineering to support requirement/cost trade-off decisions, NASA needs 
to remain cognizant that adding requirements could increase cost and 
risk. In addition, NASA has had past difficulty developing the 
realistic requirements and cost estimates needed to develop a sound 
business case. These difficulties have resulted in the termination of 
several major efforts after significant investment of resources. For 
example, in 2002 NASA terminated the Space Launch Initiative (SLI) 
program--a $4.8 billion, 5-year program to build a new generation of 
space vehicles to replace its aging space shuttle. SLI was a complex 
and challenging endeavor for NASA, both technically and from a business 
standpoint. The SLI program faced some of the same challenges that 
Prometheus 1 is struggling with today, such as the need to develop and 
advance new airframe and propulsion technologies. SLI did not achieve 
its goals, in part, because NASA did not develop realistic requirements 
and cost estimates. 

NASA's Plans to Ensure Mature Technologies Rely on Prototype 
Demonstrations:

Leading firms make an important distinction between technology 
development and product development. Technologies that are not mature 
continue to be developed in the technology base--they are not included 
in a product development. Our best practices work has also shown that 
there is a direct relationship between the maturity of technologies and 
the accuracy of cost and schedule estimates. NASA's Prometheus 1 
technologies are currently immature. The Prometheus 1 project office is 
preparing technology development plans to guide the development of each 
key technology during the preliminary design phase. 

Mature Technologies Are Key to Minimizing Risk:

Maturing technologies during the preliminary design phase is a key 
element of matching needs to resources before entering the product 
development phase. Our best practices work has shown that technology 
readiness levels[Footnote 10] (TRL)--a concept developed by NASA--can 
be used to gauge the maturity of individual technologies (see fig. 
4).[Footnote 11] (See app. I for detailed definition of TRLs.) 
Specifically, TRL 6--demonstrating a technology as a fully integrated 
prototype in a realistic environment--is the level of maturity needed 
to minimize risks for space systems entering product development. 

Figure 4: Technology Maturity Levels for Product Development:

[See PDF for image]

[End of figure]

While development of Prometheus 1 critical technologies is under way, 
the technologies will require extensive advancement before they are 
mature enough to provide the revolutionary capabilities of the 
Prometheus 1 spacecraft. The overall technology objective for 
Prometheus 1 is to safely develop and operate a spacecraft with a 
nuclear-reactor-powered electric propulsion system. To achieve this 
objective, the spacecraft will require advancement in several 
technology areas, including, nuclear electric power, power conversion 
and heat rejection systems, nuclear electric propulsion, high power 
communications, radiation-hardened electronics, and AR&D. (See app. II 
for a more detailed explanation of these technologies.) NASA's fiscal 
year 2005 budget request indicates that these technologies are either 
at TRL 3 (individual technologies have been demonstrated in a 
laboratory environment) or TRL 4 (system components have been 
demonstrated in a laboratory environment). Before NASA conducts the PDR 
in 2008, it will need to mature the technologies--each of which comes 
with a unique set of engineering challenges. 

To gauge the maturation of the Prometheus 1 technologies, the 
Prometheus 1 project office is preparing technology development plans, 
which rely on the use of maturity criteria tables (MCT), a concept 
similar to TRLs.[Footnote 12] The specific maturation criteria for each 
technology vary greatly, but all technologies are to be matured by PDR 
to the point that:

* developmental models are complete,

* all major risks to each technology are retired,

* all major manufacturing issues are resolved, and:

* plans for obtaining life data that will provide confidence that the 
hardware will meet the mission lifetime requirements are in hand. 

Prometheus 1 project officials believe these criteria roughly 
correspond to a TRL 5 (component and/or breadboard validation in a 
relevant environment) or a TRL 6 (system/subsystem model or prototype 
demonstration in a relevant environment). The program office's position 
is that using MCTs that are equivalent to TRL 5 and TRL 6 at PDR is 
appropriate because the program office is both the technology developer 
and product developer and, as such, has a thorough understanding of how 
mature the technologies need to be at certain points in time as the 
program progresses. Nevertheless, the dual role of project office as 
both technology and product developer is not unique, and our best 
practices body of work shows that a TRL 6 is the level of maturity 
needed to minimize risks for space systems entering product 
development. 

Conclusions:

NASA is quickly approaching one of the most critical phases in its 
acquisition of Prometheus 1--the preliminary design phase. While the 
impetus for the changes made to the program--subsequent to our 
providing a draft of this report to NASA for comment--recognize the 
technical, schedule, and operational risk of this program, there is 
still much work to be done. Based on the information presented at PMSR, 
now scheduled for summer 2005, NASA will need to decide at what level 
to fund the project. However, NASA will be challenged to develop the 
information required at PMSR, given the compressed time frames. 
Although PDR is still several years out, NASA will face significant 
challenges in meeting this milestone, given the immaturity of the 
revolutionary technologies that NASA anticipates will be needed to 
successfully launch Prometheus 1. While NASA is developing well-defined 
criteria tables for maturing Prometheus 1 technologies, the many 
inherent unknowns in developing technologies frequently results in 
unanticipated difficulties and delays. NASA's current policy does not 
require projects to develop knowledge-based business cases that match 
requirements to available resources and include controls to ensure that 
sufficient knowledge has been attained and therefore the agency had not 
planned to develop such a business case for Prometheus 1. We have 
found, however, that establishing a formal business case based on a 
knowledge-based approach that includes matching requirements and 
available resources--which include technical and engineering knowledge, 
time and funding--and controls to ensure that sufficient knowledge has 
been attained at critical junctures within the product development 
process is an essential part of any product development justification. 
The risk associated with failing to meet these challenges is 
considerable. If NASA decides to move forward without adequate 
information at PMSR--that matches requirements and available resources 
and provides NASA decision makers with a clear understanding of 
Prometheus 1's potential return on investment--Prometheus 1 may be 
unable to compete for funding within NASA. Ultimately, NASA could find, 
as it has in the past, that the program must be cancelled after having 
invested millions of dollars. 

Recommendations for Executive Action:

We recommend that the NASA Administrator take the following two 
actions: 

* identify at PMSR the level of resources the agency is committing to 
the project and direct project officials to develop project 
requirements based on this resource constraint and:

* ensure that prior to proceeding beyond PDR (currently planned for 
2008) a sound business case is established which includes confirmation 
that (1) critical technologies have been successfully demonstrated as 
mature, (2) systems engineering has been conducted to support 
requirements/cost trade-off decisions, (3) requirements and resource 
estimates have been updated based on the results of the preliminary 
design phase, (4) knowledge based criteria are established at each 
critical juncture to ensure that relevant product development knowledge 
is captured, and (5) decision reviews are held to determine that 
appropriate knowledge was captured to allow a move to the next phase. 

Agency Comments:

In written comments on a draft of this report, NASA's Deputy 
Administrator stated that the agency concurs with the recommendations, 
adding that the recommendations validate and reinforce the importance 
of activities underway at NASA to improve NASA's management of complex 
technical programs. 

Subsequent to our draft report being provided to NASA for comment, 
significant changes were made to the Prometheus 1 project. NASA's 
fiscal year 2006 budget request includes changes to the Prometheus 1 
project that directly address the recommendations in this report. 
According to NASA's budget justification, the agency is planning a less 
complex mission than the original JIMO mission. According to program 
officials who we consulted with following the release of the budget, 
eliminating the long reactor lifetime, stringent radiation hardening, 
multiple launches, and AR&D required for the JIMO mission will allow 
NASA "to walk before it runs" and significantly reduce cost and 
technical risks. As a result, NASA has delayed PMSR until summer 2005 
and is conducting an analysis of alternatives to identify a relevant 
mission with reduced technical, schedule, and operational risk. The 
fiscal year 2006 budget request also reshapes the Prometheus 1 funding 
profile to provide an orderly increase in developmental activities. 

Notwithstanding agreement with our recommendations, the Deputy 
Administrator stated that NPR 7120.5B requires projects to develop a 
business case. As we noted in this draft report, we recognize that NASA 
policy requires the development of elements that could support the 
formulation of a knowledge-based business case. However, we found no 
explicit requirement within NPR 7120.5B for NASA projects to develop a 
business case of any kind. More importantly, while NPR 7120.5B does 
require that projects establish controls to monitor performance against 
cost, schedule, and performance baselines and to conduct reviews 
throughout the project's lifecycle, it does not establish specific 
knowledge-based controls to ensure that the knowledge necessary to 
match resources to requirements is in hand before moving forward. For 
example, whereas NPR 7120.5B requires projects to conduct a preliminary 
design review before entering NASA's implementation phase, i.e., 
product development, it does not establish knowledge-based criteria to 
ensure that technologies needed to meet essential product requirements 
have been demonstrated to work in a realistic environment. Likewise, 
NASA policy requires a critical design review during a project's 
implementation phase but does not include knowledge-based criteria to 
ensure the design is stable. We have found that such knowledge-based 
criteria, when tied to major events on a program's schedule, can 
disclose whether gaps or shortfalls exist in demonstrated knowledge, 
which can presage future cost, schedule and performance problems. 

In his comments, the Deputy Administrator also noted that the 
Exploration Systems Mission Directorate is in the process of initiating 
a number of reforms to its project management policies and specified 
formulation dates in the coming months. He outlined these reforms and 
explained how they will allow NASA to address the recommendations in 
our report. We are encouraged by these planned changes. If properly 
implemented, they could be positive steps toward implementing a 
knowledge-based approach to project management. 

The Deputy Administrator also requested that the relationship between 
JPL and NASA project management requirements be explicitly stated in 
the report. We moved the information from a footnote into the body of 
the report to clarify that relationship. We also addressed NASA's 
technical comments as appropriate throughout the report. 

Scope and Methodology:

To determine whether NASA is establishing justification for the project 
and ensuring critical technologies are mature, we conducted interviews 
with NASA Exploration Systems Mission Directorate and Prometheus 1 
project officials at NASA Headquarters, Washington, D.C; Marshall Space 
Flight Center, Huntsville, Ala; and the Jet Propulsion Laboratory, 
Pasadena, Calif. We obtained and reviewed pertinent documents from the 
agency. We conducted quantitative and qualitative analyses of project 
schedules, risk assessments, budget documentation, technology maturity 
assessments and technology development plans. We compared these 
documents to criteria established in JPL and NASA policies governing 
developmental programs and to criteria for a knowledge based approach 
to acquisition described in GAO's best practices body of work. We 
discussed key project challenges with Prometheus 1 project officials, 
and conducted GAO team meetings to discuss analyses and developing 
issues. Our audit work was completed between April 2004 and January 
2005. 

As agreed with your office, unless you announce its contents earlier, 
we will not distribute this report further until 30 days from its 
issuance date. At that time, we will send copies to the NASA 
Administrator and interested congressional committees. We will make 
copies available to others upon request. In addition, the report will 
be available at no charge on the GAO web site at http://www.gao.gov. 

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-4841 or lia@gao.gov. Key contributors to this 
report are acknowledged in appendix IV. 

Signed by: 

Allen Li: 
Director: 
Acquisition and Sourcing Management:

[End of section]

Appendix I: Technology Readiness Levels Definitions:

Technology readiness level: 1. Basic principles observed and reported; 
Description: Lowest level of technology readiness. Scientific research 
begins to be translated into applied research and development. Examples 
might include paper studies of a technology's basic properties; 
Hardware and software: None (Paper studies and analysis); 
Demonstration environment: None. 

Technology readiness level: 2. Technology concept and/or application 
formulated; 
Description: Invention begins. Once basic principles are observed, 
practical applications can be invented. The application is speculative 
and there is no proof or detailed analysis to support the assumption. 
Examples are still limited to paper studies; 
Hardware and software: None (Paper studies and analysis); 
Demonstration environment: None. 

Technology readiness level: 3. Analytical and experimental critical 
function and/or characteristic proof of concept; 
Description: Active research and development is initiated. This 
includes analytical studies and laboratory studies to physically 
validate analytical predictions of separate elements of the technology. 
Examples include components that are not yet integrated or 
representative; 
Hardware and software: Analytical studies and demonstration of nonscale 
individual components (pieces of subsystem); 
Demonstration environment: Lab. 

Technology readiness level: 4. Component and/or breadboard. Validation 
in laboratory environment; 
Description: Basic technological components are integrated to establish 
that the pieces will work together. This is relatively "low fidelity" 
compared to the eventual system. Examples include integration of "ad 
hoc" hardware in a laboratory; 
Hardware and software: Low fidelity breadboard. Integration of nonscale 
components to show pieces will work together. Not fully functional or 
form or fit but representative of technically feasible approach 
suitable for flight articles; 
Demonstration environment: Lab. 

Technology readiness level: 5. Component and/or breadboard validation 
in relevant environment; 
Description: Fidelity of breadboard technology increases significantly. 
The basic technological components are integrated with reasonably 
realistic supporting elements so that the technology can be tested in a 
simulated environment. Examples include "high fidelity" laboratory 
integration of components; 
Hardware and software: High fidelity breadboard. Functionally 
equivalent but not necessarily form and/or fit (size weight, materials, 
etc.). Should be approaching appropriate scale. May include integration 
of several components with reasonably realistic support elements/ 
subsystems to demonstrate functionality; 
Demonstration environment: Lab demonstrating functionality but not form 
and fit. May include flight demonstrating breadboard in surrogate 
aircraft. Technology ready for detailed design studies. 

Technology readiness level: 6. System/subsystem model or prototype 
demonstration in a relevant environment; 
Description: Representative model or prototype system, which is well 
beyond the breadboard tested for TRL 5, is tested in a relevant 
environment. Represents a major step up in a technology's demonstrated 
readiness. Examples include testing a prototype in a high fidelity 
laboratory environment or in simulated operational environment; 
Hardware and software: Prototype--Should be very close to form, fit and 
function. Probably includes the integration of many new components and 
realistic supporting elements/subsystems if needed to demonstrate full 
functionality of the subsystem; 
Demonstration environment: High-fidelity lab demonstration or limited/ 
restricted flight demonstration for a relevant environment. Integration 
of technology is well defined. 

Technology readiness level: 7. System prototype demonstration in an 
operational environment; 
Description: Prototype near or at planned operational system. 
Represents a major step up from TRL 6, requiring the demonstration of 
an actual system prototype in an operational environment, such as in an 
aircraft, vehicle or space. Examples include testing the prototype in a 
test bed aircraft; 
Hardware and software: Prototype. Should be form, fit and function 
integrated with other key supporting elements/subsystems to demonstrate 
full functionality of subsystem; 
Demonstration environment: Flight demonstration in representative 
operational environment such as flying test bed or demonstrator 
aircraft. Technology is well substantiated with test data. 

Technology readiness level: 8. Actual system completed and "flight 
qualified" through test and demonstration; 
Description: Technology has been proven to work in its final form and 
under expected conditions. In almost all cases, this TRL represents the 
end of true system development. Examples include developmental test and 
evaluation of the system in its intended weapon system to determine if 
it meets design specifications; 
Hardware and software: Flight qualified hardware; 
Demonstration environment: DT&E in the actual system application. 

Technology readiness level: 9. Actual system "flight proven" through 
successful mission operations; 
Description: Actual application of the technology in its final form and 
under mission conditions, such as those encountered in operational test 
and evaluation. In almost all cases, this is the end of the last "bug 
fixing" aspects of true system development. Examples include using the 
system under operational mission conditions; 
Hardware and software: Actual system in final form; 
Demonstration environment: OT&E in operational mission conditions. 

Source: GAO. 

[End of table]

[End of section]

Appendix II: Prometheus 1 Critical Technologies:

Nuclear Reactor:

The nuclear reactor is the key element of the Prometheus 1 spacecraft. 
Without the power levels supplied by the reactor, the proposed 
propulsion, science, and communication systems are not feasible. 
Designing, constructing, and utilizing highly reliable, safe, portable 
nuclear reactors is not new--nuclear reactors have been used in 
submarines for almost 50 years. However, the United States has very 
little experience operating nuclear reactors in a space environment and 
tackling space unique nuclear application issues. The Office of Naval 
Reactors, the organizational unit in the Department of Energy 
responsible for developing nuclear reactors for the Navy, will be 
responsible for all portions of the Prometheus 1 reactor development 
effort. 

The space environment places significant weight constraints on the 
reactor design and requires semi-autonomous control. Unlike submarines 
and aircraft carriers, all spacecraft have serious weight constraints 
driven by the cost of launching payloads into orbit. Consequently, 
spacecraft designers put great effort into eliminating weight. Further, 
where conventional reactors have hands on operators, the Prometheus 1 
reactor must be remotely controlled. NASA estimates that control 
communications will take about 40 minutes to travel one way between 
Earth and the Jovian system. 

Power Conversion:

A power conversion system accepts the thermal energy from the reactor 
and converts it to useful electrical power for the spacecraft. Power 
conversion is an integral part of any power generation system taking 
the form of steam turbine generators in terrestrial utility plants and 
nuclear submarines. NASA is considering two types of power conversion 
systems--dynamic and static. According to NASA, the dynamic systems 
under consideration offer the benefits of increased efficiency, reduced 
weight and mass, and decreased nuclear fuel requirements. The static 
systems, however, have a technology heritage in prior spacecraft and 
could offer increased reliability because they have no moving parts. 

Heat Rejection:

Since the conversion process in a fission reactor is never 100 percent 
efficient, heat rejection is required to dissipate waste energy. This 
is usually accomplished with large pumped-water cooling systems on 
earth. Space based power conversion would require a large radiator 
system to dissipate the waste heat in the vacuum of space. The 
requirement to fold the large radiator system into the launch vehicle 
fairing and deploy it after launch complicates the radiator system 
design. (See fig. 1.)

Nuclear Electric Propulsion:

Operating electric propulsion systems in space applications, including 
deep space, is not new. There is extensive experience with electric 
propulsion systems on satellites. In addition, NASA's Deep Space 1 
spacecraft was propelled using an electric propulsion ion thruster, 
similar in nature to the concept being developed for Prometheus 1. The 
thruster power levels required by Prometheus 1 have been demonstrated 
in a laboratory environment. The lifetime required by Prometheus 1, 
however, has not been demonstrated. Furthermore, lifetime testing of 
existing ion thrusters has demonstrated that these thrusters were 
approaching "wear out failure"[Footnote 13] after 30,352 hours. The 
Prometheus 1 thrusters will need to be qualified for operational 
durations approaching 120,000 hours. NASA recognizes that they will 
have to develop models and accelerated aging techniques to demonstrate 
the lifetime requirement. 

Radiation Hardening:

All electronic components of the Prometheus 1 spacecraft must be 
radiation hardened or shielded from radiation produced by the onboard 
nuclear reactor and the harsh radiation environment of the Jovian 
system. Shielding electronics and the science packages, requires a 
heavy metal or some other material, which may not yet be developed, and 
increases the weight and mass of the spacecraft. The additional weight 
used to shield the spacecraft lessens the science payload package that 
can be taken onboard for scientific data collection and research and 
increases the size and power of the launch vehicle required to launch 
Prometheus 1 into earth orbit:

High Power Communications:

The nuclear reactor will provide increased electrical power for 
communications. This translates to increased bandwidth and data rates. 
The high power communications system onboard the Prometheus 1 
spacecraft, will provide tens of compact disks full of data back to 
earth. Analogous missions such as Cassini provide only a couple of 
floppy disks full of data. (A floppy disk typically holds about 1.44 MB 
of data. A compact disk typically holds about 700 MB of data.) 
According to project officials, the higher power communications system 
on the Prometheus 1 spacecraft will require upgrades to the Deep Space 
Network, which are out of the purview of the Prometheus 1 project. 

Autonomous Rendezvous and Docking:

There is no launch vehicle in the present or proposed U.S. inventory 
capable of launching the Prometheus 1 spacecraft, conceived for a 
mission to Jupiter's Icy Moons, into orbit in one piece. The conceptual 
design currently shows the Prometheus 1 spacecraft to weigh between 29 
and 36 metric tons and be about 58 meters in length. The current 
concept is to use multiple launches, 2 to 5, to place the spacecraft 
components in orbit and to use AR&D technology to assemble the 
spacecraft in orbit. Prometheus 1 is relying on NASA's Demonstration 
Autonomous Rendezvous Technology and Hubble Robotic Servicing Mission, 
and the Defense Advanced Research Projects Agency's Orbital Express 
programs for AR&D technology. These programs use different sensors and 
approaches to AR&D thereby providing Prometheus 1 with various options 
for consideration. 

[End of section]

Appendix III: Comments from the National Aeronautics and Space 
Administration:

National Aeronautics and Space Administration:
Office of the Administrator: 
Washington, DC 20546-0001:

February 15, 2005:

Mr. Allen Li:
Director, Acquisition and Sourcing Management Team:
United States General Accounting Office: 
Washington, DC 20548:

Dear Mr. Li:

The National Aeronautics and Space Administration (NASA) appreciates 
the opportunity to review and comment on your draft report General 
Accounting Office (GAO)-05-242, "NASA's Space Vision: Business Case for 
Prometheus I Needed to Ensure Requirements Match Available Resources."

We concur with your recommendations and note that they validate and 
reinforce the importance of activities currently underway at NASA to 
strengthen our management of complex technical programs, including the 
development of Prometheus 1 and its attendant supporting infrastructure 
and technologies. However, we would like to clarify the status of our 
program management practices and policies.

The draft report states that the Jet Propulsion Laboratory's (JPL) 
project implementation policy requires documentation that includes 
elements essential to developing a sound business case. However, GAO 
states that NASA does not require projects to develop knowledge-based 
business cases that match requirements to available resources. The 
report also suggests that NASA does not employ controls to ensure that 
sufficient knowledge has been attained during project development.

As NASA's Federally Funded Research and Development Center (FFRDC), JPL 
programs and practices are required to conform to NASA requirements. 
The NASA requirements for program/project management are documented in 
NASA's "Program and Project Management Processes and Requirements" 
document (NASA Procedural Requirements (NPR) 7120.513, dated November 
2002). As stated in NPR 7120.513, NASA requires development of a 
business case for all of its projects, and JPL has instituted polices 
and procedures to conform to the NASA requirements. The draft report 
text does not explicitly acknowledge the fact that JPL's "Project 
Implementation Policy" is written to conform to these NASA 
requirements. NASA believes that for the sake of clarity, the 
relationship between JPL and NASA program/project management 
requirements should be explicitly stated in the report. We would be 
happy to discuss this issue further with you or provide you with a copy 
of the NPR. The NPR is also available online at 
http://www.hq.nasa.gov/office/codeq/doctree/71205.htm.

As you noted, NASA is in the process of conducting conceptual design 
(Phase A) for the Prometheus 1 project in anticipation of entering 
preliminary design (Phase B). Concurrent with these activities, NASA's 
Exploration Systems Mission Directorate (ESMD) is developing detailed 
program management requirements on why, when, and how to implement NASA 
programs and projects. These requirements consist of a three document 
suite based on the Department of Defense (DoD) and NASA best practices 
cited in your draft report, i.e., well-defined requirements, a 
conceptual design, realistic cost estimates, and independently reviewed 
technology development plans. This suite of documents, which also 
considers lessons learned at NASA, will succinctly define the policies 
and requirements that all ESMD program/project managers must use in 
overseeing NASA's exploration activities. ESMD's document suite 
consists of the Single Acquisition Management Plan (SAMP), the Systems 
Engineering Management Plan (SEMP), and the Program Management Handbook 
(PMH).

The SAMP documents the ESMD overarching acquisition strategy. 
Recognizing that each program has unique challenges, the SAMP discusses 
application of the ESMD acquisition strategy for each major program 
under the ESMD's purview. The SAMP explicitly addresses the use of 
spiral development and implementation of the "Strategy to Task to 
Technology" (STT) approach for program management. The STT uses an 
integrated team of stakeholders and an Operational Advisory Group (OAG) 
to identify and assess user-defined operational needs and priorities. 
This process is designed to implement the best practices identified in 
GAO-04-386SP by creating a credible, auditable trail of the ESMD 
decision-making process.

The SEMP identifies the sequence of systems engineering processes and 
activities for the technical aspects of ESMD systems engineering and 
integration management. The intent of the SEMP is to combine DoD and 
NASA best practices in a way which supports NASA's unique mission and 
which defines a rigorous process for determining whether proposed 
solutions meet ESMD requirements.

The PMH defines the organizational roles, responsibilities, and 
processes required for program management. Processes are defined in 
terms of program life cycles and are tailored to the characteristics of 
basic and applied research, technology development, and mission system 
development. Program life. cycles are punctuated by Key Decision Points 
for transitioning from one program phase to the next. Throughout the 
PMH, program managers are encouraged to be candid and forthcoming about 
program status, explicitly including identifying risks and problems 
without fear of personal consequences.

The ESMD management is committed to providing the best resources and 
tools available to its staff and consistently stresses the need for 
credibility, affordability, and sustainability of its programs. The 
NASA Associate Administrator for Exploration Systems, Craig E. Steidle, 
has personally visited each NASA Center and JPL to explain the upcoming 
program management practices. He emphasizes that NASA must be prepared 
to demonstrate a positive return on our resource investments. ESMD 
goals are to:

1. Convene a diverse group of stakeholders, including designers, 
operations staff, and technicians to evaluate and identify performance 
trade-offs early in the program.

2. Apply available technologies to reduce system life-cycle costs, 
specifically infrastructure development.

3. Mature a technology before employing it.

4. Partner with industry as appropriate.

As a civilian Government agency, NASA's mission is to expand human 
knowledge of the Earth and phenomena in the atmosphere and space. 
Consequently, many of NASA's exploration projects require first-of-a- 
kind technology development. Nevertheless, NASA must ensure that 
available resources, both human and otherwise, are effectively and 
efficiently applied in our technology development activities.

The following paragraphs provide the current status and planned 
approach for addressing each of the GAO recommendations:

Recommendation 1: Identify at Preliminary Mission and Systems Review 
(PMSR) the level of resources the agency is committing to the project 
and direct project officials to develop project requirements based on 
this resource constraint.

NASA concurs with this recommendation and is documenting the ESMD 
process for transitioning from Phase A to Phase B for all programs 
under ESMD's purview. JPL's PMSR is an in-depth review of the project 
conducted shortly before the NASA program review. The project must pass 
both reviews in order for the project to move forward. The ESMD's PMH 
requires that program managers define the mission and system concepts, 
parameters, constraints, and requirements that will enable development 
of the system and mission on a given schedule to meet established goals 
within a reasonable cost estimate. Cost estimates are commensurate with 
the available information at that point in project development. As the 
project matures and the design progresses, project cost estimates 
become more refined. Program managers will periodically review program 
activities against resource constraints, using earned value management 
techniques, and report actual or anticipated deviations from program 
costs and/or schedule to senior management for resolution. 
Stakeholders, including operations, science, engineering, and the OAG 
staff will review mission objectives, technology requirements, and 
resource constraints throughout the project life cycle to ensure that 
requirements can be met within defined resource and technology 
constraints. Before transitioning to Phase B, ESMD also requires an 
independent review by an Independent Program Assessment Team to 
determine whether or not the program is ready to proceed.

The ESMD's PMH will be issued in February 2005, and the SAMP and SEMP 
will follow thereafter in the summer of 2005.

Recommendation 2: Ensure that prior to proceeding beyond Preliminary 
Design Review (PDR) (currently planned for 2008) a sound business case 
is established which includes confirmation that (1) critical 
technologies have been successfully demonstrated as mature, (2) systems 
engineering has been conducted to support requirement/cost trade-off 
decisions, (3) requirements and resource estimates have been updated 
based on the results of the preliminary design phase, (4) knowledge 
based criteria are established at each critical juncture to ensure that 
relevant product development knowledge is captured, and (5) decision 
reviews are held to determine that appropriate knowledge was captured 
to allow a move to the next phase.

The ESMD will be issuing the PMH in February 2005. The PMH will 
explicitly define the requirements for transitioning between each phase 
of a program/project. The PMH defines specific entrance and exit 
criteria for each program phase and requires periodic reassessment of 
technology maturity, systems engineering interfaces and requirements, 
independent cost and technical analysis, value engineering, updating 
requirements and resource estimates, and analysis of alternatives to 
determine whether a given program/project should proceed. Earned value 
management techniques will be used. The PMH also requires documentation 
of decision bases to ensure that the STT process is credible and 
auditable. The three-document suite will be revised to incorporate 
lessons learned as NASA gains experience in implementing this new 
program management acquisition strategy to ensure that we continue to 
improve our processes.

We also note that both the report summary page and the report itself 
cite the Congressional Budget Office's (CBO) project cost estimate as 
$10 billion dollars. The CBO estimate was not coordinated with NASA. 
NASA is currently developing a disciplined process to develop NASA's 
initial project cost estimate. We are also providing technical comments 
under separate cover.

Thank you for the opportunity to comment on the draft report. We 
appreciate your recommendations and will continue to work on improving 
how we do business.

Cordially,

Signed by: 

Frederick D. Gregory: 
Deputy Administrator:

Enclosure: 

[End of section]

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact:

Allen Li, (202) 512-4841:

Acknowledgments:

In addition to the contact named above, James Morrison, Jerry Herley, 
John Warren, Tom Gordon, Ruthie Williamson, Karen Sloan and Sylvia 
Schatz made key contributions to this report. 

[End of section]

GAO Related Products:

Best Practices: Using a Knowledge-Based Approach to Improve Weapon 
Acquisition, GAO-04-386SP. Washington, D.C.: January 2004. 

Best Practices: Setting Requirements Differently Could Reduce Weapon 
Systems' Total Ownership Costs. GAO-03-57. Washington, D.C.: February 
11, 2003. 

Defense Acquisitions: DOD's Revised Policy Emphasizes Best Practices, 
but More Controls Are Needed. GAO-04-53. Washington, D.C.: November 10, 
2003. 

Best Practices: Capturing Design and Manufacturing Knowledge Early 
Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15, 
2002. 

Defense Acquisitions: DOD Faces Challenges in Implementing Best 
Practices. GAO-02-469T. Washington, D.C.: February 27, 2002. 

Best Practices: Better Matching of Needs and Resources Will Lead to 
Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: March 8, 
2001. 

Best Practices: A More Constructive Test Approach Is Key to Better 
Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.: July 31, 
2000. 

Defense Acquisition: Employing Best Practices Can Shape Better Weapon 
System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 26, 2000. 

Best Practices: DOD Training Can Do More to Help Weapon System Program 
Implement Best Practices. GAO/NSIAD-99-206. Washington, D.C.: August 
16, 1999. 

Best Practices: Better Management of Technology Development Can Improve 
Weapon System Outcomes. GAO/NSIAD-99-162. Washington, D.C.: July 30, 
1999. 

Defense Acquisitions: Best Commercial Practices Can Improve Program 
Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999. 

Defense Acquisition: Improved Program Outcomes Are Possible. GAO/T- 
NSIAD-98-123. Washington, D.C.: March 18, 1998. 

Best Practices: DOD Can Help Suppliers Contribute More to Weapon System 
Programs. GAO/NSIAD-98-87. Washington, D.C.: March 17, 1998. 

Best Practices: Successful Application to Weapon Acquisition Requires 
Changes in DOD's Environment. GAO/NSIAD-98-56. Washington, D.C.: 
February 24, 1998. 

Major Acquisitions: Significant Changes Underway in DOD's Earned Value 
Management Process. GAO/NSIAD-97-108. Washington, D.C.: May 5, 1997. 

Best Practices: Commercial Quality Assurance Practices Offer 
Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.: August 26, 
1996. 

[End of section]

(120346):

FOOTNOTES

[1] This estimate is based on conducting the Jupiter Icy Moons Orbiter 
Mission. The Congressional Budget Office did not consult with NASA to 
develop this estimate. 

[2] The Prometheus Nuclear Systems and Technology program stems from 
the former Nuclear Systems Initiative to develop nuclear power for deep 
space exploration. 

[3] Subsequent to being provided a draft of this report for comment, 
NASA announced that it was conducting an analysis of alternatives to 
identify a new mission. 

[4] JPL's Project Implementation Policy establishes JPL's institutional 
structure for implementation and management of JPL flight projects in 
accordance with NASA Procedural Requirements 7120.5B, NASA Program and 
Project Management Processes and Requirements, which governs all NASA 
development programs and projects. 

[5] Between 10 and 15% of the budget shown is allocated to Advanced 
Systems and Technology Development activities supporting current and 
future missions. 

[6] NASA guidance requires that life-cycle costs be estimated, 
assessed, and controlled throughout a program's life cycle. The 
estimates are to be prepared to support major program reviews and the 
development of budget submissions. 

[7] A work breakdown structure is a product oriented division of 
hardware, software, services and project unique tasks that organizes 
and defines the product to be developed and serves as the basis for 
estimating both cost and schedule. 

[8] GAO, NASA: Lack Of Disciplined Cost Estimating Processes Hinders 
Effective Program Management, GAO-04-642 (Washington, D.C.: May 28, 
2004). 

[9] GAO, Best Practices: Better Matching of Needs and Resources Will 
Lead to Better Weapon System Outcomes, GAO-01-288 (Washington, D.C.: 
Mar. 8, 2001). 

[10] Technology readiness levels characterize the readiness of 
technologies for handoff to project implementers. Nine levels are 
defined representing concepts from fundamental research level through 
technologies fully qualified and demonstrated in flight. 

[11] GAO, Best Practices: Using a Knowledge-Based Approach to Improve 
Weapon Acquisition, GAO-04-386SP (Washington D.C.: January 2004). 

[12] According to the Prometheus project office, the MCT [system], 
[also developed by NASA], is an improvement over the TRL [system] 
because MCT definitions are more detailed and quantifiable and, 
therefore, provide more clarity to technology developers, managers, and 
independent reviewers. Project office officials also note that the 
criteria in the tables have been reviewed through an independent peer 
assessment to validate that they provide the comprehensive set of 
measurements that need to be made to verify that a particular maturity 
has been met in a specific technology. 

[13] Wear out failure is the point at which a system stops operating 
because its mechanical or physical parts are worn to the point they 
will no longer function. 

GAO's Mission:

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony:

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone:

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548:

To order by Phone:

	

Voice: (202) 512-6000:

TDD: (202) 512-2537:

Fax: (202) 512-6061:

To Report Fraud, Waste, and Abuse in Federal Programs:

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470:

Public Affairs:

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: