This is the accessible text file for GAO report number GAO-05-480 
entitled 'Defense Technology Development: Management Process Can Be 
Strengthened for New Technology Transition Programs' which was released 
on June 17, 2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

United States Government Accountability Office: 

GAO: 

June 2005: 

Defense Technology Development: 

Management Process Can Be Strengthened for New Technology Transition 
Programs: 

GAO-05-480: 

GAO Highlights: 

Highlights of GAO-05-480, a report to the Congressional Committees on 
Armed Services: 

Why GAO Did This Study: 

The Department of Defense (DOD) and Congress both recognize that 
Defense technology innovations sometimes move too slowly from the lab 
to the field. Three new programs have been recently created in DOD to 
help speed and enhance the transition of new technologies. A report 
accompanying the fiscal year 2003 National Defense Authorization Act 
required GAO to review two of these programs—the Technology Transition 
Initiative (TTI) and Defense Acquisition Challenge Program (DACP). The 
first is designed to speed transition of technologies from DOD labs to 
acquisition programs and the second is designed to introduce cost-
saving technologies from inside and outside DOD. We were also asked to 
review the Quick Reaction Fund, which is focused on rapidly field 
testing promising new technology prototypes. We assessed the impact the 
programs had on technology transition and the programs’ selection, 
management and oversight, and assessment practices. 

What GAO Found: 

The ability to spur and leverage technological advances is vital to 
sustaining DOD’s ability to maintain its superiority over others and to 
improve and even transform how military operations are conducted. The 
three new transition programs we reviewed are all appropriately 
targeted on what has been a critical problem in this regard—quickly 
moving promising technologies from the laboratory and commercial 
environment into actual use. Moreover, by tailoring processes and 
criteria to focus on different objectives, whether that may be saving 
time or money or broadening the industrial base, DOD has had an 
opportunity to experiment with a variety of management approaches and 
criteria that can be used to help solve transition problems affecting 
the approximately $69 billion spent over the past 3 years on later 
stages of technology development. 

However, it is too soon for us to determine the impact the three new 
DOD technology transition programs are having. At the time of our 
review, the programs—the TTI, DACP, and Quick Reaction Fund—had 
completed only 11 of 68 projects funded in fiscal years 2003 and 2004; 
of those, only 4 were providing full capability to users. Additionally, 
the programs have limited measures to gauge success of individual 
projects and return on investment. Nonetheless, reports from the 
programs have pointed to an array of benefits, including quicker 
fielding of technological improvements, cost savings, and the 
opportunity for DOD to tap into innovative technologies from firms that 
are new to defense work. Some sponsored technologies are bringing 
benefits to warfighters, such as a small, unmanned aircraft that can 
detect chemical and biological agents, and a device the size of an ink 
pen that can be used to purify water on the battlefield or in disaster 
areas. Furthermore, DOD officials credit the programs with giving 
senior leaders the flexibility to rapidly address current warfighter 
needs and for highlighting smaller technology projects that might 
otherwise be ignored. 

Long-term success for the programs likely will depend on how well the 
programs are managed and overseen. The programs must have effective 
processes for selecting the best projects, and management and oversight 
processes that will catch potential problems early. Thus far, of the 
three programs, the DACP has adopted the most disciplined and 
structured process for selecting and managing projects, and has 
encountered few problems managing projects. However, the program has 
had some difficulties processing the large number of proposals it 
receives. The TTI has also established selection criteria and 
processes, but it is unclear the extent to which it is reaching its 
intended audience and has had less success in tracking its projects. 
The Quick Reaction Fund has the least structured processes of the three 
programs—a deliberate approach seen as providing the flexibility needed 
to field innovations rapidly. It has had some difficulty selecting, 
managing and tracking projects. 

What GAO Recommends: 

GAO recommends that DOD develop data and measures that can be used to 
assess short- and long-term impacts of the programs and take other 
actions to strengthen selection, management and oversight. DOD agreed 
with our recommendations as they related to the DACP and TTI programs, 
but does not believe they apply to the Quick Reaction Fund program. 

www.gao.gov/cgi-bin/getrpt?GAO-05-480. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Mike Sullivan at (937) 
258-7915 or sullivanm@gao.gov. 

[End of section]

Contents: 

Letter: 

Results in Brief: 

Background: 

Technology Transition Programs Offering Benefits, but It Is too Early 
to Determine Their Impact: 

Selection, Management and Oversight, and Assessment Processes Could Be 
Improved by Adopting Additional Practices: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Comments from the Department of Defense: 

Tables: 

Table 1: The Office of the Secretary of Defense Sponsored Technology 
Transition Programs: 

Table 2: Technology Transition Programs: 

Table 3: Examples of Projects: 

Table 4: Projects Completed: 

Table 5: Details of Selection Process: 

Table 6: Details on Management and Oversight: 

Figures: 

Figure 1: Funding for TTI, DACP, and Quick Reaction Fund: 

Figure 2: RAM Optic Assembly and Missile Launch: 

Figure 3: DARPA Water Purification System: 

Abbreviations: 

ACTD: Advanced Concept Technology Demonstration: 

DACP: Defense Acquisition Challenge Program: 

DARPA: Defense Advanced Research Projects Agency: 

DOD: Department of Defense: 

FCT: Foreign Comparative Testing: 

MANTECH: Manufacturing Technology Program: 

S&T: science and technology: 

TRL: technology readiness level: 

TTI: Technology Transition Initiative: 

United States Government Accountability Office: 

Washington, DC 20548: 

June 17, 2005: 

The Honorable John Warner: 
Chairman: 
The Honorable Carl Levin: 
Ranking Minority Member: 
Committee on Armed Services: 
United States Senate: 

The Honorable Duncan L. Hunter: 
Chairman: 
The Honorable Ike Skelton: 
Ranking Minority Member: 
Committee on Armed Services: 
House of Representatives: 

Over the past 3 years, the Department of Defense (DOD) has invested 
almost $69 billion[Footnote 1] on a wide range of advanced technology 
development projects--from detecting and neutralizing deeply buried 
facilities that protect weapons of mass destruction, to miniaturizing 
power and energy technologies to reduce the weight of gear soldiers 
carry into battle, to improving access to space. As we have previously 
reported, the majority of these dollars are spent within large weapons 
programs that have taken too long to get to the warfighter, in large 
part because these programs often attempt to incorporate technology 
advances that have not been proven. Invariably, this practice has 
resulted not only in large cost and schedule increases for large 
programs, but less available funding and commitment for small-scale 
development. 

Both DOD and Congress recognize these as well as other problems in 
transitioning technology and have initiated a number of programs over 
the past decade aimed at spurring and fielding smaller-scale technology 
advances. Recently, the fiscal year 2003 National Defense Authorization 
Act[Footnote 2] required DOD to establish two new programs, the 
Technology Transition Initiative (TTI) and the Defense Acquisition 
Challenge Program (DACP)--the first designed to speed transition of 
technologies from DOD's science and technology (S&T) programs to 
acquisition programs and the second designed to introduce innovative 
and cost-saving technologies from inside and outside DOD. In addition, 
DOD established the Quick Reaction Fund to rapidly field-test promising 
new technology prototypes. All three programs target relatively small- 
scale projects and their total combined annual budget is about $64 
million in fiscal year 2005--a very small portion of the overall 
dollars dedicated to technology research and development. We were 
required by the conference report accompanying the act to assess the 
implementation of the first two programs and were later asked by your 
offices to also review the Quick Reaction Fund. As discussed with your 
offices, our specific objectives were to assess (1) the impact the 
programs have had on technology transition and (2) practices that have 
helped or hindered the selection, management and oversight, and 
assessment of the programs. 

In conducting our work, we interviewed the managers of the three 
programs we reviewed and analyzed documents that detailed how the 
programs have been implemented, expected project benefits, and the 
types of performance metrics they used to gauge individual project and 
overall program success. We met with several high-level DOD officials 
who were appointed to a council to oversee the Technology Transition 
Initiative, and we observed meetings of a working group in charge of 
supporting the same initiative. In addition, we judgmentally selected 
24 of 68 projects funded by the three transition programs for more in- 
depth review, targeting those that had already been completed or were 
in the completion stages as well as assuring we had projects 
representing all of the military services and key Defense agencies. We 
conducted interviews with the managers of these 24 projects to discuss 
implementation, management, and oversight activities. We analyzed 
pertinent project documents, including original proposals and quarterly 
status reports. We also analyzed documents relating to the transition 
programs as well as broader transition issues and conducted interviews 
with officials in DOD, the Army, the Navy, the Air Force, the Marine 
Corps, the Defense Advanced Research Projects Agency, the Missile 
Defense Agency, the Defense Threat Reduction Agency, Defense 
Information Systems Agency, Special Operations Command, Joint Chiefs of 
Staff, and military department research laboratories. We relied on our 
previous best practices work, other transition program guidance, and 
general management practices as criteria when assessing the program 
offices' efforts to select, manage and oversee, and assess projects. We 
also analyzed DOD budget documents to identify program funding, as well 
as applied research and advanced technology development funding for 
fiscal years 2003 to 2009 and confirmed with DOD officials responsible 
for maintaining this information that our analysis was correct. Our 
review was conducted from June 2004 to April 2005 in accordance with 
generally accepted government auditing standards. 

Results in Brief: 

It is too soon for us to determine the impact the three programs are 
having on technology transition within DOD because, at the time we 
selected projects for review, only 11of the 68 projects that were 
funded in fiscal years 2003 and 2004 had been completed, and, of those, 
only 4 were actually providing full capability to users. Moreover, the 
three transition programs have limited measures to gauge individual 
project success and return on investment. However, the Technology 
Transition Initiative and the Defense Acquisition Challenge Program, 
which are being implemented consistent with congressional intent and 
the Quick Reaction Fund are expected to result in several benefits. For 
example, each tout benefits such as quicker fielding of new or improved 
technologies, cost savings, and in the case of the Defense Acquisition 
Challenge Program, the opportunity for DOD to use new, innovative 
technologies from smaller firms and companies that have not done 
business with DOD in the past. In addition, some sponsored technologies 
are providing valued capabilities, such as the ability to detect 
chemical and biological agents via small unmanned aircraft and new ways 
of purifying water on the battlefield or disaster zones. DOD officials 
also believe the programs are worthwhile from the standpoint of giving 
senior leaders the flexibility to rapidly address current warfighter 
needs and by providing visibility for smaller technology projects that 
may not be high enough on an acquisition program's priority list to 
receive funding. As a result, DOD plans to increase funding 
collectively for these programs from about $24 million in fiscal year 
2003 to about $94 million by fiscal year 2009. 

Whether the three programs will be successful over the long run will 
likely hinge on how well they are managed and overseen, particularly if 
DOD's investment increases as planned. Specifically, it will be 
important for the programs to have effective processes for selecting 
projects, to ensure that the best possible candidates are chosen and 
that the technologies themselves are ready for final testing and 
evaluation stages. It will also be vital that they instill effective 
management and oversight processes, so that they can identify and 
correct problems before they throw projects off track and so that they 
can sustain acquisition program commitment. In addition, given the 
importance of enhancing technology transition within DOD and the 
expectation that the investment in these programs will grow, it will be 
important for all three programs to demonstrate to others that they are 
providing a worthwhile return on investment. To date, the DACP has 
adopted a fairly disciplined and structured process for selecting and 
managing projects. While its selection process has been slowed due to 
an overwhelming response and it has not yet completed a project, the 
program has encountered few problems in managing projects. The TTI has 
also established criteria and processes, but had less success in 
marketing the program and in tracking progress of individual projects. 
The Quick Reaction Fund had the least structured process and criteria, 
believing that a high degree of flexibility is needed in order to get 
technology prototypes quickly out to the field, where they can 
immediately impact military operations. At the same time, it had some 
difficulty in selecting, managing, and tracking the status of projects. 

All three programs are continuing to strengthen their management 
processes. This report recommends that DOD develop data and measures 
that can be used to assess short-and long-term impacts of the programs 
and take other actions, as appropriate, to further strengthen 
selection, management, and oversight as investments increase. DOD 
agreed with our recommendations as they related to the DACP and TTI 
programs. However, since the Quick Reaction Fund is meant to quickly 
test a new technology, DOD did not believe the recommendations should 
apply to that program. 

Background: 

DOD relies on its research laboratories and test facilities as well as 
industry and academia to develop new technologies and systems that 
improve and enhance military operations and ensure technological 
superiority over adversaries. Yet, historically, DOD has experienced 
problems in bringing technologies out of the lab environment and into 
real use. At times, technologies do not leave the lab because their 
potential has not been adequately demonstrated or recognized. In other 
cases, acquisition programs--which receive the bulk of DOD's funding in 
research, development, testing and evaluation of technology--are simply 
unwilling to fund final stages of development of a promising 
technology, preferring to invest in other aspects of the program that 
are viewed as more vital to success. Other times, they choose to 
develop the technologies themselves, rather than rely on DOD labs to do 
so--a practice that brings cost and schedule risk since programs may 
well find themselves addressing problems related to technology 
immaturity that hamper other aspects of the acquisition process. And 
often, DOD's budgeting process, which requires investments to be 
targeted at least 2 years in advance of their activation, makes it 
difficult for DOD to seize opportunities to introduce technological 
advances into acquisition programs. In addition, it is challenging just 
to identify and pursue technologies that could be used to enhance 
military operations given the very wide range of organizations inside 
and outside of DOD that are focused on technology development and the 
wide range of capabilities that DOD is interested in advancing. 

In recognizing this array of challenges, DOD and Congress have 
established a number of "technology transition" programs, each with a 
particular focus. (See table 1.) The Advanced Concept Technology 
Demonstration (ACTD) program, for example, was initiated by DOD in 1994 
as a way to get technologies that meet critical military needs into the 
hands of users faster and at less cost than the traditional acquisition 
process. Under this program, military operators test prototypes that 
have already been developed and matured in realistic settings. If they 
find the items to have military utility, DOD may choose to buy 
additional quantities or just use the items remaining after the 
demonstration. In 1980, DOD established the Foreign Comparative Testing 
(FCT) Program to identify, evaluate, and procure technologies that have 
already been developed and tested in other countries--saving DOD the 
costly burden of maturing the technology itself. Other programs include 
those that seek to quickly identify and solve production problems 
associated with technology transition (the Manufacturing Technology 
Program--MANTECH) and to partner with the commercial sector in 
completing projects that are useful to both military and industry (the 
Dual Use Science and Technology program). Even taken together, however, 
these programs represent a very small portion of DOD dollars spent on 
applied research and advanced technology development--about $9 billion 
annually--and considerably less of total money spent on the later 
stages of technology development, which includes an additional $60 
billion spent on advanced component development and prototypes, largely 
within weapons acquisition programs. As such, they cannot single- 
handedly overcome transition problems, but rather demonstrate various 
ways to ease transition and broaden participation from the industrial 
base. 

Table 1: The Office of the Secretary of Defense Sponsored Technology 
Transition Programs: 

Program: Manufacturing Technology (MANTECH) Program; 
Purpose: Mature and validate emerging manufacturing technologies to 
facilitate production of new affordable and sustainable technologies; 
Fiscal year 2004 funding: $256 million. 

Program: Advanced Concept Technology Demonstration (ACTD); 
Purpose: Expedite the transition of maturing technologies from the 
developers to the users by having military operators and users test 
technology prototypes; 
Fiscal year 2004 funding: $223 million. 

Program: Defense Production Act Title III Program; 
Purpose: Create, maintain, modernize, or expand the productive 
capacities of domestic sources for critical components, technology 
items and industrial resources essential for national defense; 
Fiscal year 2004 funding: $78 million. 

Program: Foreign Comparative Testing (FCT) Program; 
Purpose: Identify, test, and evaluate mature foreign technologies that 
DOD can adopt rather than having to sponsor development itself; 
Fiscal year 2004 funding: $34 million. 

Program: Defense Acquisition Challenge Program (DACP); 
Purpose: Identify and introduce innovative and cost-saving technology 
or products from within DOD's science and technology community as well 
as externally into existing DOD acquisition programs; 
Fiscal year 2004 funding: $17 million. 

Program: Quick Reaction Fund; 
Purpose: Identify and rapidly field-test promising new technologies 
within DOD's budget execution years; 
Fiscal year 2004 funding: $15 million. 

Program: Technology Transition Initiative (TTI); 
Purpose: Facilitate the rapid transition of new technologies from DOD 
science and technology programs into acquisition programs; 
Fiscal year 2004 funding: $13 million. 

Program: Joint Advanced Warfighting Program; 
Purpose: Assist in developing the capabilities envisioned in Joint 
Vision 2010 by leveraging advanced technology, innovative operational 
concepts, and new organizational structures; 
Fiscal year 2004 funding: $10 million. 

Program: Dual Use Science and Technology; 
Purpose: Partner with industry to jointly fund the development of dual 
use technologies needed to maintain technological superiority on the 
battlefield; 
Fiscal year 2004 funding: $0[A]. 

Program: Tech Link; 
Purpose: Mission is threefold: (1) integrate advanced commercial-sector 
technologies into DOD systems, particularly from nontraditional defense 
contractors; (2) spin-off DOD-developed technologies to the commercial 
sector to make these technologies more affordable for military 
acquisition; and (3) establish collaborative research and development 
projects with the private sector for cost- sharing of new dual-use 
technology development; 
Fiscal year 2004 funding: $3 million. 

Program: Independent Research and Development; 
Purpose: Increase efficiency and productivity of contractor independent 
research and development activities; 
Fiscal year 2004 funding: $0[B]. 

Source: DOD. 

[A] The Air Force provided $10 million for Dual Use Science and 
Technology projects in fiscal year 2004. 

[B] OSD does not spend any money on this program. Rather, DOD compiles 
data on the amount of independent research and development spending by 
major defense contractors and reports this information annually. 

[End of table]

Three of the more recent initiatives include the TTI and DACP, both 
established by Congress in fiscal year 2003, and the Quick Reaction 
Fund, established by DOD the same year. TTI is focused on speeding the 
transition of technologies developed by DOD's S&T programs into 
acquisition programs, while DACP is focused on introducing innovative 
and cost-saving technologies developed inside and outside DOD. The 
Quick Reaction Fund is focused on field testing technology prototypes. 
All three programs are managed by DOD's Office of Defense Research and 
Engineering, which reports to the Under Secretary of Defense for 
Acquisition, Technology and Logistics. 

Together, these three programs received about $64 million in fiscal 
year 2005-a fraction of the $9.2 billion DOD invested in applied 
research and advanced technology development the same year and a 
relatively small budget compared to some of the other transition 
programs. Nevertheless, DOD has been increasing its investment in these 
programs and plans to further increase it over the next few years. (See 
figure 1.)

Figure 1: Funding for TTI, DACP, and Quick Reaction Fund: 

[See PDF for image]

[End of figure]

Table 2 highlights similarities and differences between DACP, TTI, and 
Quick Reaction Fund. Table 3 provides examples of projects that have 
already been funded. 

Table 2: Technology Transition Programs: 

Program: DACP; 
Objective: Introduce innovative and cost-saving technologies or 
products into existing acquisition programs; Other objectives of the 
program include expansion of opportunities for emerging defense 
suppliers and widening the U.S. defense industrial base; 
Scope: Any person or activity inside or outside DOD; 
Completion time frames: Up to 3 years; 12 to 24 months is preferred; 
Total fiscal year 2003 and 2004 funding: $29.83 million; 
Projects funded to date: 30 projects ranging in value from $100,000 to 
$2.54 million. 

Program: TTI; 
Objective: Speed transition of technologies from DOD science and 
technology programs to acquisition programs; 
Scope: DOD science and technology programs; 
Completion time frames: Up to 4 years; up to 24 months is preferred; 
Total fiscal year 2003 and 2004 funding: $19.27 million; 
Projects funded to date: 21 projects ranging from $323,000 to $5.33 
million. 

Program: Quick Reaction Fund; 
Objective: Field and test prototypes that respond to immediate and 
emerging warfighter needs; 
Scope: DOD science and technology programs; 
Completion time frames: 6 to 12 months; 
Total fiscal year 2003 and 2004 funding: $21.54 million; 
Projects funded to date: 17 projects ranging from $240,000 to $3.5 
million. 

Source: DOD. 

[End of table]

Table 3: Examples of Projects: 

DACP: 
* A collapsible stock for the M-4 rifle that makes it more effective in 
an urban environment and ergonomic; 
* An enhanced optics system for the Rolling Airframe Missile that will 
permit all-weather operational capability; 
* A new process for repairing titanium cracks in the B-2 aircraft that 
will improve mission readiness rates and has the potential to save DOD 
millions of dollars in replacement costs; 
* A new welding process for naval propellers that will improve weld 
repair techniques and increase the strength and quality of the casting 
in repaired areas. 

TTI: 
* An ink-pen size device that purifies water; 
* A custom-design earplug that provides blast noise protection and 
increased communication capabilities; 
* A low-cost, flame resistant coverall for infantry soldiers; 
* A new coating for H-46 helicopter engine blades that reduces the rate 
of premature engine removals related to desert operations; 
* Extending the field of view for a sensor targeting system that is 
capable of detecting, classifying, and locating weapons firing in real 
time. 

Quick Reaction Fund: 
* A greaseless M-4 rifle that is more suitable for desert operations; 
* A special material that strengthens the physical structure of an 
unmanned aerial vehicle; 
* A chemical and biological detection capability for small unmanned 
aerial vehicle; 
* An accuracy enhancement for unguided rockets. 

Source: DOD. 

[End of table]

Technology Transition Programs Offering Benefits, but It Is too Early 
to Determine Their Impact: 

The three transition programs, which are being implemented consistent 
with congressional intent, reported that benefits can already be seen 
in many projects, including improvements to performance, affordability, 
manufacturability, and operational capability for the warfighter. While 
such benefits may have eventually been achieved through normal 
processes, program officials believe the three transition programs 
enabled DOD to realize them sooner due to the immediate funding that 
was provided to complete testing and evaluation as well as attention 
received from senior managers. DOD officials also emphasized that these 
programs are calling attention to emerging technologies that have the 
potential to offer important performance gains and cost savings but, 
due to their size and relative obscurity, may otherwise be overlooked 
when competing against other, larger-scaled technologies and/or 
technologies already deemed as vital to a particular acquisition 
program's success. Another benefit cited with the DACP is an expansion 
of the Defense industrial base, because the program invites 
participation from companies and individuals that have not been 
traditional business partners with DOD. Nevertheless, it is too early 
for us to determine the impact that these programs have had on 
technology transition. At the time we selected projects to review, few 
projects had been completed. In addition, the programs had limited 
performance measures to gauge success of individual projects or track 
return on investment over time. 

The following examples highlight some of the reported benefits of 
individual projects. 

* Host Weapons Shock Profile Database--DOD spends a significant amount 
of time and resources to test new accessories (e.g., night vision 
scopes) for Special Operations Forces weapons. Currently, when new 
accessories are added, they must go through live fire testing to 
determine if they work properly and will meet reliability standards. 
This process could take several months to complete as the acquisition 
office must schedule time at a test range to complete the testing. 
Program officials must also identify and pay for an expert to conduct 
the testing and pay for ammunition that will be used in the test. The 
DACP is funding the test and evaluation of a database that will 
simulate the vibration or shock of various machine guns in order to 
test new accessories for that gun. This will eliminate almost all of 
the testing costs mentioned above and greatly reduce the amount of time 
needed for testing. The project office estimates that it will save 
almost $780,000 per year in ammunition costs alone. 

* Enhanced Optics for the Rolling Airframe Missile--The Rolling 
Airframe Missile is part of the Navy's ship self-defense system to 
counter attacks from missiles and aircraft. However, the missile 
experiences operational deficiencies in certain weather conditions, and 
the program has had problems producing components for the optics. The 
DACP is providing funding to a small business to test and evaluate a 
new sapphire dome and optics for the missile to resolve these problems. 
Program officials estimate that program funding will accelerate the 
development of a solution 1 to 2 years earlier than anticipated. If the 
DACP project is successful, an added benefit will be that the dome 
material will be readily available from manufacturers in the United 
States instead of a single overseas supplier, as is currently the case. 

Figure 2: RAM Optic Assembly and Missile Launch: 

[See PDF for image]

[End of figure]

* Water Purification System--For tactical situations in which deployed 
troops do not have quick and easy access to potable water, the pen will 
allow soldiers to treat up to 300 liters of any available, non-brackish 
water source on one set of lithium camera batteries and common table 
salt. The pen eliminates the risk of the soldiers' exposure to diseases 
and bio-chemical pollutants. TTI funding was used to purchase 
approximately 6,600 water pens for distribution to the military 
services. In addition, TTI funding enabled this item to be placed on a 
General Services Administration schedule, where approximately 8,600 
additional water pens have been purchased by DOD customers. DOD and the 
company that produces the pen donated hundreds of these systems to the 
tsunami relief effort in Southeast Asia. 

Figure 3: DARPA Water Purification System: 

[See PDF for image]

[End of figure]

* Dragon Eye--The Dragon Eye is a small, unmanned aerial vehicle with 
video surveillance capabilities used by the marines. To address the 
concerns over a chemical and biological threat to troops in Iraq, the 
Quick Reaction Fund funded the integration of a small chemical 
detection and biological collection device on the Dragon Eye. The low- 
flying Dragon Eye can tell troops in real time where and when it is 
collecting samples, and in cases where a plume is detected, it can 
determine the direction the plume is moving. According to program 
officials, Quick Reaction funding allowed the chemical and biological 
detection capability to be developed 2 years ahead of schedule. The 
technology was available to a limited number of Special Operations 
Forces at the beginning of the Iraqi conflict. 

Despite the evident benefits of certain projects, it is too early to 
determine the programs' impact on technology transition. At the time we 
selected projects for review, only 11 of 68 projects started in fiscal 
years 2003 and 2004 had been completed, and, of those, only 4 were 
currently available to warfighters. These include one TTI project--a 
miniaturized water purification system that is now being offered 
through a General Services Administration schedule to the warfighter-- 
and three projects under the Quick Reaction Fund, including the Dragon 
Eye chemical and biological sensor, planning software used by Combatant 
Commanders dealing with weapons of mass destruction targets, and 
special materials that strengthen unmanned aerial vehicles. 

Since the time we selected projects, 20 have been reported as completed 
and 13 have been reported as available to warfighters. The latest 
project completion information by program is shown in Table 4. 

Table 4: Projects Completed: 

Program: TTI; 
Projects funded since the program began: 21; 
Projects completed as of 2/28/2005: 11. 

Program: DACP; 
Projects funded since the program began: 30; 
Projects completed as of 2/28/2005: 0. 

Program: Quick Reaction Program; 
Projects funded since the program began: 17; 
Projects completed as of 2/28/2005: 9. 

Program: Total; 
Projects funded since the program began: 68; 
Projects completed as of 2/28/2005: 20. 

Source: DOD. 

[End of table]

It is important to note that, even though 20 TTI and Quick Reaction 
Fund projects are considered to be complete, not all of the 
capabilities have reached the warfighter. For example: 

* The T58 Titanium Nitride Erosion Protection is a TTI project that has 
transitioned to an acquisition program but has not yet reached the 
warfighter. The project is being developed to improve the reliability 
of T-58-16A helicopter engines used in Iraq. While the compressor 
blades are designed for 3000 operating hours, the Marine Corps has had 
to remove engines with fewer than 150 operational hours due to sand 
ingestion. The project received funding from the TTI in fiscal years 
2003 and 2004 to develop a titanium nitride coating for engine blades 
that would significantly mitigate erosion problems in a desert 
environment. According to program documents, blades with the new 
coating will be included in future production lots beginning in July 
2005. Modification kits will also be developed for retrofitting engines 
already produced. Program officials expect the project will double the 
compressor life of the engine in a sand environment and save about $12 
million in life-cycle costs through fiscal year 2012. 

* The Ping project, funded by the Quick Reaction Fund, is an example of 
a project that is considered complete, but a prototype was never field 
tested by the warfighter. The Air Force had hoped to broaden the 
capability of the microwave technology it used to identify large 
objects such as tanks or cars to also detect concealed weapons or 
explosives--such as suicide vests. However, the project was cancelled 
after some initial testing revealed that the technology was not 
accurate enough to determine the microwave signatures of small arms or 
suicide vests that could have numerous configurations and materials. 
DOD officials stated that, even though the project was unsuccessful, 
they gained a better understanding of microwave technologies and are 
continuing to develop these technologies for other applications. 

The long-term impact of the programs will also be difficult to 
determine because the technology transition programs have a limited set 
of metrics to gauge project success or the impact of program funding 
over time. While each funded project had to identify potential impact 
in terms of dollar savings, performance improvements, or acceleration 
to the field as part of the proposal process, actual impact of specific 
projects as well as the transition programs as a whole is not being 
tracked consistently. The value of having performance measures as well 
as DOD's progress in adopting them for these transition programs is 
discussed in the next section of this report. 

Selection, Management and Oversight, and Assessment Processes Could Be 
Improved by Adopting Additional Practices: 

To ensure that new technologies can be effectively transitioned and 
integrated into acquisitions, transition programs need to establish 
effective selection, management and oversight, and assessment 
processes. For example, programs must assure that proposals being 
accepted have established a sound business case, that is, technologies 
being transitioned are fairly mature and in demand and schedules and 
cost for transition fit within the program's criteria. Once projects 
are selected, there needs to be continual and effective communication 
between labs and acquisition programs so that commitment can be 
sustained even when problems arise. To assure that the return on 
investment is being maximized, the impact of programs must be tracked, 
including cost and time savings as well as performance enhancements. 
Our work over the past 7 years has found that high-performing 
organizations adopt these basic practices as a means for successfully 
transitioning technologies into acquisitions. Moreover, several larger 
DOD technology transition programs, such as the ACTD program and some 
Defense Advanced Research Projects Agency (DARPA) projects, embrace 
similar practices and have already developed tools to help sustain 
commitment, such as memorandums of agreement between technology 
developers and acquirers. Both DARPA and ACTD manage budgets that are 
considerably larger than the programs included in this review. As such, 
the level of detail and rigor associated with their management 
processes may not be appropriate for TTI, DACP, or Quick Reaction Fund. 
However, the concepts and basic ingredients of their criteria and 
guidance could serve as a useful starting point for the smaller 
programs to strengthen their own processes. 

The three programs we reviewed adopted these practices to varying 
degrees. Overall, the DACP had disciplined and well-defined processes 
for selecting and managing, and overseeing projects. The TTI had 
disciplined and well-defined processes for selecting projects, but less 
formal processes for management and oversight. The Quick Reaction Fund 
was the least formal and disciplined of all three, believing that 
success was being achieved through flexibility and a high degree of 
senior management attention. All three programs had limited performance 
measures to gauge progress and return on investment. Generally, we 
found that the more the programs adopted structured and disciplined 
management processes, the fewer problems they encountered with 
individual efforts. 

Selection: 

Success in transitioning technologies from a lab to the field or an 
acquisition program hinges on a transition program's ability to choose 
the most promising technology projects. This includes technologies that 
can substantially enhance an existing or new system either through 
better performance or cost savings and those with technologies at a 
fairly mature stage, in other words, suitable for final stages of 
testing and evaluation. A program can only do this, however, if it is 
able to clearly communicate its purpose and reach the right audience to 
submit proposals in the first place. It is also essential that a 
program have a systematic process for determining the relative 
technical maturity of the project as well as for evaluating other 
aspects of the project, such as its potential to benefit specific 
acquisition programs. Involving individuals in the selection process 
from various functions within an organization--e.g., technical, 
business, and acquisition--further helps to assure that the right 
projects are being chosen and that they will have interested customers. 
An analytical tool that can be particularly useful in selecting 
projects is a technology readiness level (TRL) assessment, which 
assesses the maturity level of a technology ranging from paper studies 
(level 1), to prototypes that can be tested in a realistic environment 
(level 7), to an actual system that has proven itself in mission 
operations (level 9). Our prior work has found TRLs to be a valuable 
decision-making tool because it can presage the likely consequences of 
incorporating a technology at a given level of maturity into a product 
development. 

As further detailed in table 5, the DACP program has a fairly robust 
selection process. The program relies on internet-based tools to 
communicate its goals and announce its selection process and ensure a 
broad audience is targeted. As a result, it receives a wide array of 
proposals from which the program office assesses their potential for 
generating improvements to existing programs as well as actual interest 
from the acquisition community. The DACP also solicits technical 
experts from inside and outside DOD to assess potential benefits and 
risks. Once the number of projects is whittled down, the program takes 
extra steps to secure commitments from acquisition program managers as 
well as program executive officers. The program's popularity, however, 
has had some drawbacks. For example, the sheer number of proposals have 
tended to overwhelm DACP staff and slowed down the selection process, 
particularly in the first year. In addition, while technology benefits 
and risks are assessed in making selection decisions, DACP does not 
formally confirm the technology readiness levels being reported. The 
TTI program also has a fairly rigorous selection process, with specific 
criteria for selection, including technology readiness, and a team of 
representatives of higher-level DOD S&T officials in charge of 
disseminating information about the program in their organization, 
assessing their organization's proposals based on TTI criteria as well 
as other criteria they developed, and ranking their top proposals. The 
program, which is focused on reaching DOD's S&T community rather than 
outside industry, had been communicating in a relatively informal 
manner and it was unclear during our review the extent to which the TTI 
was reaching its intended audience. The program, however, has been 
taking steps to strengthen its ability to reach out to the S&T 
community. In addition, TTI does not confirm TRLs. At the time of our 
review, the Quick Reaction Program selection process was much less 
structured and disciplined than DACP and TTI. This was by design, 
because the program wants to select projects quickly and get them out 
to the field where they can be of use in military operations in Iraq, 
Afghanistan, and elsewhere. However, the program experienced problems 
related to selection and as a result--for example, significant gaps in 
knowledge about technology readiness led to the cancellation of one 
project. To program officials, the risk associated with less formal 
selection is worth the benefit of being able to move rapidly evolving 
technologies into an environment where they can begin to immediately 
enhance military operations and potentially save lives. Nevertheless, 
the program is now taking steps to strengthen selection processes. 

Table 5: Details of Selection Process: 

How program is communicated; 
DACP: Each year, the program office issues an electronic broad agency 
announcement to communicate program goals and funding availability to 
both internal and external technology developers. Program officials are 
satisfied that this approach has helped them reach their target 
audience as evidenced by the large number of proposals received overall 
and also by the number received by small and non-traditional 
businesses. For example, almost 400 proposals were received during the 
first and second solicitation periods combined. Of those, about 52 
percent were submitted by small and non- traditional businesses; 
TTI: TTI established a working group comprised of senior-level S&T 
representatives from each of the military services to help disseminate 
information about program goals and funding availability to their 
respective organizations; It is unclear whether the TTI is reaching its 
intended audience. At the time of our review, six project managers we 
met with stated they found out about the TTI by happenstance and not 
through any formal mechanism. Furthermore, even though they are now 
aware of the program, many of these project managers still have not 
seen the program formally advertised throughout their respective 
organizations. The program office has been working to improve its 
ability to reach its target audience and assure that it is considering 
the best projects for funding by including presentations about the 
program at S&T forums and seminars and developing a program brochure; 
Quick Reaction Fund: Communication was informal. Each military service 
and selected defense agency was asked to submit its top 10 proposals; A 
new website was recently developed to automatically distribute the 
proposal data call and collect proposals. 

How projects are evaluated and selected; 
DACP: DACP's selection process is modeled after DOD's FCT program, 
which already has a well-defined selection process. The process begins 
with program officials performing an initial review of all proposals to 
filter out the ones that are not relevant or feasible. Next, proposals 
are reviewed by a panel of technical experts. If the acquisition 
program office and prime contractor are interested in the proposed 
technology, the acquisition program office must develop a final, more 
in-depth proposal for the program office to consider for funding. 
Overall, projects are assessed against 24 criteria, including 
acquisition funding commitment, technology readiness, and whether the 
project addresses a valid requirement, which are identified in the 
broad area announcement used to solicit proposals; However, the program 
office has had some problems processing the large volume of proposals 
received with a small staff of about 3.5 full-time equivalent 
positions. Currently, it takes about 9 months for a project to be 
selected for funding. According to the program manager, the expert 
technical review is the most time- consuming part of the process. To 
date, the DACP has had to solicit the services of over 1,000 experts 
from within the Department, private industry, and academia nationwide 
to review proposals. Program officials are now considering identifying 
thrust areas for prioritizing proposals as a way to reduce cycle time. 
This was permitted by the legislation that set up DACP; 
TTI: TTI proposals are judged against a set of eight criteria, 
including time needed to transition, technology readiness, whether the 
project satisfies a critical requirement, and willingness on the part 
of an acquisition program to share costs of transition. Working group 
members play a key role in the selection process by reviewing proposals 
from their organization and ranking their top proposals for program 
office consideration. Program officials believe this approach expedites 
the selection process because it reduces the number of proposals the 
program office needs to review and also gives the officials an idea of 
which proposals are the most important to the service or agency; 
Quick Reaction Fund: Selection was not based on well defined criteria 
at the time of our review. Instead, 2-page proposals were evaluated by 
the office running the Quick Reaction Fund with an eye toward whether 
they could bring immediate benefits to ongoing military operations. It 
usually takes 30 to 45 days between the time the solicitation is issued 
to the time projects are selected for funding. 

How TRLs are used; 
DACP: DACP guidance specifically states that technology must have been 
demonstrated using a representative model or prototype system in a 
relevant environment, the equivalent of a technology readiness level 6 
on a scale of 1-9, to be considered for funding. Technology readiness 
is also considered by technical experts reviewing proposals. However, 
DACP does not have a formal mechanism in place to ensure that the 
technology readiness levels have actually been achieved; 
TTI: One of the four criteria program officials consider during the 
selection process is whether the proposed new technology is mature. 
However, TTI does not have a formal mechanism to confirm technology 
readiness levels identified in the proposals; 
Quick Reaction Fund: Program officials have limited information about 
maturity levels. Proposals are required to disclose technical risks to 
assist in this evaluation, but do not identify specific technology 
readiness levels. We identified one project where the Quick Reaction 
Fund invested $1.5 million but later needed to stop the project because 
technology was not as mature as the project manager originally thought; 
According to the program manager, TRL maturity will be included as an 
evaluation factor in the fiscal year 2006 assessment process. 

Source: GAO analysis. 

[End of table]

Management and Oversight: 

Selecting promising projects for funding is not enough to ensure 
successful transition. Program managers must also actively oversee 
implementation to make sure that project goals are being met and the 
program is working as intended and to identify potential barriers to 
transition. They must also sustain commitment from acquirers. Moreover, 
the transition program as a whole must have good visibility over 
progress and be positioned to shift attention and resources to problems 
as they arise. 

A tool that has proven particularly useful for other established DOD 
technology transition programs is designating individuals, preferably 
with experience in acquisitions or operations and/or the S&T world, as 
"deal brokers" or agents to facilitate communication between the lab 
and the acquisition program and to resolve problems as they arise. 
DARPA employs such individuals, for example, as well as some Navy- 
specific transition programs. Both have found that these agents have 
been integral to transition success. Another tool that is useful for 
sustaining commitment from the acquirers is a formal agreement. Our 
previous work found that best practice companies develop agreements 
with cost and schedule targets to achieve and sustain buy-in and that 
the agreements are modified as a project progresses to reflect more 
specific terms for accepting or rejecting a technology. DARPA develops 
similar agreements that describe how projects will be executed and 
funded as well as how projects will be terminated if the need arises. 
The agreements are signed by high-level officials, including the 
director of DARPA and senior-level representatives of the organizations 
DARPA is working with. The ACTD program develops "implementation 
directives" that clarify roles and responsibilities of parties 
executing an ACTD, time frames, funding, and the operational parameters 
by which military effectiveness is to be evaluated. The agreements are 
also signed by high-level officials. 

DACP has fairly robust management and oversight mechanisms. Status is 
monitored via formal quarterly reporting as well as interim meetings 
which, at a minimum, involve the customer, the developer, and the DACP 
project manager. The meetings provide an opportunity to ensure the 
acquisition program is still committed to the project and to resolve 
problems. Though formal memoranda of agreements are not usually 
employed, the program establishes test and evaluation plans that detail 
pass/fail criteria so that funding does not continue on projects that 
experience insurmountable problems. TTI also employs periodic status 
reports and meetings; however, communication has not been as open. In 
two cases, projects ran into significant problems, such as loss of 
acquisition program office support in one case and logistics issues 
that had not been addressed to transition a technology smoothly in the 
other, which had not come to the attention of the TTI program office. 
As a result, the TTI office thought the projects had transitioned when 
in actuality, significant problems still needed to be addressed. Per 
legislation, TTI had also established a formal council comprised of 
high-level DOD officials to help oversee the program; however, the 
Council has only met once in 2 years, while the act requires that it 
meet at least semiannually. In addition, there is some confusion among 
Council members and others we spoke with as to what the purpose of the 
Council should be--that is, focused on TTI only or broader transition 
issues. Congressional officials expressed that they intended for the 
Council to focus on broader transition issues and how best to solve 
them. Although the Quick Reaction Fund does not require status reports 
to assess progress, project managers are required to submit after- 
action reports. However, these were not regularly reviewed by the 
office. We identified several problems that arose during transition 
that were not known to the Quick Reaction Fund program manager. The 
program manager is currently taking steps to improve the management and 
oversight of projects. For example, a website has been developed to 
help monitor and execute the program. Among other things, the website 
will allow for the automatic collection of monthly status reports. 

Table 6: Details on Management and Oversight: 

How commitment is secured; 
DACP: DACP employs a formal commitment process that includes pass/fail 
criteria that will either sustain commitment or result in the 
cancellation of a project based upon testing outcomes; DACP guidance 
includes practices that help the project manager achieve and maintain 
buy-in until a technology has been successfully transitioned. As part 
of the selection criteria, both the acquisition program office and 
prime contractor must agree to accept a technology if it is 
successfully demonstrated. For example, one of the projects we reviewed 
established pass/fail criteria for a new technology to repair titanium 
alloy cracks on the aft section of the B- 2 aircraft. Further, selected 
projects are expected to use integrated product teams (which include at 
least the vendor, developer, and DACP manager) to exchange information 
and deal with potential problems very quickly; 
TTI: TTI employs a formal commitment process; it has not used formal 
agreements; Initially, all projects must show the acquirer has included 
funding for the technology in a future budget if the project is to be 
considered for funding. For the most part, once this happens, there is 
no requirement for additional interaction through methods like formal 
agreements. We identified one project that was not able to sustain 
support from the acquisition program--the IROS3 Spartan project, which 
is intended to enhance the Navy's shipboard protection. TTI officials 
believed it had transitioned to an acquisition program, when, in fact, 
the customer had dropped its support. According to the project manager, 
the Naval Surface Warfare Center at Crane was supposed to develop the 
first block of software for the project and a contractor would be 
selected to develop later blocks. However, after successful field-
testing, the acquisition program office decided that it would be less 
risky to select a contractor to develop all three software blocks than 
to hand off the government-developed software to the contractor to 
maintain and upgrade; 
Quick Reaction Fund: No formal commitment process used because the 
program goal is to demonstrate the military utility of emerging new 
technologies through field testing, and not necessarily to transition 
those technologies at this time. 

How status is tracked; 
DACP: Project managers are required to submit quarterly progress 
reports and final reports once a project has been completed. In 
addition to these reports, periodic meetings are held with various 
participants to assess progress. Reports contain data on project 
accomplishments, planned actions for the next quarter, issues that need 
to be addressed, the transition strategy, and the current funding plan. 
All quarterly reports are submitted and maintained on an electronic 
system that can be accessed by program officials and the project 
manager; 
TTI: Periodic status reports are required and interim progress meetings 
are held, though not with the same range of representation as DACP; We 
identified one project--the Terminal Attack Communications Earplug 
System--where problems had arisen during transition that had not come 
to the attention of TTI officials. Specifically, problems related to 
how the earplug's use could be actualized (for example, training 
technicians to pour the new, custom ear molds and what organization 
would pay for the training) had not been brought to the attention of 
the TTI office. In fact, the office had thought the transition had gone 
smoothly; 
Quick Reaction Fund: After-action reports are required, though not 
always reviewed for purposes of tracking progress; At the time of our 
review, the Quick Reaction Fund manager had problems addressing 
concerns identified in status reports and obtaining reports from some 
project managers. For example, the program manager was unaware that one 
project, thought to have been successfully field tested, actually ran 
out of funding prior to field testing. Although the project manager 
reported that the project had not been completed because it ran out of 
funding, the Quick Reaction Fund manager did not have enough time to 
thoroughly review the report; The program manager expects the newly 
established web-based system to improve his ability to track project 
progress. 

Use of deal brokers; 
DACP: No formal use of "deal brokers", though involvement of vendors, 
developers, and customers in status discussions have helped to identify 
and resolve transition problems; 
TTI: No formal use of "deal brokers"; 
Quick Reaction Fund: No formal use of "deal brokers." 

Source: GAO analysis. 

[End of table]

Assessment: 

Though the transition programs we reviewed are relatively small in 
scale compared to other transition programs in DOD, the government's 
investment is still considerable and it will continue to grow if DOD's 
funding plans for the programs are approved. As a result, it is 
important that these programs demonstrate that they are generating a 
worthwhile return on investment--whether through cost savings to 
acquisition programs, reduced times for completing testing and 
evaluation and integrating technologies into programs, and/or enhanced 
performance or new capabilities. Developing such information can enable 
transition program managers to identify what is or is not working well 
within a program; how well the program is measuring up to its goals, as 
well as to make trade-off decisions between individual projects. On a 
broader level, it can enable senior managers and oversight officials to 
compare and contrast the performance of transition programs across DOD. 

Finding the right measures to use for this purpose is challenging, 
however, given the wide range of projects being pursued, the different 
environments to which they are being applied, and difficulties 
associated with measuring certain aspects of return on investment. For 
example, measuring long-term cost savings could be problematical 
because some projects could have impacts on platforms and systems that 
were not part of the immediate transition effort. As a result, the best 
place to start may be with high-level or broad metrics or narratives 
that focus on the spectrum of benefits and cost savings being achieved 
through the program, complemented by more specific quantifiable metrics 
that do not require enormous efforts to develop and support, such as 
time saved in transition or short-term cost savings. At this time, 
however, the transition programs have limited measures to gauge 
individual project success and program impact or return on investment 
in the long term. At best, they are collecting after action reports 
that describe the results of transition projects, and occasionally 
identify some cost savings, but not in a consistent manner. In 
addition, there are inconsistencies in how the reports are being 
prepared, reviewed, and used. The Quick Reaction Fund program manager, 
in fact, had trouble just getting projects to submit after action 
reports. 

Officials from all three transition programs we reviewed as well as 
higher level officials agreed that they should be doing more to capture 
information regarding return on investments for the programs. They also 
agreed that there may already be readily available starting points 
within DOD. For example, the Foreign Comparative Testing Program has 
established metrics to measure the health, success, and cost- 
effectiveness of the program and has developed a database to facilitate 
return on investment analyses. The program also captures general 
performance enhancements in written narratives. The program has refined 
and improved its metrics over time and used them to develop annual 
reports. The specific metrics established by the FCT program may not be 
readily transferable to DACP, TTI, or the Quick Reaction Fund because 
the nature of FCT projects is quite different--technologies themselves 
are more mature and costs savings are achieved by virtue of the fact 
that DOD is essentially avoiding the cost of developing the 
technologies rather than applying the technologies to improve larger 
development efforts. However, the process by which the program came to 
identify useful metrics as well as the automated tools it uses could be 
valuable to the other transition programs. In addition, DOD has asked 
the Naval Post Graduate School to study metrics that would be useful 
for assessing the ACTD program. The results of this study may also 
serve as a starting point for the transition programs in developing 
their own ways to assess return on investment. 

Conclusions: 

The ability to spur and leverage technological advances is vital to 
sustaining DOD's ability to maintain its superiority over others and to 
improve and even transform how military operations are conducted. The 
three new transition programs are all appropriately targeted on what 
has been a critical problem in this regard--quickly moving promising 
technologies from the laboratory and commercial environment into actual 
use. Moreover, by tailoring processes and criteria to focus on 
different objectives, whether that may be saving time or money or 
broadening the industrial base, DOD has had an opportunity to 
experiment with a variety of management approaches and criteria that 
can be used to help solve transition problems affecting the 
approximately $69 billion spent annually on advanced stages of 
technology development. Already, it is evident that an element missing 
from all three programs is good performance measurement. Without having 
this capability, DOD will not be able to effectively assess which 
approaches are working best and whether the programs individually or as 
a whole are truly worthwhile. In addition, it is evident that having 
well-established tools for selecting and managing projects as well as 
communicating with technology developers and acquisitions helps 
programs to reduce risk and achieve success, and that there are 
opportunities for all three programs for strengthening their 
capabilities in this regard. In light of its plans to increase funding 
for the three programs, DOD should consider actions to strengthen 
selection and management capabilities, while taking into account 
resources needed for implementing them as well as their impact on the 
ability of the programs to maintain flexibility. 

Recommendations for Executive Action: 

We recommend that the Secretary of Defense take the following five 
actions: 

To optimize DOD's growing investment in the Technology Transition 
Initiative, the Defense Acquisition Challenge Program, and the Quick 
Reaction Fund, we recommend that the Secretary of Defense direct the 
Under Secretary of Defense (Acquisition, Technology, and Logistics) to 
develop data and measures that can be used to support assessments of 
the performance of the three transition programs as well as broader 
assessments of the return on investment that would track the long-term 
impact of the programs. DOD could use measures already developed by 
other transition programs, such as FCT, as a starting point as well as 
the results of its study on performance measurement being conducted by 
the Naval Post Graduate School. To complement this effort, we recommend 
that DOD develop formal feedback mechanisms, consisting of interim and 
after action reporting, as well as project reviews if major deviations 
occur in a project. Deviations include, but are not limited to, changes 
in the technology developer, acquirer, or user, or an inability for the 
technology developer to meet cost, schedule, or performance parameters 
at key points in time. 

We also recommend that the Secretary of Defense direct the Under 
Secretary of Defense (Acquisition, Technology, and Logistics) to 
implement the following, as appropriate, for each of the transition 
programs: (1) formal agreements to solidify up-front technology 
development agreements related to cost, schedule, and performance 
parameters that must be met at key points in time and (2) confirmation 
of technology readiness levels as part of the proposal acceptance 
process. 

In addition, we recommend that DOD identify and implement mechanisms to 
ensure that transition program managers, developers, and acquirers are 
able to better communicate to collectively identify and resolve 
problems that could hinder technology transition. There may be 
opportunities to strengthen communication by improving the structure 
and content of interim progress meetings and possibly even designating 
individuals to act as deal brokers. 

Lastly, as DOD considers solutions to broader technology transition 
problems, we recommend that Secretary of Defense direct the Under 
Secretary of Defense (Acquisition, Technology, and Logistics) to assess 
how the Technology Transition Council can be better used. 

Agency Comments and Our Evaluation: 

DOD provided us with written comments on a draft of this report. DOD 
partially concurred with four of the five recommendations and concurred 
with one recommendation. The reason DOD only partially concurred with 
four of the recommendations is because it does not believe the Quick 
Reaction Fund fits the definition of a transition program. However, we 
continue to believe it is important for DOD to institute better 
management controls and have better visibility of the Quick Reaction 
Fund as it increases its investment in this program over the next 
several years. DOD comments appear in appendix I. 

DOD partially concurred with our recommendation that the programs 
develop data and measures that can be used to support assessments of 
the performance of the three transition programs as well as broader 
assessments of return on investment that would track the long term 
impact of the programs. DOD agreed that performance measures for the 
DACP and TTI programs could be improved but does not believe that 
measuring the impact of the Quick Reaction Fund is necessary because it 
does not technically fit the definition of a transition program. We 
disagree. DOD should track the progress of its various programs to 
determine if the programs are worthwhile and should be continued, if 
the program should receive additional funding, or if changes should be 
made in the selection or implementation process that could result in 
better outcomes. Further, failure to track even the most basic 
information, such as the number of projects completed, could result in 
a lack of ability to manage the program properly and poor stewardship 
of taxpayer money. 

DOD partially concurred with our recommendation that the three programs 
develop formal feedback mechanisms consisting of interim and after 
action reporting, as well as project reviews if major deviations occur 
in a project. DOD agrees that the TTI and DACP can be improved and has 
recently taken steps in this regard. However, DOD believes that due to 
the limited scope and duration of Quick Reaction Fund projects, formal 
feedback mechanisms may not be necessary for this program. We believe 
that regular feedback on the progress of each program is important to 
help program managers mitigate risk. As stated in the report, the Quick 
Reaction Fund program manager was unaware that one project ran out of 
funding prior to field testing the technology. Had the program manager 
been aware of the problem, money that had not yet been allocated could 
have been used to finish the project. In addition, based upon our 
discussions with the current program manager, DOD is planning to 
require monthly status reports for funded projects. 

DOD partially concurred with our recommendation that the programs 
implement, as appropriate: (1) formal agreements to solidify up-front 
technology development agreements related to cost, schedule, and 
performance parameters that must be met at key points in time and (2) 
confirmation of technology readiness levels as part of the proposal 
acceptance process. DOD indicated that it recently implemented 
Technology Transition Agreements for the TTI, and the DACP program also 
uses formal agreements. However, DOD does not believe formal agreements 
are necessary for the Quick Reaction Fund because it is not intended to 
be a transition program. Also, it does not believe TRLs should be a 
factor in the proposal acceptance process. As stated in the report, we 
agree that formal agreements may not be appropriate for Quick Reaction 
Fund projects. However, TRLs should be considered during the selection 
process. Since the goal of this particular program is to prototype a 
new technology in 12 months or less, it is important that DOD has some 
assurance that the technology is ready to be field tested. As discussed 
in the report, the Quick Reaction Fund had to cancel one project, after 
$1.5 million had already been spent, because it had only achieved a TRL 
3. Had the selecting official known the TRLs of each proposed project 
during the selection phase, he may have decided to fund another, more 
mature project instead. 

DOD also partially concurred with our recommendation that the programs 
identify and implement mechanisms to ensure that transition program 
managers, developers, and acquirers better communicate and collectively 
identify and resolve problems that could hinder technology transition. 
DOD established a Transition Overarching Integrated Product Team 
earlier this year to provide the necessary oversight structure to 
address this issue, but does not believe this recommendation applies to 
the Quick Reaction Fund program. We believe that if DOD receives 
monthly status reports on the Quick Reaction Fund, as planned by the 
program manager, it should be in a good position to identify and 
resolve problems that could hinder the testing of new technology 
prototypes. 

DOD concurred with our recommendation that the Under Secretary of 
Defense (Acquisition, Technology and Logistics) assess how the 
Technology Transition Council can be better used as DOD considers 
solutions to broader technology transition problems. Although DOD did 
not indicate how it plans to do this, the Deputy Under Secretary of 
Defense (Advanced Systems and Concepts) has a goal that the Council not 
limit itself to just the TTI program, but look at broader technology 
transition issues across DOD. 

We are sending copies of this report to the Secretary of Defense, the 
Director of the Office of Management and Budget, and interested 
congressional committees. We will also make copies available at no 
charge on the GAO Web site at http://www.gao.gov. 

If you or your staff have any questions concerning this report, please 
contact me at (937) 258-7915. Key contributors to this report were 
Cristina Chaplain, Cheryl Andrew, Art Cobb, Gary Middleton, and Sean D. 
Merrill. 

Signed by: 

Michael J. Sullivan: 
Director: 
Acquisition and Sourcing Management: 

[End of section]

Appendix I: Comments from the Department of Defense: 

DIRECTOR OF DEFENSE RESEARCH AND ENGINEERING: 
3030 DEFENSE PENTAGON: 
WASHINGTON, D.C. 20301-3030:

Mr. Michael Sullivan:
Director, Acquisition and Sourcing Management:
U.S. Government Accountability Office:
441 G Street, NW: 
Washington, DC 20548:

MAY 23 2005:

Dear Mr. Sullivan:

This is the Department of Defense (DoD) response to the General 
Accountability Office (GAO) draft report, `DEFENSE TECHNOLOGY 
DEVELOPMENT: Management Process Can Be Strengthened for New Technology 
Transition Programs', dated May 9, 2005, as revised (GAO Code 120408/ 
GAO-05-480).

We believe a definition for "transition" should be part of this report. 
From the Department's perspective in the context of this GAO report, 
"transition" occurrs when a project/program moves from using only R&D 
funding to using procurement and/or sustainment funding to provide the 
required/desired capability.

While we generally concur with the report's recommendations, we believe 
the Quick Reaction Fund (QRF) exists to quickly test, accelerate and/or 
field an emerging and/or immediate capability that may not require 
further procurement or sustainment. Thus, QRF should not be viewed as a 
"transition program". Comments are provided at the Enclosure.

The staff has worked closely with the GAO team and appreciates their 
inputs to help us strengthen our technology transition programs. Thank 
you for the opportunity to comment on the subject draft report.

Sincerely,

Signed by: 

Ronald M. Sega:

Enclosure: As stated:

GAO DRAFT REPORT - DATED MAY 9, 2005 GAO CODE 120408/GAO-05-480:

"DEFENSE TECHNOLOGY DEVELOPMENT: MANAGEMENT PROCESS CAN BE STRENGTHENED 
FOR NEW TECHNOLOGY TRANSITION PROGRAMS":

DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS:

NOTE: The Quick Reaction Fund (QRF) exists to quickly test, accelerate 
and/or field an emerging and/or immediate capability that may not 
require further procurement or sustainment. Thus, QRF should not be 
viewed as a "transition program".

RECOMMENDATION 1: The GAO recommended that the Secretary of Defense 
direct the Under Secretary of Defense (Acquisition, Technology and 
Logistics) to develop data and measures that can be used to support 
assessments of the performance of the three transition programs as well 
as broader assessments of return on investment that would track the 
long term impact of the programs. (p. 24/GAO Draft Report):

DOD RESPONSE: Partially Concur.

Two of the programs under review (Defense Acquisition Challenge Program 
(DACP) and Technology Transition Initiative (TTI)) currently collect 
data and have measures that support assessments of their performance; 
however these can and are being strengthened. The Quick Reaction Fund 
(QRF) is not a transition program, but rather a program to quickly 
test, accelerate and/or field an emerging and/or immediate capability 
that may not require further procurement or sustainment. Removal of 
references to the QRF as a "transition program" throughout the report 
would enable the Department to fully concur with the GAO 
recommendation. 

RECOMMENDATION 2: The GAO recommended that the Secretary of Defense 
direct the Under Secretary of Defense (Acquisition, Technology and 
Logistics) to develop formal feedback mechanisms consisting of interim 
after action reporting, as well as project reviews if major deviations 
occur in a project. Deviations include, but are not limited to, changes 
in the technology developer, acquirer, or user or an inability for the 
developer to meet cost, schedule, or performance parameters at key 
points in time. (p. 25/GAO Draft Report):

DOD RESPONSE: Partially Concur.

Formal feedback mechanisms do exist for two of the three programs under 
review; however their structure and implementation can be improved. The 
Department established a Transition Overarching Integrated Product Team 
(OIPT) in February 2005 within USD(AT&L) that is addressing management 
and oversight of transition projects which will include the Defense 
Acquisition Challenge Program (DACP) and the Technology Transition 
Initiative (TTI). To reiterate, the Quick Reaction Fund (QRF) is not a 
transition program, but rather a program of limited scope and duration 
to quickly test, accelerate and/or field an emerging and/or immediate 
capability which may not lend itself to reporting under formal feedback 
mechanisms and project reviews.

RECOMMENDATION 3: The GAO recommended that the Secretary of Defense 
direct the Under Secretary of Defense (Acquisition, Technology and 
Logistics) to implement the following, as appropriate, for each of the 
transition programs: (1) formal agreements to solidify up-front 
technology development agreements related to cost, schedule, and 
performance parameters that must be met at key points in time and (2) 
confirmation of technology readiness levels as part of the proposal 
acceptance process. (p. 25/GAO Draft Report):

DOD RESPONSE: Partially Concur.

With respect to the use of formal agreements to solidify agreements 
between the S&T and Acquisition communities, TTI recently implemented 
Technology Transition Agreements and the DACP uses Program Manager 
Letters of Endorsement. To reiterate, the Quick Reaction Fund (QRF) is 
not a transition program, but rather a program of limited scope and 
duration to quickly test, accelerate and/or field an emerging and/or 
immediate capability which may not lend itself to developing formal 
agreements related to cost, schedule, and performance, as well as 
developing technology readiness levels (TRL) as a part of the proposal 
acceptance process. By the nature of the QRF, TRLs should not be a 
factor.

RECOMMENDATION 4: The GAO recommended that the Secretary of Defense 
direct the Under Secretary of Defense (Acquisition, Technology and 
Logistics) to identify and implement mechanisms to ensure that 
transition program managers, developers, and acquirers are able to 
better communicate and to collectively identify and resolve problems 
that could hinder technology transition. (p. 25/GAO Draft Report):

DOD RESPONSE: Partially Concur.

As described in Table 6 (page 22) of the GAO report, some mechanisms 
were already in place. With the help of the GAO audit, the Department 
has identified additional measures to improve communications and 
resolve problems that could hinder technology transition. Additionally, 
the Department established a Transition Overarching Integrated Product 
Team (OIPT) in February 2005 within USD(AT&L) which will provide the 
necessary oversight structure to address this issue. The Quick Reaction 
Fund (QRF) is not a transition program, but rather a program to quickly 
test, accelerate and/or field an emerging and/or immediate capability 
that may not require further procurement or sustainment.

RECOMMENDATION 5: The GAO recommended that the Secretary of Defense 
direct the Under Secretary of Defense (Acquisition, Technology and 
Logistics) to assess how the Technology Transition Council can be 
better used as DoD considers solutions to broader technology transition 
problems. (p. 25/GAO Draft Report):

DOD RESPONSE: Concur.

The TTI program was established by Congress and prescribes a Technology 
Transition Council for advice and assistance. As discussed with the 
GAO, it is the DUSD (AS&C) goal that the Council not limit itself to 
just the TTI program, but looks at broader technology transition issues 
across the Department. 

[End of section]

FOOTNOTES

[1] This represents funding for applied research, advanced technology 
development projects, and advanced component and development 
prototypes. It does not include basic research. 

[2] Bob Stump National Defense Authorization Act for Fiscal Year 2003, 
P.L. 107-314, §§ 242 and 243, Dec. 2, 2002. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: