Appendix B:

Advanced Technology Program Project Selection Process

The Advanced Technology Program selects projects through a rigourous competitive process. Project proposals are accepted only in response to specific competition solicitations published in Commerce Business Daily and widely publicized. ATP general competitions are open to proposals from any field of technology. Focused program competitions are open to any proposal that falls within the general scope of the particular program and supports the technical and business goals of the program.

Selection Criteria

Within the basic competition limits, all proposals, regardless of the type of competition, are evaluated against five weighted criteria. Only these criteria are used in evaluating proposals:

Scientific and Technical Merit of the Proposal (30 percent)

• the quality, innovativeness, and cost-effectiveness of the proposed technical program;

• the appropriateness of technical risk and feasibility of the product, i.e., is there a sufficient knowledge base to justify the level of technical risk involved and is the risk commensurate with the potential payoff;

• the coherency of technical plan, the clarity of the vision of the technical objectives, and the degree to which the technical plan meets program goals;

• the adequacy of systems-integration and multi-disciplinary planning, including integration of appropriate downstream or upstream production, manufacturing, quality assurance, and customer service requirements; and

• the potential for broad impact on U.S. technology and knowledge base.

No proposal is funded unless the program determines that it has high scientific and technical merit, no matter how meritorious the proposal might be with respect to the other selection criteria.

Potential Broad-based Economic Benefits of the Proposal (20 percent)

Including:

• the potential to improve U.S. economic growth;

• the timeliness of the proposal. i.e., that the potential project results will not occur too late or too early to be competitively useful;

• the degree to which ATP support is essential for the achievement of the broad-based benefits from the proposed R&D and appropriateness of proposed R&D for ATP support (this factor takes into consideration the likelihood of the results being achieved in the same general time frame by the proposer or by other U.S. researchers without ATP support, and whetherother feeral agencies or other sponsors are already funding very similar kinds of work); and

• the cost effectiveness of the proposal (probable benefits relateive to costs).

Adequacy of Plans for Eventual Commercialization (20 percent)

Including:

• evidence that if the project is successful, the proposer will pursue further development of the technology toward commercial application, either through their own orgnaziations(s) or through others; and

• the degree to which the proposal identifies potential applications of the technology and provides evidence that the proposer has credible plans to assure prompt and widespread use of the technology if the R&D is successful, and to ensure adequate protection of the intellectual property of the participant(s) and, as appropriate, by other U.S. businesses.

Proposer's Level of Commitment and Organizational Structuer (20 percent)

Including:

• the level of commitment of the proposer as demonstrated by contribution of personnel, equipment, facilities, and cost sharing; the extent to which the proposer assigns the company's best people to the project; and the priority given to this work vis-a-vis other projects;

• the extent to which the venture has been structured (vertical integration, horizontal integration, or both) so as to include sufficient participants possessing all of the skills required to complete successfully the proposed work;

• for joint ventures, appropriate participation by small businesses;

• the appropriateness of subcontrator/supplier/collaborator participation and relationships to achieve project goals; and

• the clarity and appropriateness of management plan, the extent to which the proposers have clarified who is responsible for each task and the chain of command, and the extent to which those responsible for the work have adequate authority and access to higher level management.

Experience and Qualifications of the Proposing Organization(s) (10 percent)

Including:

• the adequacy of proposer's facilities, equipment, design, and manufacturing tools and other tecnical, financial, and administrative resources to accomplish the proposed program objectives (this factor includes consideration of resources possessed by subcontractors to the proposer or other collaborators);

• the quality and appropriateness of the technical staff to carry out the proposed work program and to identify and overcome barriers to meeting project objectives; and

• the past performance of the company or joint venture members in carrying out similar kinds of efforts successfully, including technology application (consideration of this factor in the case of a start-up company or new joint venture takes into account the past performance of the key people in carrying out similar kinds of efforts).

Evaluation Process

At the beginning of each ATP competition, a Source Evaluation Board (SEB) is formed to rank the proposals. A typical general competition SEB consists of senior-level federal scientists, engineers, and managers with broad experience and a variety of different technical backgrounds, e.g., electrical engineers, chemists, chemical engineers, biotechnologists, materials scientists, manufacturing engineers, and computer scientists, as well as specialists with backgrounds in business and economics. A typical focused program competition SEB is smaller, with more specialized experts in the subject area of the competition.

The ATP supplements the SEB with outside technical reviews, generally federal government experts in the specific technology of the proposal. Independent business experts also are hired on a consulting basis. These business experts include high-tech venture capitalists, people who teach strategic business planning, retired corporate executives from large and small high-tech businesses, economists, and business-development specialists.

The selection of technical and business reviewers is done carefully. All SEB members and outside reviewers must sign non-disclosure statements, agree to protect proprietary information, and certify that they have no conflicts of interest.

All proposals received by the competition deadline are first screened for conformance to the ATP regulations. Historically, about 10 percent of the proposals have been rejected at the screening stage because the proposal is incomplete, inappropriate for the ATP (e.g., does not involve any technical challenge or has no commercialization plan), clearly outside the technical scope if submitted under a focused program competition, or because the proposer has not complied with the ATP rules (e.g., a single company unwilling to agree to pay indirect costs). Proposals with obvious, major flaws in either the technical or business plans may be rejected by the SEB at this stage. Proposals determined to be complete then proceed to technical and business reviews.

Technical Review

Proposals are evaluated with regard to the technical-related selection criteria by technical experts. Multiple, independent technical reviews are conducted. The ATP reserves the right to use previously completed reviews for resubmitted proposals that were evaluated during a previous competition and have not been revised significantly.

When the technical reviews of each proposal are completed, the proposal and the reviewers' comments are considered by a Technical Panel consisting of the technically oriented members of the SEB plus any technical consultants they may choose to involve. This Technical Panel serves as a standing committee for the duration of a specific competition.

The Technical Panel members deliberate each proposal individually and determine its technical merit. The key technical strengths and weaknesses of each proposal are documented, and a recommendation is made to the full SEB.

Business Review

The Business Panel members deliberate each proposal individually and determine its business merit. The key business strengths and weaknesses of each proposal are documented, and a recommendation is made to the full SEB.

Oral Reviews of Semifinalists

Following completion of the technical and business reviews, the proposals are evaluated by the full SEB. The proposers whose proposals are deemed by the SEB to be most meritorious with regard to all five selection criteria are invited to NIST for oral reviews. This determination is made in consideration of available funding and the requested funding for the highest-quality proposals. Proposals not identified as semifinalists are removed from further consideration. In general competitions, roughly 15 percent of the submitted proposals reached the oral review stage.

A limited number (no more than four for single companies and no more than five for joint ventures) of representatives is allowed to participate in this oral review at NIST. The ATP may request a site visit, if deemed appropriate. The oral review focuses on detailed technical and business questions. Proposers are encouraged to update the SEB briefly on new technical or business developments since submission of the proposal, but the majority of the one- to two-hour review is devoted to questions and answers.

SEB Ranking

Final Selection

Final decisions regarding awards are made by a Source Selecting Official, named by NIST for each competition, who receives the ranked list of proposals from the SEB. The NIST Director or Deputy Director served as the SSO from the inception of ATP until 1993. Since then, the ATP Director and other senior managers in ATP have held this responsibility. Historically, about half to two-thirds of the proposals that underwent oral reviews received awards.

Return to main page. 

Date created: April 16, 1996
Last updated: May 19, 2005