Procedural
Requirements
Effective Date: December 20, 2001
Expiration Date: April 20, 2009
|
|
NASA Procedural Requirements |
NPR 7500.1 Effective Date: December 20, 2001 Expiration Date: April 20, 2009 |
| | TOC | Change History | Preface | Chp1 | Chp2 | Chp3 | Chp4 | Chp5 | Chp6 | Chp7 | Chp8 | AppdxA | AppdxB | AppdxC | AppdxD | Fig2-1 | FigB-1 | FigB-2 | FigC-1 | ALL | |
Metrics and trend indicators are a major tool for managing the effectiveness of NASA's technology commercialization process. At this time, technology commercialization metrics and trend indicators will focus on "output" from the technology commercialization process; e.g. how many new technologies were identified; how many partnerships were implemented; and how many partnerships resulted in success stories. Studies are underway to determine if and what metrics/indicators could be used to evaluate the "outcomes" from the technology commercialization process; e.g., what economic impacts are resulting from our "outputs." This NPR does not anticipate establishing metrics and indicators which focus on "process efficiencies," e.g. how long does it take to process a "partnership lead." However, each CTO has the option of defining and using additional metrics and indicators at their respective Centers.
Given the Government Performance and Results Act (GPRA), it is essential to correlate and directly trace technology commercialization performance to NASA's resource investments. This is particularly true given that over 80% of NASA's resources are invested in contracts, grants and agreements. This correlation and traceability is established and maintained via monthly data updates from NASA's financial and procurement data system. Each month, summary financial and procurement data is obtained and integrated into each Centers' NASATechTracS system.
All metrics and indicators are collected, calculated and maintained in NASATechTracS. A standard metrics module is contained in each Center's NASATechTracS system. Each month the metrics module automatically:
(1) Calculates the metrics and indicators for that Center's activities;
(2) Creates reports that can be utilized by designated Center and Enterprise managers; and
(3) Updates the Agencywide NASATechTracS server.
Technology commercialization metrics and trend indicators will also be available via the Knowledge Integration and Management System (KIMS). KIMS is a major component of the NASATechTracS network and a primary interface for NASA managers (see paragraph 2.3.2).
As addressed in Chapter 7, there are five performance categories. Each of these relates to one of the major technology commercialization process elements shown in Figure 2-1 of Chapter 2.
This category is directed towards the initial process element in Chapter 3 - identifying those NASA activities which have commercial potential. Activities are defined as those contracts, grants or agreements for which budgetary obligations or costing activity is occurring. This is determined with an automated procedure in NASATechTracS, which utilizes the monthly data inputs from NASA's financial and procurement system. NASA invests approximately $12 billion annually in these activities. The metrics process will refer to these as NASA's extra-mural activities. Centers can also include other activities, which are not supported with contracts, grants or agreements such as Memorandum of Agreements (MOAs) and Memorandum of Understanding (MOUs). NASA also invests approximately $2 billion in support of its in-house workforce and activities. The metrics process will refer to these as NASA's intra-mural activities. The commercial assessment process does not address NASA's intra-mural activities due to the difficulty in tracing NASA's intra-mural resources to programs and projects. Once NASA has implemented its full cost accounting process, the commercial assessment process will be revised to incorporate the intra-mural activities and a revision to this NPR will be issued.
Each NASA activity manager is responsible for assessing their activities as having, or not having, commercial potential. As discussed in Chapter 3, having some likelihood of yielding an innovative technological asset is the minimum criteria for demonstrating commercial potential. As directed by NPD 7500.2, technological assets include new technologies, innovations, facilities and expertise. At this time, however, this NPR is focusing on new technologies and innovations. As discussed in Chapters 3 & 4 new technologies and innovations include both "reportable items" and "subject inventions" developed under NASA contracts, grants and cooperative agreements. Work is underway to further define how facilities and expertise shall be incorporated into the technology commercialization process. Future updates to this NPR will be issued to address these areas.
NASA activity managers can provide their assessment using the KIMS module of the NASATechTracS system. By registering at KIMS (http://kims.larc.nasa.gov) the manager can log in and assess the status of their activities. Once logged in, the manager can select the "COTR" item on the menu bar as shown below.
 |
Until assessed, each activity is shown as having unassessed/unkown commercial potential by default. If the manager assesses the activity as having "no potential", then no additional information is required. If the activity is assessed as having commercial potential - then some additional data is required at the 7-digit UPN level. The 7-digit data is required in order to correlate "assessments" with the appropriate Strategic Enterprise. Once an initial assessment is baselined, the NASA activity manager should update it whenever new factors warrant it - but at least annually.
As new technologies and innovations are reported, NASATechTracS automatically correlates these with their source activity and financial data. Again, this allows the technologies to be correlated with NASA's Strategic Enterprises.
The specific trend indicators currently calculated for this category are:
(1) Activities with Potential (trend indicator) - this indicator is a percentage which compares the fiscal year $$s obligated to that activity with its total obligations for the year. At an agency-wide level, it provides a general understanding of what portion of NASA activities contain commercial potential. This measurement is calculated at the 7- digit UPN level for every NASA contract, grant and agreement. Purchase orders under $50K are not included.
(2) Activities with No Potential (trend indicator) - this is the inverse of the above indicator.
(3) Unassessed/Unknown(trend indicator) - this indicator shows that either the activity has not been assessed or has been assessed and the manager is unsure as to whether the activity has potential.
(4) Activities with New Technologies (trend indicator) - this is the percentage of those activities defined in (1) above which have actually reported at least one new technology or innovation.
(5) Activities without New Technologies (trend indicator) - this is the inverse of item (4) above.
Again, each manager can use KIMs to both provide the assessment data and to see the status of their activities. Each CTO can assist managers in getting access to, and using, KIMS.
This category is directed towards determining how well the "develop and report technologies" process element (in Chapter 3) is being performed. Specifically, it tracks the output status of those activities assessed as having commercial potential and which have actually produced/reported new technologies as discussed above. As new technologies are reported, NASATechTracS will automatically notify the NASA activity manager or designated recipient via email. The email will contain an electronic link (URL) which will take the manager to the new technology listing where the manager can review and concur that the new technology is associated with their activity. From that point forward NASATechTracS will automatically track and compute the applicable indicators and metrics described below. If the manager has any questions they can contact that Center's New Technology Representative.
Tracking the status of these new technologies is extremely important in that they are an essential ingredient for effective commercial technology partnerships. Because of this importance, NASA is currently using 5 measurements to determine the performance status of this category. Four of these are trend indicators and one is a GPRA metric as follows:
(1) Activities With New Technologies (trend indicator) - this indicator measures the percentage of those activities which have commercial potential which have actually reported a new technology or innovation. Again as used here, activities are those contracts, grants and agreements which are incurring budgetary obligations and/or costing and which have been assessed as having commercial potential.
(2) Year-to-Date Technology Portfolio (trend indicator) - this indicator shows the number of new technologies that have been reported to-date for the fiscal year. Currently, it is broken down into two categories.
- Extra-mural items are those from contracts, grants and agreements. These items are reported by both Center and Strategic Enterprises.
- Intra-mural items are those directly from NASA's in-house activities, reported by "Center."
(3) Total Current Technology Portfolio (trend indicator) - this indicator shows the total number of new technologies and innovations that NASA considers most current. The reasoning behind this "currency concept," is that NASA most likely has ready access to the technical points of contact for those new technologies and innovations. Again, this indicator is broken down into extra-mural and intra-mural items. For extra-mural items, it is all those new technologies from the preceding five years and any others which are related to ongoing extra-mural activities. For intra-mural items, it is all those within the preceding five years
(4) Technologies Released to the Public (metric) - this indicator represents the total number of new technologies and innovations that have been released for public access for the fiscal year- to-date. Specifically, it consists of those items made available to the public via the NASA TechFinder web site. Public availability is important to both our marketing and outreach activities. In addition, it is also one of the items reported annually to GPRA in support of NASA's Communicate Knowledge (CK) process. The specific goal is to release 100 items each year. Because we have a specific target, it has been designated a "metric." Currently this metric is under review. Any changes will be incorporated in future releases of this NPR.
(5) Portfolio Available to the Public (trend indicator) - this indicator shows the percentage of the total current technology portfolio (item (3) above) that is available to the public via TechFinder.
This category focuses on the development and implementation of the partnership process elements (Chapter 5). Specifically, it identifies and tracks those NASA's activities which qualify as commercial technology partnerships (see Chapter 5 for partnership criteria definition); develops estimates of the NASA resources invested in the partnership; and compares that investment to the resources NASA invests in its R&D activities (heretofore referred to as R&D base). Currently there are two measurements in this category as follows:
(1) Active Partnership Portfolio (trend indicator) - this measurement identifies those NASA activities which qualify as commercial technology partnerships. It is a count of those partnerships which have been active for any portion of that fiscal year.
(2) Partnership Investment (metric) - this measurement compares the resources which NASA invests in its partnerships (numerator) to its R&D base (denominator - defined below). NASA reports this metric annually as part of its GPRA response, derived from the 1993 National Performance Review recommendations. NASA's GPRA goal for investment in commercial technology partnerships is approximately 10-20% of its R&D investment. As described above, this percentage is calculated by comparing NASA resources invested in partnerships to NASA resources invested in its R&D base. Note that "resources invested in partnerships" include:
(a) Direct Funds to the partnership - funds that are obligated to the partnership through the traditional budgeting and procurement process.
(b) In-House resources - NASA engineer or researcher's time; and
(c)?In-Kind resources - resources such as NASA facilities.
R&D Base Definition - the R&D base includes all of NASA's investment in acquisition activities that have been categorized as R&D by the Federal Product and Services Classification (FPSC) coding structure. This tool is used government-wide by all federal procurement offices. As full cost accounting is implemented at NASA, NASA's in-house R&D investment will be added to the overall R&D base. NASATechTracS automatically calculates the R&D base from the monthly data input from NASA's financial system.
This category focuses on the identifying and reporting success story process element Chapter 6. It's focus is to determine which partnerships are actually yielding success stories. Currently this category has two measurements:
(1) Year to Date Portfolio (trend indicator) - this measurement identifies those success stories which have been identified and released to the public during the current fiscal year.
(2) Total Current Portfolio (trend indicator) - this measurement is the total cumulative set of success stories.
This performance category is currently under study. If developed, measurements in this category would provide insight into the type of economic impacts that the NASA technology commercialization activities are having on the economy.
| TOC | Change History | Preface | Chp1 | Chp2 | Chp3 | Chp4 | Chp5 | Chp6 | Chp7 | Chp8 | AppdxA | AppdxB | AppdxC | AppdxD | Fig2-1 | FigB-1 | FigB-2 | FigC-1 | ALL | |
| | NODIS Library | Program Formulation(7000s) | Search | |