NIST GCR 02-829
Universities as Research Partners

2. Analysis of the Data

No systematic data exist regarding universities as research partners at either the firm level or the project level. While general information can be gleaned about formal research joint ventures and university participation in them from the Federal Register (such information is filed in accordance with the National Cooperative Research Act of 1984), it is insufficient for a detailed investigation of universities as research partners.⁽⁷⁾ We preferred project-level data. One source of project-level data is the ATP.

As background, ATP was established within the National Institute of Standards and Technology (NIST) by the Omnibus Trade and Competitiveness Act of 1988,⁽⁸⁾ and was modified by the American Technology Preeminence Act of 1991. The goals of ATP, as stated in its enabling legislation, are to assist U.S. businesses in creating and applying the generic technology and research results necessary to (a) commercialize significant new scientific discoveries and technologies rapidly and (b) refine manufacturing technologies.

These same goals were restated in the Federal Register on July 24, 1990: “The ATP . . . will assist U.S. businesses to improve their competitive position and promote U.S. economic growth by accelerating the development of a variety of pre-competitive generic technologies by means of grants and cooperative agreements.” The ATP received its first appropriation from Congress in FY 1990.

Because ATP has a very particular set of goals, it is important to emphasize that studying ATP projects will not give a complete picture of the university-industry R&D interaction. When compared with a random sample of university-industry projects, the projects analyzed in this study are more likely to be perceived as having high social value, will generally be riskier, involve generic technology, and be at such an early stage in development that the technology is not easily appropriable. In spite of this qualification we feel it is worth obtaining a picture of this section of the public R&D infrastructure while keeping the nature of the selection process firmly in mind.

THE POPULATION OF ATP-FUNDED PROJECTS

The ATP classifies each funded project by the size of the lead participant. Each lead participant is placed into one of four ATP-defined size categories. Not-for-profit organizations are designated as a size category; small is defined as an organization with fewer than 500 employees; large is defined as a Fortune 500 or equivalent organization (a moving definition; at the time of our analysis, a Fortune 500 had at least $2.578 billion in revenue); and medium organizations are all others. Small companies lead more than one-half of the projects, including single company applicant projects and joint venture projects. The following results present a snapshot of the projects awarded between 1991 and 1997 at the time of this study.

ATP awards presented from April 1991 through October 1997: 352; that is, 256 projects were active, 75 had been completed, 16 had been terminated for not meeting project goals, and 5 had been terminated during the negotiation stage before a cooperative agreement was signed.

Number of single company applicant projects: 234, with 54.7 percent involving a university as a subcontractor.

Number of joint venture projects: 118, with 60.2 percent involving a university either as a research partner or as a subcontractor.⁽⁹⁾

Mean total (ATP plus industry funding) proposed cost of funded projects: $6.59 million, with a range from $490,000 to $62.97 million. By statute, ATP’s maximum contribution to single company applicant projects is $2 million in direct costs; ⁽¹⁰⁾ ATP’s maximum contribution to joint venture projects cannot exceed 50 percent of total costs (direct and indirect costs). The mean project cost for a joint venture project is just over four times that of a single company applicant project: $13.24 million compared with $3.24 million.

Percent of total cost funded by ATP: 56.1 percent, with a range between 11.8 percent and 94.6 percent. ⁽¹¹⁾ Average ATP contribution for joint venture projects is less than for single company applicant projects: 47.9 percent compared with 60.3 percent. Not only is the average level of ATP support, in percentage terms, less for joint venture projects but the range of that support is more narrow. The range for single company applicant projects is between 11.8 percent and 94.6 percent, compared with between 32.4 percent and the statutory 50.0 percent limit for joint venture projects.

Percent of ATP-funded projects: information/computer systems, 29 percent; biotechnology, 19 percent; materials,16 percent; electronics, 12 percent; discrete manufacturing, 11 percent; chemicals and chemical processing, 7 percent; energy and the environment, 6 percent.

Involvement of universities as research partners, by type of project: biotechnology, 42 percent; discrete manufacturing, 39 percent; information/computer systems, 33 percent; and electronics, 7 percent. Other technology areas did not have university involvement.

Percent of funded projects expected to last three years or longer: 70 percent. (By statute, single company applicant projects cannot exceed three years, and joint venture projects cannot exceed five years.⁽¹²⁾

SELECTION OF A SAMPLE OF ATP-FUNDED PROJECTS

Samples were selected by using a series of filters, some under our control and others not. The process of selection is summarized in Table 1. Twenty-one projects terminated early and were therefore unavailable for sampling. (An analysis of the reasons for early termination is provided in the next section.) Each project must be active and must have been so for at least one year. A priori, we reasoned that these constraints would help to ensure the respondent’s capability to rely on a research project history when answering the questions. These two filters reduced the population of 352 projects to 192 projects (see column 2 in Table 1).

These 192 projects were then grouped according to the six types of projects with/without university involvement listed in Table 1 (column 1). From each of the categorical groupings, a sample of nine projects was selected (column 4). Attention was also given in the selection of nine projects to technology areas, size of lead participant, length of time the project had been active, and the total proposed research budget of the projects. Also reported in Table 1 (column 5) are the sampling probabilities by type of university involvement. ⁽¹³⁾ This process of random stratified sampling yielded 54 projects.

Separate and distinct survey instruments were designed to obtain information about the nine projects selected in each of the six categories of type of university involvement. ⁽¹⁴⁾ The surveys were pre-tested with at least one lead participant of a project that could in principle have been included in the sample of nine but was not.

1. Filtered projects are projects that were active one year or more and were still active in the beginning of 1998.
2. Sample projects were selected from the filtered project universe to ensure an equal number in each category. The projects are mutually exclusive.
   

The ATP provided the name of a contact person in each of the 54 companies who was then contacted by telephone, explained the nature of the study, asked to participate in a survey, and assured that specific responses would remain confidential and reported only in summary form. Each agreed to participate in the survey. The respective category-specific survey was sent to each respondent. Each non-respondent was re-contacted up to three times on a weekly basis and urged on each occasion to complete and return the survey. Table 1 (column 6) shows the number of surveys received by category of university involvement. ⁽¹⁵⁾ The sample for analysis became 47, as shown at the bottom of Table 1. Seven did not respond.

We emphasize, again, that we are aware of the limitations of the self-reported data that we analyzed. While our survey instruments were pre-tested, the possibility that our primary data reflect the personal attitudes of the respondents as well as objective characterizations of their program is still present. Thus, efforts to generalize from our findings should be made with caution.

ANALYSIS OF TERMINATED PROJECTS IN THE POPULATION

Reasons for the early termination of the 21 projects were investigated and ranged from the financial health of the participant(s) to lack of research success in the early part of the project: 11 were joint venture projects, and 10 were single company applicant projects. Joint ventures represent 34 percent of the population of ATP-funded projects, but they are 52 percent of terminated projects. Thus, joint ventures appear to have a higher probability of termination than single company projects. Of the 11 joint ventures that were terminated, three included a university as a research partner and two others included a university as a subcontractor. Four of the single company applicant projects included a university as a subcontractor. Thus, 9 of the 21 terminated projects involved a university in some research capacity.

To consider in a more systematic manner the relationship between university involvement in an ATP-funded project and the probability that the project will terminate early, we estimated a probit model of termination probability conditional on ATP’s share of funding, involvement of a university, type of project, size of the lead participant, and technology area. A time variable denoting the year in which each project was initially funded was also included.

To be precise, we estimated the following model:

Pr (project i terminates early) = F(X_i ß) (1)

where F is the cumulative normal probability function and X_i is a vector of variables that characterizes project i.

The probit estimates from alternative specifications of equation (1) are reported in Appendix A (Table A1), and the predicted probabilities as a function of key variables are shown in Table 2. Of particular interest is the nature of the relationship between university involvement and termination. Results imply that projects with university involvement as either a research partner or subcontractor have a lower probability of early termination. The probability of early termination decreases as ATP’s share of funding increases, although the effect is barely significant, and only for the specification to simulate the results shown in Table 2. Termination rate does not vary across technology area,⁽¹⁶⁾ but projects where the lead partner is of medium size are more likely to terminate early than do the others.

The top portion of Table 2 presents the calculated probabilities for a project terminating early by size of the lead participant., For this example (information technology projects begun in 1991), the calculated probability of early termination is lower for each size category when a university is involved in the project. Similarly (bottom portion of Table 2), the calculated probability of each termination is lower for each discrete level of ATP's share of funding when a university is involved in the project. Similar relationships exist across other research technology areas. In the population of ATP projects, university involvement is clearly associated with a lower probability of early termination. ⁽¹⁷⁾

Perhaps university participation reduces the likelihood of early termination simply because the projects are more complex and thus project managers may have more difficulty seeing that the project will fail to reach the technical goals until late in the project. Also, more complex projects, even if they fail to achieve their ultimate objective, may still generate knowledge of potential utility to the award recipients.

ESTIMATION OF THE PROBABILITY OF RESPONSE TO THE SAMPLE SURVEY

Only two of the six categories of university involvement listed in Table 1 (column 6) had a 100 percent response rate. Contact persons in joint venture projects were less likely to respond, with the least responsive category being joint venture projects with universities as both partners and subcontractors (only five of nine surveys were returned). The probability of survey response was examined using a probit model to quantify the potential bias because of non-response.

The probit estimates for a model of the probability of responding are reported in Appendix A (Table A2). When all of the independent variables are included, the results are not very significant. The only variable that is even marginally informative about the probability of survey response is the dummy for joint ventures with universities as both partner and subcontractor, ⁽¹⁸⁾ which are arguably the most complex arrangement contractually. Other factors held constant, contact persons in joint ventures with universities as research partners and as subcontractors have a lower probability of response than other contact persons. The associated predicted probabilities of response by selected technology areas and type of university involvement are reported in Table 3.⁽¹⁹⁾

Table 3. Predicted Probability of Survey Response

In the results presented later, response bias will be corrected in two ways: (a) by simply including the dummy for joint ventures with universities as both partner and subcontractor in estimations to test for response bias ⁽²⁰⁾ and (b) estimating a full two equation model using maximum likelihood, where one equation is the equation of interest and the other is the equation for response probability. The implication of the first strategy will be that we cannot identify the direct effects of being a joint venture with a university participating as a partner and as a subcontractor separately from the impact on the probability of survey response.

____________________
7. These data have been analyzed in Link (1996). See also Hagedoorn, Link, and Vonortas (2000).

8. This section of the Omnibus Trade and Competitiveness Act of 1988 is also known as the Technology Competitiveness Act.

9. The generic term "partner" is being used to refer to a university-industry relationship where the university is either a subcontractor to a single company or to a joint venture or where the university is a research partner in a joint venture, which means that the university is a formal member of the joint venture. To refer to this latter case, we describe the university as a "research partner."

10. Since December 1997, single applicant, large company participants must provide for at least 60 percent of direct and indirect project costs.

11. Participants in joint venture projects must provide for at least half the total costs of the project while single applicant, non-large company, participants are at a minimum responsible for indirect costs.

12. Expected project duration is agreed upon at the time ATP funds the project.

13. Variability in these probabilities reflects the fact that the sample size is constant at nine and that the size of the population of appropriate projects to sample, by category type, varies (column 4).

14. Copies of the survey instruments are in Appendix B.

15. Because there are multiple dimensions of ATP-funded projects, we do not claim that our sample of 47 respondents is representative of the filtered population or of the whole population in all dimensions. We offer our sample as one sample to consider, and possibly to generalize about, given the stated filtering and selection process.

16. This conclusion needs to be qualified slightly. Because no projects in discrete manufacturing terminated early, these projects could not be included in the models estimated in the first 2 columns of Table A1 (where technology dummies are used). Clearly projects in this technology have a lower early termination rate than projects in the other technology areas.

17. The information in Table A1 is used to calculate a hazard rate for the probability that a project does not terminate early for use in the subsequent statistical analyses of a sample of ATP-funded projects to control for possible sample selection bias. To anticipate the use of this variable in later survey question equations it is important to note that its inclusion in an ordered probit or tobit is not really econometrically correct if it actually enters. That is, if the probability distribution in the termination equation and the distribution in the survey question equation are dependent, then the appropriate method is to specify a full maximum likelihood model for the two random variables and estimate jointly (such a model is outlined in the appendix to Hall, Link, and Scott 2000). In fact, we found that the termination hazard and the sample response hazard never entered significantly, and that joint maximum likelihood estimates did not differ significantly from our single equation estimates, which implies that sample selection is unlikely to produce significant bias in our estimates. However, our sample size is small, so the power of all these tests is low.

18. The same university cannot be both a partner and a subcontractor in a joint venture.

19. The sample size in Tables A2 and Table 3 is quite small (only 29 observations), because all projects with large lead participants or whose technology area was electronics, biotechnology, chemicals, energy, or the environment responded to the survey and hence these projects could not be used to estimate the probability equation (they had one or more characteristics that were perfect predictors). In later estimations, a response probability equation was used that does not depend on technology and is therefore defined for the whole sample.

20. As with our analysis of the probability of early termination, the results in Table A2 could be used to calculate a survey hazard rate to be used in the statistical analyses that follow. The survey hazard rate is the conditional probability density of responding to the survey. However, in practice, the only variable that predicted response or non-response in a simple probit model was joint venture projects with universities as both partner and subcontractor. We therefore used a simpler and more robust method to correct for response bias, by including the dummy for joint ventures with universities as both partner and subcontractor directly in the estimated model. Unlike the use of a hazard rate, this correction does not require normality of the response probability equation to be valid. In the case of a single dummy variable predictor, of course, the two approaches for converting any response bias would be equivalent if normality held.

Return to Contents or go to next section.

Date created: October 18, 2002
Last updated: August 2, 2005

ATP website comments: webmaster-atp@nist.gov  / Technical ATP inquiries: InfoCoord.ATP@nist.gov.

NIST is an agency of the U.S. Commerce Department
Privacy policy / Security Notice / Accessibility Statement / Disclaimer / Freedom of Information Act (FOIA) /
No Fear Act Policy / NIST Information Quallity Standards / ExpectMore.gov (performance of federal programs)