The redesign activities addressed the issue of surveying the appropriate population. Because the purpose of the survey is to provide national estimates on research facility capacity and needs of colleges and universities, all colleges and universities conceivably might be included in the population. However, many institutions do not define research as an important part of their mission, and thus, NSF's practice has been to define eligibility criteria for the survey.
Currently, the survey has three different criteria for determining eligibility: the institution must either (1) offer doctorate degrees in S&E fields, (2) report at least $150,000 in S&E R&D expenditures in the applicable year, or (3) be an HBCU with R&D expenditures of any amount. The redesign process looked at whether these were appropriate criteria. If some institutions are included unnecessarily, both the cost and the burden associated with the survey can be reduced. Data quality also might be improved by both potentially increasing the overall response rate (if institutions for which the survey is less relevant are less likely to respond as appears to be true of smaller institutions in general) and allowing the data collection activities to focus on those institutions with the greatest amount of space.
The expert panel indicated that because using a $150,000 threshold was somewhat arbitrary, they did not object to changing that threshold. On the other hand, they opposed establishing comprehensive constraints such as excluding institutions that do not grant doctorates because such constraints would implicitly suggest that no such institutions had important roles in research. For this reason, the redesign study focused on the implications of changing the dollar threshold. Data from previous years indicate that both space and research expenditures are concentrated among a relatively small number of institutions so that even if many of the smallest institutions were dropped from the survey, NSF could still maintain wide coverage in terms of space and research expenditures.
Table 4 shows the implications of changing the eligibility criteria for academic institutions by setting a higher minimum threshold of R&D expenditures.[5] If R&D expenditures were used as the sole criterion for eligibility and the $150,000 threshold were increased, it would be possible to drop many of the institutions and still have extensive coverage. For example, if the $150,000 threshold were increased to $5 million, 251 institutions would be dropped (49 percent of the total currently in the population) and the remaining institutions still would have 99 percent of R&D expenditures and 94 percent of the research space. Alternatively, if the $150,000 threshold were increased to $1 million, 126 institutions would be dropped (25 percent of the total currently in the population) and the remaining sample would have 99.8 percent of R&D expenditures and 98 percent of research space.
Table 4. R&D expenditures (1999) and total NASF (2001), by amount of R&D expenditures
|
The above statistics are based on eligibility being determined solely by R&D expenditures. Dropping automatic eligibility for HBCUs with any R&D expenditures would likely be undesirable if the $5 million threshold is used. That change would reduce the number of HBCUs from 57 to 13. On the other hand, if the $1 million threshold is used, then dropping HBCUs under that threshold would be less critical, (only 19 of 57 HBCUs would be dropped). Dropping the automatic eligibility of institutions granting S&E doctorates in is a less serious issue: the change would result in dropping 66 of the 309 doctorate-granting institutions if the $5 million threshold is used, and 25 of the 309 doctorate-granting institutions if the $1 million threshold is used. Thus, doctorate-granting institutions would still be well represented even without automatically defining them as eligible.
In short, a $1 million threshold could be used as the sole criterion for eligibility and the survey would still cover 99.8 percent of the R&D expenditures and 98 percent of the research space and would well-represent HBCUs and doctorate-granting institutions. If a $5 million threshold is used, one would want to supplement the sample with additional HBCUs (e.g., either including all HBCUs with any R&D expenditures, or at least those with $1 million in R&D expenditures), but the newly defined population would still have 99 percent of the R&D expenditures and 94 percent of the space.
NIH has defined the eligibility of nonprofit biomedical research organizations based on a single criterion: receiving at least $150,000 in NIH funding. Table 5 shows the implications of raising the dollar amount. Many of the smallest institutions could be dropped from the survey, yet still allow the survey to capture the great bulk of NIH-funded research. If the NIH funding threshold were increased to $2 million, then 96 institutions would be dropped (44 percent of the total currently in the population) and the remaining institutions would still receive 97 percent of NIH research funding and have 85 percent of the research space. If the NIH funding threshold were increased to $1 million, then the 70 smallest institutions would be dropped (32 percent of the total currently in the population) and the remaining sample would still receive 99 percent of NIH research funding and have 89 percent of the research space.
Table 5. NIH funding (2000) and total NASF (2001), by amount of NIH funding
|
[5] Nonrespondents are excluded from this table since net assignable square feet (NASF) is not known for them. The total number of institutions in each research and development (R&D) expenditure category, the total amount of R&D expenditures, and the total NASF all would be greater if nonrespondents were included.
|
|
|
|
|
|