Intellectual Properties, Inc., 280803.2, May 10, 1999
![[Select for PDF file]](/images/pdf.gif)
John C. McIntosh and Robert G. McIntosh for the protester.
William S. Zanca, Esq., and Maj. Gerald Brown, Ballistic Missile Defense Organization, for the agency.
John L. Formica, Esq., and James Spangenberg, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Agency's determination not to award the protester phase II funding for a project proposed under the Department of Defense Small Business Innovation Research program on the basis that the proposal is not innovative and lacks technical merit was not reasonably justified where the contemporaneous documentation references only the negative comments of the evaluators, and neither the contemporaneous documentation nor the arguments, explanations, and testimony in the record adequately explain the basis for the determination in light of the numerous positive comments made by the evaluators about the proposal's innovativeness and technical merit.
DECISION
Intellectual Properties, Inc. (IPI) protests the Ballistic Missile Defense Organization's (BMDO) determination not to award it phase II funding for a project IPI proposed under the Department of Defense (DOD) Small Business Innovation Research (SBIR) program. IPI contends that its proposal to perform research on "Passive Multistatic Hitchhiking Array for Search and Track" was improperly evaluated.
We sustain the protest.
The SBIR program is conducted pursuant to the Small Business Innovation Development Act, 15 U.S.C. § 638 (1994 & Supp. II 1996), which requires certain federal agencies to reserve a portion of their research and development funds for awards to small businesses. The program is made up of three phases.
The program description set forth in the solicitation provided that "[p]hase I is to determine, insofar as possible, the scientific, technical, and commercial merit and feasibility of ideas submitted under the SBIR program." DOD Fiscal Year 1997 SBIR Program Solicitation 97.1 at 1. The solicitation added with regard to phase I that "[p]roposals should concentrate on that research or research and development which will significantly contribute to proving the scientific, technical, and commercial feasibility of the proposed effort, the successful completion of which is a prerequisite for further DOD support in Phase II." Id.
Firms that receive phase I awards may submit proposals for further development work under phase II of the SBIR program. The solicitation's program description provided that "[s]ubsequent Phase II awards will be made to firms on the basis of results of their Phase I effort and the scientific, technical, and commercial merit of the Phase II proposal." Id.
Phase III contemplates that non-SBIR funds will be used to pursue commercial applications of research and development. 15 U.S.C. § 638(e)(4)(C); Microexpert Sys., Inc., B-233892, Apr. 13, 1989, 89-1 CPD ¶ 378 at 1. Specifically, the program description on page 1 of the solicitation stated with regard to phase III that "[u]nder Phase III, the small business is expected to use non-federal capital to pursue private sector applications of the research or development."
Section 4.3 of the solicitation set forth the following evaluation criteria for phase II proposals:
IPI received a phase I award for its proposal submitted under the BMDO topic entitled "sensors," and subsequently submitted a proposal seeking phase II funding.
The record of the agency's initial evaluation of IPI's phase II proposal consisted of two completed evaluation forms. One of the evaluators recommended the acceptance of IPI's phase II proposal for funding, and the other evaluator recommended the rejection of IPI's proposal. IPI was subsequently informed by letter from the cognizant BMDO Program Manager that its phase II proposal had not been approved for funding.
After requesting and receiving a debriefing, IPI filed a protest with our Office, arguing that the agency did not properly apply the evaluation criteria stated in the SBIR program solicitation in determining not to fund IPI's phase II proposal. Specifically, IPI argued that BMDO's decision not to fund IPI's proposal was based solely on IPI's lack of private sector funding for its phase II proposal, rather than a reasoned evaluation of IPI's proposal under the criteria set forth in the SBIR program solicitation.
In Intellectual Properties, Inc., B-280803, Nov. 19, 1998, 98-2 CPD ¶ 115, we sustained IPI's protest, concluding that the agency's reliance on IPI's lack of private sector funding for its phase II proposal as the primary basis for the agency's determination not to fund IPI's proposal, without consideration of whether there were other indicators of commercial potential for IPI's proposal, was inconsistent with the terms of the solicitation. We also rejected the agency's argument, based upon a reevaluation of IPI's phase II proposal prepared during the course of the protest, that IPI was not prejudiced by the flawed evaluation because the reevaluation assertedly showed that IPI's proposal lacked technical merit. We were not persuaded by the agency's reevaluation because it was prepared in the heat of the adversarial process and justified the rejection of IPI's proposal on an entirely different basis than that previously asserted, and thus may not have represented the fair and considered judgment of the agency. We recommended that the agency reevaluate IPI's phase II proposal in a reasonable manner consistent with the evaluation criteria set forth in the solicitation, and reconsider whether IPI's phase II proposal should be funded.
In response to our recommendation, the agency reevaluated IPI's phase II proposal. In this regard, the BMDO Program Manager forwarded IPI's proposal and an evaluation form to be completed to 22 individuals in BMDO and other DOD activities who have radar expertise and have previously supported BMDO's SBIR program in the evaluation of radar technology proposals. Agency Report, Tab 12, Declaration of the Program Manager, at 1-2. The evaluation form includes a number of questions, and provides spaces for the evaluator to respond to the questions as well as spaces for the evaluator to recommend the acceptance or rejection of the proposal.1[1]
(1) How does the technology support BMDO?
(2) What (and how much) is the novelty?
(3) Did Phase 1 succeed?
(4) What technology will BMDO programs be able to leverage during/after Phase 2?
(5) What's the future market potential and the firm's market attitude?
Agency Report, Tab 9. Five of the 22 individuals responded to the agency's request, with four evaluators recommending the conditional acceptance of IPI's phase II proposal for funding, and one evaluator recommending the rejection of the proposal.
The BMDO Program Manager considered the five new evaluations, as well as the two evaluations that had been completed during the agency's initial consideration of IPI's phase II proposal, and again determined that IPI's proposal should not be funded because IPI's phase II proposal was not innovative, in that it duplicated previous research efforts, and lacked technical merit. Agency Report, Tab 11, Letter from the Program Manager to IPI (Jan. 21, 1999). After being informed by the agency that its phase II proposal had again been rejected, IPI filed this protest.
IPI first protests that the reevaluation of its entire proposal as well as consideration of the results of that reevaluation was precluded by our Office's prior decision, where we found the agency's reevaluation of the technical merits of IPI's phase II proposal unpersuasive because, in part, it was prepared in the heat of the adversarial process. Protest at 4.
The details of implementing our protest decision recommendations for corrective action are within the sound discretion and judgment of the contracting agency. DynaLantic Corp., B-274944.5, Aug. 25, 1997, 97-2 CPD ¶ 75 at 4. Our Office will not question an agency's ultimate manner of compliance, so long as it remedies the procurement impropriety that was the basis for the decision's recommendation. QuanTech, Inc., B-265869.2, Mar. 20, 1996, 96-1 CPD ¶ 160 at 2. For example, an agency may conduct a new evaluation of proposals where the record shows that the agency made the decision to reevaluate proposals in good faith, that is, without the specific intent of changing a particular offeror's technical rating or avoiding an award to a particular offeror. See PRC, Inc., B-233561.8, B-233561.9, Sept. 29, 1992, 92-2 CPD ¶ 215 at 3.
We note that the reference in Intellectual Properties, Inc., supra and other decisions of our Office to the "heat of an adversarial process" applies only to that period in time where our Office is actively considering a bid protest and has yet to render a decision. As explained in Boeing Sikorsky Aircraft Support, B-277263.2, B-277263.3, Sept. 29, 1997, 97-2 CPD ¶ 91 at 15, our concern is that reevaluations performed in the heat of an adversarial process may not represent the fair and considered judgment of the agency. That being said, the reevaluation of IPI's proposal, performed in response to the decision issued by our Office sustaining IPI's protest and recommending the reevaluation of its proposal, was not performed "in the heat of an adversarial process." Rather, the reevaluation was performed at a time when the agency was in a position to fully and fairly consider the merits of IPI's phase II proposal and arrive at a reasonable decision.
With regard to the agency's decision to reevaluate IPI's proposal in its entirety rather than simply using the initial evaluation, the agency explains that, as reflected by the reevaluation it prepared during the pendency of the initial protest, it "was on notice of some concerns about the technical viability of [IPI's] proposal and simply desired to review that question . . . in more detail before deciding what the appropriate course of action was." Agency Report at 7. Given the agency's concerns, which appear to be reasonably based, we have no basis to object to the agency's decision to reevaluate IPI's phase II proposal in its entirety.
IPI argues that the agency's evaluation and decision not to fund its phase II proposal were unreasonable. IPI contends that the ultimate conclusion of the Program Manager, as well as certain of the views expressed by the evaluators, that IPI's phase II proposal lacked technical merit and was not innovative, are erroneous.
As a preliminary matter, the agency argues that its "evaluator and selection authority technical conclusions must be upheld absent demonstrated fraud or bad faith." Agency Post-Hearing Comments at 3. BMDO notes here that "[w]hen the GAO reviews SBIR decisions, invariably there is some mention that agencies have broad discretion in award determinations," id. at 2, and that our decisions in the area generally state that our Office reviews agency determinations regarding which proposals will be funded under the SBIR program to see whether the determinations were made consistent with the terms of the solicitation and any applicable statutes or regulations, or whether the agency acted fraudulently or in bad faith. See, e.g., Microexpert Sys., Inc., supra, at 2. BMDO argues that because of this, our Office should only consider whether BMDO acted fraudulently or in bad faith in determining not to fund IPI's phase II proposal, rather than employing the "more onerous standard" of reasonableness. Agency Post-Hearing Comments at 2, 4.
Our Office has reviewed and issued decisions in response to numerous protests objecting to agency actions regarding the SBIR program and as noted by BMDO, has repeatedly recognized in those decisions that "agencies have broad discretion to determine which proposals will be funded under the SBIR program." Deborah Bass Assocs., B-257958, Nov. 9, 1994, 94-2 CPD ¶ 180 at 3; Microexpert Sys., Inc., supra, at 2. However, our recognition that agencies have "broad discretion" in a particular context, including the SBIR program, does not equate to a statement that such actions will be reviewed only for fraud or bad faith. For example, our Office has recognized that agencies have "broad discretion" in evaluating proposals, 51 Comp. Gen. 743, 749 (1972); determining whether to place a proposal within the competitive range, Pharmaceutical Sys., Inc., B-221847, May 19, 1986, 86-1 CPD ¶ 469 at 8; determining the reasonableness of a bidder's price, Satin Am. Corp., B-261068, Aug. 16, 1995, 95-2 CPD ¶ 70 at 4; evaluating the realism of an offeror's proposed costs, Burns & Roe Indus. Servs. Co., B-233561, Mar. 7, 1989, 89-1 CPD ¶ 250 at 3; determining the manner and extent to which source selection officials will make use of technical and cost evaluation results in a negotiated procurement, GTE Hawaiian Tel. Co., Inc., B-276487.2, June 30, 1997, 97-2 CPD ¶ 21 at 16-17; and selecting firms for consideration for award of energy savings performance contracts (which are not subject to competitive procedures), Strategic Resource Solutions Corp., B-278732, Mar. 9, 1998, 98-1 CPD ¶ 74 at 4-5. Although in each of these decisions it was noted that the contracting agency had "broad discretion" regarding its actions, that discretion was nevertheless subjected to the test of reasonableness. Accordingly, BMDO's reliance on our Office's use of the phrase "broad discretion" in bid protest decisions involving the SBIR program as indicating that BMDO's actions here will not be reviewed for reasonableness is misplaced.
We similarly find erroneous BMDO's insistence that statements in our bid protest decisions that determinations regarding which proposals the agency will fund under the SBIR program will be reviewed to see whether they were made consistent with the terms of the solicitation and any applicable statutes or regulations, or whether the agency acted fraudulently or in bad faith, indicate that our Office restricts its review of such determinations to consideration of whether the agency acted fraudulently or in bad faith. These statements clearly provide that our Office will review protested agency determinations regarding whether the agency will fund a SBIR phase II proposal to ensure that they were compliant with applicable statutes or regulations, as well as for fraud or bad faith. As indicated in a number of decisions concerning SBIR contract awards or non-awards, our review for statutory or regulatory compliance includes a determination as to the reasonableness of the contracting agency's exercise of its discretion under those statutes and regulations. See I.S.Grupe, Inc., B-278839, Mar. 20, 1998, 98-1 CPD ¶ 86 at 4; Quantum Magnetics, Inc., B-257968, Nov. 30, 1994, 94-2 CPD ¶ 215 at 4; Deborah Bass Assocs., supra, at 4; S&H Sys., B-250561, Feb. 4, 1993, 93-1 CPD ¶ 102 at 3; Noise Cancellation Techs., Inc., B-246476, B-246476.2, Mar. 9, 1992, 92-1 CPD ¶ 269 at 4. To hold otherwise would be to effectively abdicate legal review of these determinations altogether. See Four-Phase Sys., Inc.--Second Recon., B-201642.3, B-201642.4, Oct. 21, 1983, 83-2 CPD ¶ 473 at 5.
In sum, while our Office recognizes that agencies have broad discretion to determine which proposals will be funded under the SBIR program, we will review such determinations to ensure that they were made consistent with the terms of the solicitation and applicable statutes or regulations and that the agency did not act fraudulently or in bad faith. Agency discretion, though broad, is not unfettered, and continues to be subject to the test of reasonableness. As explained in detail below, we cannot conclude, based upon the entire record, including the explanations and testimony elicited at a hearing, that the agency's decision not to fund IPI's phase II proposal was reasonably based.
The BMDO Program Manager's letter to IPI informing that firm that its phase II proposal had been reevaluated and would not be funded stated, among other things, that IPI's proposal "appears to duplicate an almost identical research effort in a paper published in 1995," and that because of this the Program Manager had concluded that "no innovation was demonstrated."2This letter also stated that the fact that many individuals whose evaluations were requested did not respond is "an indication that the proposed effort did not support their work as it relates to the BMDO mission." Agency Report, Tab 11, Letter from the Program Manager to IPI 1 (Jan. 21, 1999). However, when the legitimacy of this conclusion was challenged by the protester, the agency stated that this "indication" was not considered in the reevaluation. Agency Report at 10; Agency Report, Tab 12, Declaration of the Program Manager, at 3rd unnumbered page. Agency Report, Tab 11, Letter from the Program Manager to IPI 2 (Jan. 21, 1999). The Program Manager further explained the basis for his conclusion that IPI's phase II proposal lacked innovation in a declaration submitted to our Office with the agency report. Agency Report, Tab 12, Declaration of the Program Manager, at 4th-6th unnumbered pages. In this declaration, the Program Manager again references the research paper, and refers to statements made by certain of the evaluators on their evaluation sheets which, in the Program Manager's view, indicate that IPI's proposal lacked innovation. Id. at 4th-5th unnumbered pages. However, as explained below, neither the Program Manager's declaration, nor his letter to IPI, accurately reflects the record regarding the evaluators' views as to the degree of innovation demonstrated in IPI's phase II proposal.
First, although both the letter and declaration reference only evaluator statements which indicate that IPI's phase II proposal lacks innovation, the record shows that the evaluators had differing views in this regard, which were not discussed in the letter or declaration. For example, evaluator 1, although recommending the rejection of IPI's proposal, characterized that proposal as "a very interesting and novel concept."3The two evaluators who initially reviewed IPI's phase II proposal are referred to in this decision as evaluators 1 and 2, and the five evaluators who reviewed IPI's phase II proposal after we rendered our prior decision are referred to as evaluators 3, 4, 5, 6 and 7. Agency Report, Tab 4. Evaluator 2 also appeared to find IPI's phase II proposal innovative, stating in his evaluation sheet, for example, that IPI was "trying to develop an innovative technology." Id. Evaluator 3 appeared to find certain aspects of IPI's proposal innovative, as did evaluator 6, who in describing the novelty of IPI's proposal wrote that "I've never seen anything like this proposal." Agency Report, Tab 9. When the Program Manager was given the opportunity at the hearing to explain why his conclusion that IPI's phase II proposal lacked innovation was reasonable in light of the above evaluator comments (which he conceded indicated that evaluators 1, 3 and 6 thought that the proposal was innovative, videotape transcript (VT) at 11:04:22; 11:31:21; 11:39:06), the Program Manager stated that he understood the problem caused by the agency's failure to address these evaluator comments, but was unable to state why he did not agree with these evaluators' judgments, or why they were wrong or should not be given weight. VT at 11:58:05.
The Program Manager explained, and the record reflects, that the Program Manager's determination that IPI's proposal lacked innovation was based in large part on the views of evaluator 5. The Program Manager testified that evaluator 5 was familiar with BMDO programs, and was viewed as more credible by the Program Manager because it was evaluator 5 who had provided the research paper referenced above.4In its report, in response to the protest argument that the views of evaluator 2, who was most familiar with IPI's Phase I effort, should be accorded weight, the agency asserted that all of the evaluators were treated as equals. Agency Report at 9-10. After the hearing, the agency stated that the Program Manager gave evaluator 5's views "controlling weight." Agency Post-hearing Comments at 11. VT at 11:42:21; 11:54:25; 11:56:07. However, with regard to the research paper, the Program Manager conceded that, while the paper reflected an effort at detecting targets using global positioning satellite (GPS) signals, IPI's proposal was found to be innovative by the evaluators mentioned above because IPI proposes the use of GPS signals to track targets, as well as to detect them. VT 11:58:30. Although the agency was requested to address, through testimony or written comments, the reasonableness of its conclusion that IPI's phase II proposal to develop a system to both detect and track targets using GPS signals lacked innovation because it was duplicative of an effort apparently aimed only at detecting targets, the agency did not do so.
The record also reflects that certain of the evaluator statements referenced by the Program Manager in his declaration are taken out of context or speculative to such an extent they cannot reasonably be said to support the Program Manager's view that IPI's phase II proposal was not innovative. For example, the Program Manager refers to the evaluation of evaluator 3 as support for his determination that IPI's proposal lacked innovation. However, although evaluator 3's evaluation sheet provides that "[t]here may be work related to this proposal being sponsored by other defense organizations," Agency Report, Tab 9, there is no indication in the worksheet or the declaration that evaluator 3 submitted in response to the protest that he was actually aware of any other programs doing related work. Rather, it appears clear from the statement--in context--that evaluator 3 was only speculating as to the existence of other related work.
Similar problems exist with regard to the Program Manager's conclusion that IPI's Phase II proposal lacked technical merit. The Program Manager's letter to IPI informing it that its phase II proposal would not be funded in part because of the proposal's lack of technical merit did not provide any reasons why this was the case. The Program Manager's declaration referenced only negative evaluator statements about IPI's phase II proposal, stating that they were "serious" with no explanation or assertion that they could not be explained or resolved, or should otherwise preclude a phase II award to IPI.5IPI has provided detailed responses to each of the negative comments regarding its proposal's technical merit. Protest, Tab C; Protester's Comments at 10-12.
Moreover, as indicated previously, the evaluators' conclusions varied, with some apparently concluding that IPI's phase II proposal had technical merit. However, here too there was no discussion in the record as to why these evaluators' views should not be given weight or were not valid. For example, evaluator 1 stated that it seemed as if IPI's phase I proposal had succeeded. Evaluator 2, who was most familiar with IPI's efforts because of his position as the technical monitor of the work IPI performed as a result of its phase I award, VT 1:48:55, stated that IPI's phase I proposal had "exceeded its requirements."6The Program Manager testified that in considering a phase II proposal under evaluation factor 1, which consists primarily of the consideration of a proposal's technical merit, the agency reviews whether the phase I work was successfully performed. VT at 10:24:31. Agency Report, Tab 4. Evaluator 2 referred here to IPI's performance of a "hardware-in-the-loop" experiment that apparently demonstrated the viability of IPI's proposal. Id. Evaluator 3 wrote in response to the question "Did Phase I succeed?" that "Progress during Phase I gives sufficient promise to warrant continuing this research," and added that if IPI's proposed technology worked, it "would indeed be a prize." Agency Report, Tab 9. Additionally, evaluator 6 noted that "Phase I seems to indicate that this is a viable concept," with evaluator 7 adding in response to the question concerning the success of phase I that "phase I was done well."7Evaluator 7 also noted that the copy of IPI's phase II proposal provided for evaluation by the Program Manager was in black and white and gray, and that the "data in color could not be read." Agency Report, Tab 9. IPI explained here that its phase II proposal as provided to the agency included graphs done in color which would have been difficult or impossible to understand if they had been reproduced in black and white. VT at 1:01:15. Id.
The agency has not reasonably explained why IPI's phase II proposal lacked technical merit in light of the significant number of positive comments made by the evaluators regarding the proposal's technical merit. In this regard, the Program Manager was unable to reasonably explain why the comments are either invalid or overcome by the other evaluator determinations or agency findings. Because of this, we cannot find that there was a reasonable basis for the Program Manager's conclusion that IPI's phase II proposal lacked technical merit.
The Program Manager also testified at the hearing that one of the primary reasons IPI's phase II proposal was rejected for funding was the Program Manager's understanding that there were other programs being pursued by BMDO which were technologically similar to IPI's phase II proposal and further along in development. VT 1:57:40; 2:21:19. In support of this assertion, the Program Manager pointed to a statement made by evaluator 5 that "the BMDO programs that could benefit [from IPI's efforts] are ahead of this proposed research effort."8As noted above, the Program Manager explained that he placed considerable weight on the conclusions of evaluator 5 because this evaluator was employed by BMDO and thus had unique knowledge of BMDO programs. VT 1:58:54. The agency asserts in its post-hearing comments (at 13) that "[t]his decision basis, even if all others fail for whatever reasons, would render [the Program Manager's] overall non-selection decision reasonable and rational." However, the agency has not identified these BMDO programs.
We are not persuaded by the agency's position here, not only because no evidence has been presented to support this assertion, but also because it was in essence raised for the first time during the hearing on the protest. That is, there is no indication in the agency's letter to IPI informing it that its phase II proposal had not been selected for funding that the nonselection was based in part or in whole on the existence of similar BMDO efforts. Further, the agency's 16-page legal memorandum does not appear to contain any mention of the existence of similar BMDO efforts as a basis for the rejection of IPI's phase II proposal, and the six-page declaration of the Program Manager contains only one sentence (that consists primarily of a quote from evaluator 5's evaluation sheet) that raises this concern.9The Program Manager's declaration stated the following:
Additionally, the negative evaluator who provided the reference to the [research paper] . . . indicated that this proposed research was "an almost identical research effort." Moreover, [this evaluator said that] potential "BMDO programs that could benefit are ahead of this proposed research effort."
Agency Report, Tab 12, Declaration of the Program Manager, at 5th unnumbered page.
As noted in the first protest, we accord lesser weight to post-protest judgments such as the Program Manager's conclusion that the protester's proposal was duplicative of other BMDO programs. Intellectual Properties, Inc., supra, at 7; Boeing Sikorsky Aircraft Support, supra. In this case, irrespective of the weight given to the belated basis for rejection, without evidence supporting the conclusion of duplicative research (such as an identification of the BMDO programs that are technically similar) we are unable to find this decision basis reasonable.
Accordingly, we sustain the protest. We recommend that, since we have twice sustained IPI's protests, the agency appoint a new source selection official to reconsider and document its determination as to whether IPI's phase II proposal should be funded. We also recommend that IPI be reimbursed the costs of filing and pursuing its protest. Bid Protest Regulations, 4 C.F.R. § 21.8(d) (1999). IPI's certified claim for such costs, detailing the time and costs incurred, should be submitted within 60 days after receipt of this decision. 4 C.F.R. § 21.8(f)(1).
The protest is sustained.
Comptroller General
of the United States