Analysis of SBIR phase I and phase II review results at the National Institutes of Health.

A cohort of phase I and phase II summary statements for the SBIR grant applications was evaluated to determine the strengths and weaknesses in approved and disapproved applications. An analysis of outcome variables (disapproval or unfunded status) was examined with respect to exposure variables (strengths or shortcomings). Logistic regression models were developed for comparisons to measure the predictive value of shortcomings and strengths to the outcomes. Disapproved phase I results were compared with an earlier 1985 study. Although the magnitude of the frequencies of shortcomings was greater in the present study, the relative rankings within shortcoming class were more alike than different. Also, the frequencies of shortcomings were, with one exception, not significantly different in the two studies. Differences in the summary statement review may have accounted for some differences observed between the 1985 data and results of the present study. Comparisons of Approved/Disapproved and Approved-Unfunded/Funded yielded the following observations. For phase I applicants, a lack of a clearly stated, testable hypothesis, a poorly qualified or described investigative team, and inadequate methodological approaches contributed significantly (in that order) to a rating of disapproval. A critical flaw for phase II proposals was failure to accomplish objectives of the phase I study. Methodological issues also dominate the distinctions in both comparison groups. A clear result of the data presented here and that published previously is that SBIR applicants need continuing assistance to improve the chances of their success. These results should serve as a guide to assist NIH staff as they provide information to prospective applicants focusing on key elements of the application. A continuing review of the SBIR program would be helpful to evaluate the quality of the submitted science.