For applications reviewed at NIAID, our scientific review officers ask reviewers to use the nine-point numerical scale shown below for assigning criterion and overall impact scores. A score of 1 indicates an exceptionally strong application (or exceptionally strong Significance, Investigators, Innovation, Approach, and Environment) with essentially no weaknesses. A score of 9 indicates serious and substantive weaknesses with very few strengths. Reviewers provide scores in whole numbers only, not decimals.
Criterion Scoring
Criterion scores rate each of the five review criteria—Significance, Investigators, Innovation, Approach, and Environment—using the 1 to 9 scoring scale. For each criterion rating, the strengths and weaknesses within that review criterion should be considered. In considering strengths and weaknesses, reviewers should consider the relative importance of the strengths and weaknesses noted, not simply the number of strengths and weaknesses.
These criterion scores are included in the summary statement to give applicants of both discussed and nondiscussed (i.e., streamlined) applications a sense of how consideration of the review criteria influenced the overall evaluation of the application. The individual criterion scores are not mathematically related to the overall impact score. This allows for an assessment of how the application as a whole comes together.
Additionally, because the relative importance of each individual criterion to the overall score differs for each application, reviewers should not use a formula of weighted or unweighted averages of the criterion scores across applications to determine the overall impact score.
Overall Impact Score
When determining the overall impact score, the far left “Impact” column in the table below provides guidance for assigning scores to applications based on the project’s likelihood of having a sustained, powerful influence on the research field(s) involved. In addition, human subjects, vertebrate animal care, and RFA-specific criteria also should be considered in determining the overall impact score.
| Overall Impact or Criterion Strength | Score | Descriptor |
|---|---|---|
| High | 1 | Exceptional |
| 2 | Outstanding | |
| 3 | Excellent | |
| Medium | 4 | Very Good |
| 5 | Good | |
| 6 | Satisfactory | |
| Low | 7 | Fair |
| 8 | Marginal | |
| 9 | Poor |
With nine possible scoring discriminations, it is imperative that reviewers distribute or spread their scores as widely as possible among applications. The descriptors (above) associated with each rating were designed to encourage the spreading of scores. Therefore, although score distributions may vary by review meeting, reviewers should use the full range of 1 to 9; the expectation, however, is that there will be few 1s and few 9s.
Due to the large number of applications, some review meetings discuss only a percentage of the applications that were submitted. For discussed applications, the assigned reviewers state their final overall impact scores, defining the score range. Based on the discussion, all nonconflicted reviewers also score the application. Unassigned reviewers may score outside this range. However, they must announce their intent to do so during scoring and briefly describe the reason to ensure that the reasons have been discussed adequately.
Any score outside the range established by the assigned reviewers should be declared, even if the range is a single score (i.e., all assigned reviewers give the same final score). It is important that all points of view and opinions of reviewers are discussed. Reviewers should feel free to score outside the range based on their determination of the overall impact of the application. However, reviewers should maintain their same standards throughout the review.
This guidance is for applications reviewed at NIAID only. For applications reviewed by the Center for Scientific Review, read the NIH Additional Scoring Guidance for Research Applications.