Strategy for NIH Funding

Initial Peer Review and Your Next Steps   ·   Strategy for Second-Level Review

How Reviewers Score Applications

Your overall impact score is the main basis for our funding decision, but how do reviewers assign scores? 

Reviewers judge your application for its ability to make a strong impact on its field. Impact is a function of 1) the importance of the topic you proposed and 2) the likelihood that you can complete the experiments you designed and get the work done.

See how the peer review criteria that reviewers use to assess applications relate to their scores in ways that may surprise you.

Table of Contents

• Just the Facts
  • How NIH Review Criteria Affect Your Score
    • Role of the Review Criteria
    • Other Critical Factors Can Affect Your Score
  • Assigning an Overall Impact Score
• Our Advice
  • Impact Is a Function of Importance and Likelihood

Just the Facts

(This section has factual information only; for advice on this topic, go to Our Advice below.)

How NIH Review Criteria Affect Your Score

Your overall impact score is the key review outcome, the main basis for a funding decision by an NIH institute. Your score reflects your reviewers' judgment of the extent to which your project can make an impact.

NIH defines impact as the likelihood that your project will exert a powerful influence on its field. Reviewers also usually comment on its relevance to the NIH mission: improving human health through science.

To arrive at your overall impact score, reviewers consider the following core review criteria:

• significance
• investigator
• innovation
• approach
• environment

They use the significance and innovation criteria to assess a project's importance, and they use approach, investigator, and environment to assess its likelihood of success (feasibility).

Reviewers—particularly your assigned reviewers—look to your Research Strategy for the most detailed information on significance, innovation, and approach, and they mostly read your Biosketches and Resources sections to gauge the investigator and environment criteria, respectively.

Though your overall impact score reflects all the criteria, it does not represent a mathematical sum. Rather, it is a gestalt—an integrated whole that cannot be derived from the sum of its parts.

While overall impact may sound like the significance criterion, it's different.

Significance is the importance of your project: will it advance your field and fit the NIH mission to improve health through science? It does not take into account your ability to conduct the research. So while reviewers will penalize you if they think success is unlikely, that penalty will not be reflected in their assessment of the significance criterion.

Depending on the nature of your application, other review criteria may affect you, including protections for human subjects, vertebrate animals, and others. If you are responding to an institute-specific program announcement or a request for applications, it may have special review criteria as well.

Read the Review Criteria SOP for more information.

Role of the Review Criteria

Because a score is ultimately a gestalt, peer reviewers don't score strictly by the review criteria; here are some thoughts on how they relate to your overall impact score.

Ideal application. To some extent, reviewers judge your application based on their ideal of an outstanding application in your field of science.

Usage varies. Adherence to the criteria varies by committee.

Weight varies. Though all review criteria can affect your score, an application does not need to be strong in all of them to get an outstanding score. Here are two examples:

• Reviewers assign an exceptional score to important research that is not innovative but is essential to move a field forward.
• An application with very high significance receives an outstanding overall impact score even though reviewers are less enthusiastic about the other criteria.

Other Critical Factors Can Affect Your Score

Your reviewers consider other items besides the review criteria.

Special areas. Depending on the experiments you propose, they make sure you have complied with NIH policies for sensitive areas such as recombinant DNA research, human subjects, research animals, and select agents.

Presentation. Your presentation can make or break your application.

Though reviewers assess the science, they are also influenced by the writing and appearance of your application.

If your application has lots of typos and internal inconsistencies, your score can suffer. Read more at Master the Application in Part 3.

Assigning an Overall Impact Score

Taking the review criteria into account, here are the steps the review committee takes to arrive at an overall impact score.

Before the meeting, your assigned reviewers score each criterion and give your application a preliminary overall impact score.

As a result of the discussion at the meeting, the assigned reviewers may suggest a different overall impact score to the group.

Next, all reviewers vote.

• Assigned reviewers enter their official scores for each criterion and an overall impact score on the vote sheet. The other reviewers can see these scores.
• Other reviewers give an overall impact score and usually have an option of scoring each criterion.
• Each member marks scores privately on a vote sheet, assigning a whole number from 1 (best) to 9 (worst).
• At the end of the meeting, the scientific review officer (SRO) collects vote sheets and adds the scores.
• After the meeting, reviewers can edit their criterion scores and critiques, but they do not change their final overall impact scores.
• To create a raw overall impact score:
  • Scores are averaged and rounded mathematically to one decimal place, e.g., a 1.34 average yields 1.3.
  • That number is multiplied by 10 to yield an overall impact score; in the example above, it would be 13.
• R01 applications also get a percentile. Learn how NIH creates percentiles at Understand Paylines and Percentiles in Part 7.

The table below shows the relationship between the level of impact, scores, and descriptors.

Scoring Table for Research Grants

Impact	Impact Score	Descriptor	Additional Guidance on Strengths/Weaknesses
High	1	Exceptional	Exceptionally strong with essentially no weaknesses
	2	Outstanding	Extremely strong with negligible weaknesses
	3	Excellent	Very strong with only some minor weaknesses
Moderate	4	Very Good	Strong but with numerous minor weaknesses
	5	Good	Strong but with at least one moderate weakness
	6	Satisfactory	Some strengths but also some moderate weaknesses
Low	7	Fair	Some strengths but with at least one major weakness
	8	Marginal	A few strengths and a few major weaknesses
	9	Poor	Very few strengths and numerous major weaknesses
Definitions
Minor: easily addressable weakness that does not substantially lessen the impact of the project. Moderate: weakness that lessens the impact of the project. Major: weakness that severely limits the impact of the project.

More Resources

Center for Scientific Review's Insider's Guide to Peer Review for Applicants

Our Advice

(This section has advice only; you should also read the factual information above at Just the Facts.)

Impact Is a Function of Importance and Likelihood

In the peer review universe, the primary outcome is an overall impact score, reflecting reviewers' judgment of two broad concepts: importance and likelihood.

1. Importance—the significance and innovation of the research problem—its ability to move the frontier of knowledge forward.

2. Likelihood—the ability that you, the principal investigator, can achieve your ends, as judged by your experimental design, the expertise of your team, and the resources at your disposal to execute the project.

Together importance and likelihood form impact. It may help you to remember these relationships using this simple formula.

impact = function (importance, likelihood)

Here's an expanded formula that shows the relationship of these concepts to the five review criteria. Keep in mind that, though the criteria do not add up mathematically to yield the score, reviewers do take them all into account.

impact = function (importance (significance, innovation), likelihood (approach, investigator, environment)

In the Just the Facts section above, we discussed where in the application reviewers look to find information related to the different criteria.

But because other sections count too, it's best to think of your application as an integrated whole, and keep the review criteria in mind as you write all parts of the application.

For example, your citations and other references in your Approach should highlight your expertise and that of your colleagues. And you probably want to point out the project's significance and to a lesser extent innovation in several places to meet the needs of the full reviewer audience.

Read more on these pages:

• Pick a Research Project in Part 2—choose a high-impact project
• Initial Peer Review and Your Next Steps in Part 5—learn about peer review
• Part 3. Write Your Application—create an application that reviewers can readily assess

Strategy for NIH Funding

Initial Peer Review and Your Next Steps   ·   Strategy for Second-Level Review

See the other sections of
Part 5. Assignment and Review

Table of Contents for the Strategy

We welcome your comments, questions, or suggestions. Email deaweb@niaid.nih.gov.