Liberty A. Walton, Brandie K. Taylor, Dione I. Washington, Larry S. Solomon, and Jane C. Lockmuller
The NIAID of the National Institutes of Health uses several approaches to support research, including the P01 Program Project Grant mechanism. This mechanism provides funding for at least two inter-related research projects, which are needed to address a central research problem in a conceptually inter-related manner, as well as administrative and scientific core services. The individual P01 research projects are each roughly equivalent in scale and scope to a NIH R01 grant, but the aim is that, together, the P01 projects produce higher synergy and address more complex scientific questions than individuals R01s. The central question of this study was: given the higher funding and administrative costs of P01 grants, does the unsolicited P01 mechanism provide additional value in comparison to the R01 mechanism in terms of research synergy, productivity, and scientific impact?
- Are the Division of Allergy, Immunology, and Transplantation (DAIT) and Division of Microbiology and Infectious Diseases (DMID) approaches to review and management of unsolicited P01 grant applications/awards appropriate in terms of cost/process compared with a matched sample of R01 grant applications/awards?
- What are the characteristics and scope of unsolicited P01 awarded grants at DAIT and DMID and do these differ from those of matched R01 grant awards?
- Do unsolicited P01 grant awards result in greater synergy than matched unsolicited R01 awards?
- Do unsolicited P01 grants differ in research productivity from matched R01 grants?
- Do unsolicited P01 projects and matched R01 grants differ in scientific impact over time?
- What is the relationship among synergy, research productivity, and scientific impact for awarded unsolicited P01s compared to R01 awards?
Mixed-method, quasi-experimental study that included:
- A focus on DAIT and DMID, which manage the largest share of NIAID P01 grants
- The portfolio of unsolicited P01 and R01 grants that were active during a five-year interval (fiscal years 07–11)
- Unsolicited NIAID R01 grants matched to specific P01 projects by Division, Scientific Program Code, Product Development Pathway Code, Project Year, and PI Career Status
- A mixed-method data collection approach that permitted triangulation across sources
- An analytic strategy that included pair-level analyses comparing P01 projects and individual R01 grants, as well as grant-level analyses that compared the P01 grant with the group of R01 grants matched to its projects. The final sample included 260 R01 grants and 72 P01 grants consisting of 260 individual P01 projects.
- Archival databases – NIH IMPAC II QVR, RePORTER, and RPAB and DEA databases
- Semi-structured interviews with 14 NIAID DAIT and DMID Program Officers and with 9 PIs who received NIAID P01 and R01 awards
- Bibliometric analyses from Thomson Reuters and the NIH Library’s Bibliometric Services Program
- Compared to R01s, interviewees agreed that unsolicited P01 grants are broader in scope and scale, may be more oriented toward translational research, allow PIs greater flexibility to investigate unexpected findings, may be more useful when a multidisciplinary approach is necessary, and provide more specialized equipment or facilities.
- P01 project direct costs are significantly costlier than individual R01 grants. P01 grants (projects and core services) are significantly more costly than grouped R01 grants.
- R01 grants were likely to have significantly larger research teams and involve more partnering institutions and scientific disciplines than the P01 projects.
- Both mechanisms are equally productive (as measured by publications). For grants in their first funding cycle, the average number of publications produced per year peaks by the third project year and declines thereafter.
- P01 publications involved significantly more co-authors and partnering institutions per publication than R01 publications.
- P01 publications had a higher average number of citations per publication and a higher Hirsch index score than R01 publications.
- Expanded upon earlier NIH studies
- Entailed an advanced grant matching process
- Studied a larger sample size of grants
- Used a mixed method approach that allowed for triangulation across data sources
- Some of the evaluation questions could not be answered due to the matching variables that were used to construct the comparison group
- It was not possible to develop a systematic measure of “research synergy”
- Some data on research synergy, productivity, and impact were not able to be collected since it was not feasible to field a survey of Principle Investigators during the study timeframe
- Self-reported data from stakeholders may be biased
The Madrillon Group, Inc. and the NIH Library
Poster Presented at the 2017 American Evaluation Association (AEA) Annual Meeting:
- Walton, L., Taylor, B., Washington, D., Solomon, L., & Lockmuller, J. (2017, November). Comparative Study of P01 and R01 Grant Mechanisms at the National Institute of Allergy and Infectious Diseases (NIAID). Presented at the Annual AEA Conference Meeting, Washington, DC.