Witnesses
Cora Marrett
Deputy Director, National Science Foundation
Keith R. Yamamoto
Vice Chancellor for Research, University of California San Francisco
Nancy B. Jackson
President, American Chemical Society
Jorge José
Vice President for Research, Indiana University

 

Subcommittee Members Present
Mo Brooks, Chair (R-AL)
Daniel Lipinski, Ranking Member (D-IL)
Hansen Clarke (D-MI)
Paul Tonko (D-NY)
Larry Bucshon (R-IN)
Andy Harris (R-MD)
Randy Hultgren (R-IL)

 

On July 26, 2011 the House Committee on Science, Space, and Technology Subcommittee on Research and Science Education held a hearing to discuss the merit review process for awarding federal grants at the National Science Foundation (NSF). Chairman Mo Brooks (R-AL) introduced the hearing stating it is the responsibility of this subcommittee to “ensure federal dollars are being spent on the best science.” Brooks acknowledged that the process is not flawless, as any process involving human decision-making is not perfect. He cited that only 23 percent of the proposals received were funded in fiscal year (FY) 2010. With such limited funds, Brooks stressed the importance of a transparent and effective merit review process.

 

Ranking Member Daniel Lipinski (D-IL) related a personal story about a NSF grant he was awarded in graduate school.  He spoke of his “strong belief in the NSF and its mission” in his opening statement. Lipinski referred to NSF’s merit review process as the “gold standard for the world,” but stated it is important to continually evaluate the process and look for innovative improvements. He suggested the committee revisit the topic once the National Science Board has finished the review underway with a report expected to be released in the fall of 2011. 

 

Cora Marrett gave an overview in her testimony of the merit review process and how it helps NSF achieve its mission of funding research across science and engineering fields. Ultimately its mission is to “maintain and strengthen the vitality of the U.S. science and engineering enterprise.” She continued, “The NSF merit review process lies at the heart of the agency’s strategy for accomplishing its overall mission. As such, NSF is continuously striving to maintain and improve the quality and transparency of the process.” Proposals are evaluated by their “intellectual merit” and the “broader impacts” of the research. Marrett praised the success of the merit review process, but mentioned NSF is actively looking at new approaches. One of these innovative approaches is the use of virtual panels which would reduce travel costs and expand the pool of reviewers. She explained that 96 percent of the proposals undergo internal and external merit review. The others are waived from the external merit review due to the urgency or high-risk nature of the research, in which case they are handled internally through the Grants for Rapid Response Research (RAPID) or the Early Grants for Exploratory Research (EAGER) process. RAPID funding is reserved for urgent proposals that may have limited access to data or are a post disaster “quick-response” research. EAGER funding is reserved for potentially “high risk-high payoff” exploratory research.

 

Keith Yamamoto spoke generally of how a merit review process must work and what his experiences have taught him about an effective process. In any merit review process, the challenge is to eliminate any conflict of interest on behalf of the reviewers. Yamamoto praised that “in general, these intrinsic conflicts have been addressed successfully by well-crafted regulations, and more importantly, by a universal ‘culture of respect’” on behalf of the reviewers. He stated dissatisfaction with how the broader impacts are evaluated by telling the subcommittee, “This criterion, as stated, would in my view adversely affect the merit review process because it departs from the singular focus on scientific merit that is essential to the process, and because it obligates peer reviewers to judge grant applications by metrics outside of their expertise.”

 

Nancy Jackson testified on the importance of government funded research. “Although much of the nation’s chemical research is carried out by scientists, engineers, and technicians employed in industry and academia, the federal government is an important source of support, particularly for the basic research conducted by our nation’s universities and government laboratories.” She spoke of the success of the NSF merit review process, attributing it in part to the involvement of the “collective wisdom of the scientific community.” Regarding the broader impacts criterion, Jackson believed it is a way to ensure the funds go to the “most pressing” research needs. She suggested that giving NSF research managers the freedom to remove those proposals at the “very bottom of the pile” would alleviate and balance the NSF workload.

 

In his testimony, Jorge José stated his support for merit reviews, calling it “the most effective process we have for ensuring that federal funds are used most effectively.” José testified, however, that the broader impacts criterion should only be used to distinguish between proposals of equal intellectual merit and should not “outweigh scientific considerations.” He spoke of the need for the U.S. to be wise with the scarce resources available to ensure that it remains the “envy of the world.”

 

During questioning, Brooks challenging Marrett on why, in FY 2010, 1,300 “excellent” proposals were denied, yet 98 “fair to good” and 2,643 “good to very good” proposals were funded. Marrett responded that those apparent discrepancies come because of the role of the Program Officers, who oversee the funding. NSF does not rely exclusively on the recommendations of the panel. Additionally, she said it was important to keep in mind that the amount of funding varies for all the proposals. The chairman requested Marrett submit the specific award amounts for each of the ratings. Chairman Brooks asked how to best ensure funding for transformative research is granted based on merit rather than the interest of the reviewer. José explained that from his experience at Indiana University scientists are usually conservative in that they are less willing to risk funds towards projects that are not likely to succeed. Representative Larry Bucshon (R-IN) questioned Marrett on how NSF keeps politics out of the review process and ensures it is a consistent and impartial process. She explained that all reviewers sign a conflicts form, program officers must go to ethics training, and that NSF has begun offering ethics courses and workshops for students at the undergraduate level and above. Additionally, if they see any evidence of problems or fraud, the issue is taken to the Inspector General. Marrett stated the process has its checks and balances to ensure the decisions do not rely on a single individual.

 

Representative Hansen Clarke (D-MI) asked witnesses to suggest any additional measures by which NSF can ensure taxpayers’ dollars are going towards broader impact goals. Marrett responded although they are constantly seeking to improve the process and diligently spend taxpayers’ dollars, challenges remain. NSF realizes that it is much easier to think in terms of short-term returns, yet they cannot dismiss the mid- and long-term returns. Jackson and Yamamoto agreed with Marrett, each stating the importance of long-term returns rather than immediate returns. José spoke of investigations trying to quantify the jobs and revenues created from individual research projects, which he stated is a difficult problem to solve. 

 
Representative Randy Hultgren (R-IL) questioned whether reviewers are “privy” to what is occurring in Congress and are aware of the national goals and priorities if they are to review proposals based on broader impacts. José believed scientists are aware, but warned that restrictions based on these goals might create more harm in the long run. Yamamoto felt it is “inappropriate” for reviewers to address the broad impact criterion. He stated they should not be mandated to “come up” with a broader impact, often forcing the scientists outside their expertise. Marrett explained that NSF has always tried to evaluate research based on the societal impact it could have. The America Competes Reauthorization Act of 2011 (P.L. 111-358) simply forced NSF to make the objections clearer. 

 

During the second round of questioning, the chairman asked the witnesses if they had other suggestions to improve the review process. Marrett stated that NSF’s commitment must be to the merit of the proposal and that any changes to the process must be marginal and should not change the core of the merit. She said she would welcome the opportunity to return to the committee once NSF has finished its current investigations on the merit review process. Jackson suggested a greater emphasis on workshops in writing proposals, especially for younger scientists. This would save time in the review process. Lastly, all witnesses expressed an interest in exploring virtual meetings for the review process as a way to reduce costs. The chairman stated the importance of informing the public on what kind of research is being funded by NSF, referring to the “shrimp on a treadmill” misconception that some Americans have about NSF.   

 

Opening statements from the chair, witness testimonies, and an archived webcast can be found at the committee web page.