When the pie shrinks

When I read the title of James Langer’s editorial in the 12 October issue of Science—"Enabling scientific innovation"—I expected a generic exhortation for the US to invest more money in basic research. What Langer wrote, however, was a despairing indictment of how the US evaluates and funds scientific proposals in these times of tight budgets.

According to Langer, just when advances in experimental and computational techniques have opened up new areas of research, opportunities to fund such research are contracting. Worse, the shrunken funding pie has made peer reviewers and proposal writers averse to risk. As Langer puts it:

In my area of condensed-matter and materials physics, the U.S. National Science Foundation (NSF) can fund only about 10% of the individual-investigator proposals it receives. [The Department of Energy (DOE) has similar difficulties.] Each proposal is sent to a group of peer reviewers, who rank it on a scale ranging from “excellent” to “poor.” NSF then funds only those proposals that receive the uniformly highest reviews. One less-than-"excellent” review, no matter how misguided, is usually enough to doom a proposal. Any proposal that is truly innovative, interdisciplinary, or otherwise unusual is almost certain to be sent to at least one reviewer who will be less than enthusiastic about it. Sensible investigators know not to submit such proposals; as a result, some of the most urgent research areas are disappearing.

Evidence to justify Langer’s fears appeared this week in the form of a commentary in Nature entitled “Research grants: Conform and be funded.” The authors, Joshua Nicholson of Virginia Tech and John Ioannidis of Stanford University, looked at the relationship between a biomedical researcher’s citations (a rough measure of scientific significance) and the level of his or her funding from the National Institutes of Health (NIH).

In particular, Nicholson and Ioannidis examined authors of papers that have garnered more than 1000 citations since 2001. The pair found that

three out of five authors of these influential papers do not currently have NIH funding as principal investigators. Conversely, we found that a large majority of the current members of NIH study sections—the people who recommend which grants to fund—do have NIH funding for their work irrespective of their citation impact, which is typically modest.

Such correlations don’t prove that review panels aren’t funding high-risk, high-reward proposals. But at least to Nicholson and Ioannidis, the correlations suggest that NIH is failing to fulfill its mandate of funding “the best science, by the best scientists.”

In his Science editorial, Langer struggled to come up with a more effective way of funding the best science. He even entertained—but dismissed—the idea of disbursing money through a lottery, which could well do less damage, he wrote, than the current system does.

But there is no more effective way than peer review. When funds are limited, it’s hardly surprising that reviewers become more cautious. Investors do so too. The solution to the review problem is both simple and hard: Increase the size of the funding pie so that reviewers are emboldened to take risks.