A proposed solution to arbitrary evaluations

It was a great pleasure to read Toni Feder’s feature “When tenure fails ” (Physics Today, October 2023, page 44). In particular, I enjoyed her insightful discussion of unwritten rules, including that of being a “good fit,” in the tenure process. As Meg Urry, director of the Yale Center for Astronomy and Astrophysics, is quoted as saying of tenure candidates, “The vast majority could go either way, depending on how people spin it. It’s kind of arbitrary.” That tells me that the tenure evaluation system has failed.

Unfortunately, such flaws exist not only in job securement—including when a candidate is selected for a tenure-track position from among the few interviewed—but also in many other activities of scientific research, including peer review of scientific publications and grant applications. The need for evaluation reform is imperative for continued scientific innovation.

The ideal environment for innovation is open, diverse, democratic, tolerant, and motivating. How can we create and maintain such an environment?

As one possible solution, I have proposed a new open-science initiative: the Open ePrint and Rigorous Evaluation System for STEM. ^{1 , 2} Three types of research activities—original research, indirect contributions (for example, participation in the evaluation system), and funding requests—would be evaluated using quantitative, continually refined metrics. The rigorous, community-based evaluation system would reward the quality, not the quantity, of accomplishments. Community members would earn credits for their research and other activities, and as they accumulate experience and credits, they could advance in their role in the community. High-risk, high-reward research projects would have a better chance of being funded.

By implementing a sophisticated credit- and role-based incentive mechanism, we could establish a new self-sustaining ecosystem for the entire scientific community. Rigorous science requires rigorous evaluation, and scientific innovation won’t develop efficiently until such a system is built.