Science magazine sting investigation finds widespread abuse in open-access publishing

The “open-access world includes many legitimate journals,” stipulates Science magazine’s introduction to a 4 October special collection of articles on science communication. But abuse, it charges, is “prevalent.” In an investigation over the past 10 months, the magazine explains, “contributing correspondent John Bohannon submitted faux papers with blatant scientific flaws to 304 open-access journals. More than half accepted the paper.”

Bohannon’s report , which immediately began drawing media attention around the world, details his elaborate methods for submitting “hopelessly flawed” papers about purported wonder-drug-yielding experiments. “Acceptance was the norm, not the exception,” he writes. “The paper was accepted by journals hosted by industry titans Sage and Elsevier. The paper was accepted by journals published by prestigious academic institutions such as Kobe University in Japan. It was accepted by scholarly society journals.”

An article at the Verge sums things up this way:

Anticipating criticism for Bohannon’s methods, Science also stipulates, “Granted, some ‘traditional’ print publications might have fallen for our hoax, too.” The criticism indeed appears, for example, at the blog Retraction Watch, with its motto “Tracking retractions as a window into the scientific process.” The blog observes that “it’s important to note, given the heated and endless debates between open access advocates and traditional publishers, that there was no control group.” At the website of the Australian Broadcasting System, open-access advocate Paula Callan writes , “It was at least acknowledged in the article in Science that it was entirely possible that the fake articles would also have been accepted by bottom tier subscription journals. It is a shame this was not tested.”

One passage in Callan’s commentary summarizes where Bohannon’s sting fits into the open-access world and ends by offering a judgment on what’s been learned:

Elsewhere overseas, Bohannon has drawn attention in Calcutta. He writes, “About one-third of the journals targeted in this sting are based in India—overtly or as revealed by the location of editors and bank accounts—making it the world’s largest base for open-access publishing; and among the India-based journals in my sample, 64 accepted the fatally flawed papers and only 15 rejected it.” At the Telegraph in Calcutta, an article about Bohannon’s investigation opens by introducing Madhukar Pai, identified as an associate professor at McGill University in Montreal who “is among scientists worried that India has emerged a hub for dubious open-access research journals that compromise” on peer review. The article continues:

An article at National Geographic quotes a response from Nicholas Steneck , who is director of the Research Ethics and Integrity Program of the Michigan Institute for Clinical and Health Research, a professor emeritus of history at the University of Michigan, and a consultant to the US Department of Health and Human Services’ Office of Research Integrity. He said, “The public wanted open access to scientific literature, and now they are getting it. They now need to get over the idea that they can get all that information for free without someone doing the real hard work of reviewing papers.” But the article also quotes Michael Eisen, a University of California, Berkeley, biologist and vocal open-access advocate, concerning Bohannon’s spoof. He said, “In all honesty, I think it is kind of a general indictment of peer review.”

Within the first few hours after Science released Bohannon’s article, coverage of it appeared at the Boston Globe , the Los Angeles Times , Discovery News , SciDev.net , UPI.com , Inside Higher Ed, and the Toronto Star , among other news sites.

In a few cases—the Economist , National Geographic, and Retraction Watch—observers have compared Bohannon’s sting with the mid 1990s hoax by physicist Alan Sokal, who submitted the parody article “Transgressing the boundaries: Toward a transformative hermeneutics of quantum gravity” to the journal Social Text. In a parallel Lingua Franca article, Sokal explained that he had wanted to see whether “a leading North American journal of cultural studies” would “publish an article liberally salted with nonsense.”

It did. In August 1996, Nobel laureate physicist Steven Weinberg commented on that incident at some length in the New York Review of Books. Sokal, Weinberg wrote, aimed at “surreal” views of “fashionable academic commentators on science who question the claims of science to objectivity"—that is, at postmodernists who “surf through avant garde fields like quantum mechanics or chaos theory"; at sociologists, philosophers, andhistorians “who see the laws of nature as social constructions"; and at others who find political and cultural coloring “not only in the practice of scientific research but even in its conclusions.” Sokal’s hoax, Weinberg declared, “served a public purpose, to attract attention to what Sokal saw as a decline of standards of rigor in the academic community, and for that reason it was unmasked immediately by the author himself.”

At Science, Bohannon too has sought to serve a public purpose by attracting attention to a decline of academic standards, and Bohannon too has unmasked his own hoax. The article at the Economist warns, “With the number of open-access papers forecast to grow from 194,000 in 2011 (out of a total of 1.7m publications) to 352,000 in 2015, the Bohannon hoax ought to focus editors’ minds—and policymakers’, too.”

---

Steven T. Corneliussen, a media analyst for the American Institute of Physics, monitors three national newspapers, the weeklies Nature and Science, and occasionally other publications. He has published op-eds in the Washington Post and other newspapers, has written for NASA’s history program, and is a science writer at a particle-accelerator laboratory.