Heeding the lessons of BICEP2

On 28 February we learned that an RF telescope isolated from the electromagnetic clamor of civilization had detected a signal with enormous implications. After years of painstaking analysis to filter out noise from the atmosphere, the galaxy, and technology, researchers concluded that the faint residual radiation hails from the dawn of the universe. The result broadly confirms mainstream theoretical models but also includes a twist: The signal was stronger than expected, which raises hopes of new physics.

My colleague Steve Blau explains the scientific details in a 12 March dispatch. In brief, scientists with EDGES (Experiment to Detect the Global Epoch of Reionization Signature) suspect the signal stems from hydrogen-atom emissions over a 90-million-year period shortly after the universe’s first stars ignited. The EDGES team, which extracted the signal from an FM antenna in the Australian desert, supplemented its discovery paper with a theorist’s proposal that the signal’s unexpected strength indicates the mingling of dark matter and ordinary atoms via a weak nuclear interaction.

If something about a monumental whisper from the early universe sounds familiar, perhaps it’s because almost exactly four years ago, a research team made a cosmological claim that also had theorists simultaneously celebrating and scrambling to make sense of it all. In a much-hyped March 2014 press conference, the BICEP2 collaboration announced that its telescope in Antarctica had detected a definitive signature of inflation (see Physics Today, May 2014, page 11 ). Within weeks, researchers expressed concern that the team had not properly accounted for the signal-contaminating effects of galactic dust; by the following year, the evidence for inflation had been almost entirely negated (see Physics Today, April 2015, page 17 ).

In astrophysics circles, BICEP2 is already a legendary cautionary tale. “BICEP2 has entered into the experimental subconscious,” says Brian Keating , a physicist and BICEP2 researcher from the University of California, San Diego, who just wrote a book called Losing the Nobel Prize: A Story of Cosmology, Ambition, and the Perils of Science’s Highest Honor. Considering the actions of the EDGES scientists and the journalists who covered their result, it’s clear that BICEP2’s impact extends further. The phantom detection of cosmic inflation has influenced and will continue to influence the way high-risk, high-reward results are analyzed, presented, and reported.

EDGES principal investigator Judd Bowman says the first hints of a tantalizing result came in the fall of 2015. Like their BICEP2 counterparts, he and his team sat on their data long before making them public. “We originally thought it was wrong,” he says. The signal “was bigger than we thought [it should be], and we were only looking at a small slice of the frequency band.” Yet as more data arrived over a broader spectrum, the signal persisted.

Bowman and his team spent the next two years ruling out foreground sources of the radio-band signal, including the antenna itself. They ensured that the data were consistent whether or not the halo of the Milky Way was dominating the sky. (Due to the frequencies in question, ruling out galactic effects should be much easier than it was for BICEP2 scientists, who infamously estimated Milky Way dust emission via a PowerPoint slide.) They rotated the antenna, removed a potentially noise-generating component, and tested the antenna in the lab. “We were cautious,” Bowman says. “Our mind-set was that this was an error in the instrument.” Yet by last fall, they still had a surprising but highly significant result that, unlike BICEP2’s, wasn’t in tension with previous measurements. Their interpretation had survived every verification test. The EDGES team decided to submit a paper to Nature.

Over the course of a half-hour phone call with Physics Today covering the ins and outs of the data analysis and the decision to publish, Bowman never brought up BICEP2. But when asked about it, he acknowledged that “BICEP2 was certainly on our minds. It still is.” He’s far from the only experimental physicist since 2014 to have BICEP2 on the brain. Keating notes that Harry Collins’s Gravity’s Kiss, a book that offers an inside view of the ultimately successful quest to directly detect gravitational radiation (see Physics Today, December 2017, page 53 ), is peppered with BICEP2 references from anxious LIGO physicists.

Avoiding two major pitfalls of the BICEP2 affair, the EDGES researchers went through the full peer-review process and never planned to announce their results in a flashy press conference. But they faced a similar challenge in having to defend a measurement that was both extraordinarily difficult to attain and extraordinary in value. Not only was the signal amplitude higher than expected, just as BICEP2’s was, but the bottom of the curve was oddly flat. Bowman’s team decided to contact a few close theorist colleagues to see if there was a viable explanation for the quirky curve. Rennan Barkana , a theoretical astrophysicist at Tel Aviv University, came back with the idea that the interaction of hydrogen and low-mass particles of dark matter could account for the anomaly.

In the 1 March issue of Nature, Barkana’s work appears immediately following the EDGES team’s paper , which has a modest title (“An absorption profile centred at 78 megahertz in the sky-averaged spectrum”) and focuses solely on the 21-cm signal and its verification. “We intentionally kept [the studies] separate so the instrument paper could say what it needed to say,” Bowman says. But by pairing their paper with Barkana’s, Bowman and his team took a calculated risk: They assessed that providing a plausible explanation for their results was worth the overhype that would result from proposing new physics, especially about dark matter.

Just as Bowman and his team faced a challenge in framing, so did the science writers who covered the pair of papers. Judging by how many instances journalists used the term “if confirmed,” at least some of them seemed to have BICEP2 in mind. National Geographic used the question-in-the-headline approach to convey skepticism: “Universe’s first stars detected? Get the facts. ” It helped that in media interviews, Bowman and his colleagues emphasized the need for a confirming measurement from a different telescope.

Yet largely because of the Barkana paper, the general public was also exposed to a lot of speculation about exotic dark matter and Nobel Prizes. Four years ago Harvard astronomer Avi Loeb said that if confirmed, the BICEP2 result was worth a Nobel ; this time multiple outlets quoted him saying that the EDGES result could net two Nobels . The New York Times‘s article explained the significance of the dark-matter interpretation but didn’t hedge about the EDGES measurement until the 20th and final paragraph.

A Google News search indicates that no article to date has included the word BICEP2, although one writer referenced it indirectly by noting that “we have been burned before.” We’ll soon find out whether that will happen again with the EDGES claim. A handful of scientists have brought up potential instrument errors, though Bowman says he hasn’t seen anything that concerns him. Meanwhile, theorists are already slicing and dicing the data, and at least one team has all but rejected Barkana’s dark-matter interpretation. The self-correcting nature of science is in full force.

Whether the EDGES results hold up or go the way of BICEP2’s, the scientists in both collaborations are inexorably linked. They accepted the enormous challenge of using a single instrument to look for a faint signal they thought was there, leaving them vulnerable to charges of both systematic errors and confirmation bias. After performing check upon check of the data, they had to decide whether to publish a result that could either yield a Nobel Prize or get nullified in an instant by an observant engineer on Twitter. Keating and his BICEP2 colleagues came out humbled. For their part, Bowman and his team continue to tinker with their instrument and verify their data as they await that all-important confirming measurement.