The death of expertise has been exaggerated

In June 2011 the National Academies Press made all of its reports freely available to anyone to download. The press is the publisher for the National Academies of Sciences, Engineering, and Medicine (NASEM). When people access the free reports, NASEM presents users with an optional request: “Please take a moment and tell us how you will be using this PDF.” Two and a half million people worldwide have since left a comment.

Diana Hicks of Georgia Tech and her collaborators have used machine learning to analyze all the comments that originated in the US. ¹ The largest group of users turned out to have personal IP addresses rather than academic or business ones. That finding is heartening to those worried, as the authors put it, about “information bubbles, fake news, the spread of misinformation, manipulation of social media users, Twitter bots, etc.”

About 23% of downloaders left comments, which ranged from single words, such as “research,” to multiple sentences, such as “I have a long-time interest in astronomy. I am a newspaper reporter but probably cannot use this in my work. One never knows, however.” To crunch through the heterogeneous data, Hicks and her collaborators used a natural-language classification algorithm called BERT (Bidirectional Encoder Representations from Transformers). The result was a 64-term taxonomy of report usage divided into six broad categories.

Of the broad categories, “education and research” accounted for the largest share at 48%. Next came “governance” at 18% and “information activity” at 17%. The third-placed category encompasses recreational learning of the kind I do when I read scholarly books about art, history, and literature.

In their paper, Hicks and her collaborators list the 25 most downloaded NASEM reports out of the 10 275 included in their study. The top three are A Framework for K–12 Science Education: Practices, Crosscutting Concepts, and Core Ideas (206 000 downloads), The Future of Nursing: Leading Change, Advancing Health (125 000 downloads), and How People Learn: Brain, Mind, Experience, and School: Expanded Edition (74 000 downloads). Education, health, and policy accounted for almost the entire top 25.

The absence of basic science from the top 25 prompted me to ask Hicks about the 2010 astronomy and astrophysics decadal survey, New Worlds, New Horizons in Astronomy and Astrophysics. (The most recent decadal survey fell outside the study’s time range.) She told me that the report had been downloaded 28 000 times, which placed it at 37 on the most downloaded list. As for the comments, the report attracted a lower than average fraction that were classified as research. Whereas personal use accounted for 8% of average usage, it accounted for 22% of the 2010 decadal survey’s usage. It was clear from the comments that the report was popular among amateur astronomers.

It’s not surprising that the most downloaded reports are about topics that affect people’s lives rather than, say, the prospects for research in biological physics, which is the subject of one of NASEM’s most recent reports. ² But I can’t help wondering if another factor is at play. The two most downloaded reports in Hicks and company’s study were covered in the New York Times and elsewhere, as was the 2010 decadal survey. It’s conceivable that media coverage is a necessary condition for a NASEM report to be popular. Indeed, a 2017 survey by the Pew Research Center found that most Americans receive science news through general media outlets. ³

Newspapers, radio stations, TV channels, and other media have to make money to survive. The editors and producers who decide what science news to publish and broadcast have to gauge what interests their audiences. Should NASEM and other science organizations follow mainstream media and prioritize public interest when it comes to reports? It would be a pity if biological physics and other topics that the general public considers esoteric were no longer the subjects of reports. Still, given the resources deployed to produce them, some consideration should be given to the likely, not hoped-for, readership.

The research conducted by Hicks and her collaborators illuminates the rewards for successfully engaging the public. In the concluding paragraph of their paper, they write: “The overall impression is of adults motivated to reach higher, seek out the most credible sources, engage with challenging material, and use it to improve the services they provide or learn more about the world they live in.”