Science is currently in the grip of a ’reproducibility crisis’ so severe that the editor of a prominent journal has declared that ‘much of the scientific literature, perhaps half, may simply be untrue’. Media coverage declaring that ’science is broken’ has become commonplace.
Prior to the 2009 Copenhagen climate summit, Peter Doherty, winner of the Nobel Prize in medicine, defended the Intergovernmental Panel on Climate Change (IPCC) from its critics. The IPCC involves hundreds of scientists and ‘draws its evidence ex- clusively from peer-reviewed, published scientific literature’, he wrote. Around the same time, the IPCC chairman was asked if an Indian environment ministry report might alter the IPCC’s pessimistic view of Himalayan glaciers. The ‘IPCC studies only peer-review science’, Rajendra Pachauri replied dismissively. Until the report’s data appears in ‘a decent credible publication’, he said, ‘we can just throw it into the dust- bin’.
Peer-reviewed research is reliable, so the reasoning goes. Non-peer-reviewed re- search is not. The IPCC makes exclusive use of the former, therefore its conclusions can be trusted. This argument has long been used to deflect criticism and to repel contrary climate perspectives.
But behind it lies a dubious assumption: that academic publications are a sound foundation on which to base real-world decisions. In fact, science is currently in the grip of a ’reproducibility crisis’ so severe that the editor of a prominent journal has declared that ‘much of the scientific literature, perhaps half, may simply be untrue’. Media coverage declaring that ’science is broken’ has become commonplace.
Part 1 of this report demonstrates that a journal’s decision to publish a paper pro- vides no assurance that its conclusions are sound. Large swathes of peer-reviewed work contain errors. Fraudulent research makes it past gatekeepers at even the most prestigious journals. And while science is supposed to be self-correcting, the process by which this occurs is haphazard and byzantine.
A policy cannot be considered evidence-based if the evidence on which it de- pends was never independently verified. Peer review does not perform that function. News from the worlds of astrobiology, ecology, economics, chemistry, computer sci- ence, management studies, medicine, neuroscience, psychology, and physics all tell the same tale: ’peer-reviewed’ does not equal ’policy-ready.’
Part 2 of this report invites us to re-examine what we think we know about the climate. While good scientists have always understood that peer review doesn’t cer- tify accuracy, IPCC officials – supported by politicians, activists, and journalists – think global climate decision-making should rest on this shaky foundation.
If half of all peer-reviewed research ‘may simply be untrue’, half of all climate re- search may also be untrue. The policy implications of this idea are immense.
