In 2005 Gavin Schmidt and Michael Mann, of Real Climate and CRU email fame, carefully explained that the process of peer review is a messy, incremental way to advance knowledge in fits and starts:The current thinking of scientists on climate change is based on thousands of studies (Google Scholar gives 19,000 scientific articles for the full search phrase “global climate change”). Any new study will be one small grain of evidence that adds to this big pile, and it will shift the thinking of scientists slightly. Science proceeds like this in a slow, incremental way. It is extremely unlikely that any new study will immediately overthrow all the past knowledge.They explained that even when results are published that do not stand the test of time, the process of peer review can successfully winnow out those arguments with the greatest merit:
. . . even when it initially breaks down, the process of peer-review does usually work in the end. But sometimes it can take a while.With this perspective as background, one of the most damning aspects of the CRU emails was the behind-the-scenes efforts of the activist scientists to -- in their own words -- "redefine what the peer reviewed literature is."
Peer review as related to scientific publishing is a process in which experts are asked to judge the appropriateness of a paper for publication in a scientific journal. It is often cursory and focused on the merits of an argument, rather than a detailed replication or decomposition of the data or methods. Peer review does not mean that a result is right or will stand the test of time, but that it has met some minimal standards of acceptability for publication. The scientific community is replete with vignettes about papers that were rejected for publication in one venue only to be published elsewhere and which later turned out to be seminal. Similarly, every so often even Science and Nature find themselves in trouble with a paper that is badly wrong or even fraudulent. But despite these shortcomings in the process, peer review is widely viewed much as Winston Churchill viewed democracy: the worst possible system except for all the others.
Peer review works because over the long-term good ideas win out, and this process happens organically and through a decentralized process. Peer review takes place through many independent journals, with editing and reviewing conducted by many independent scholars from a diversity of disciplinary and experiential backgrounds, and with their own idiosyncratic biases and views. No one group or perspective owns the peer review process, and that diversity is part of its core strength. Truth -- meaning a convergence to agreement on scientific questions -- thus is a product of the peer review process over time. Of course the path to truth can be convoluted and indirect. For instance, it used to be true that there were 9 planets in our solar system. Now that is less true.
Some issues relevant to decisions are characterized by uncertainties and contested certainties making the distribution of scientific views not readily apparent simply by looking at the sprawling literature. In such situations a formal assessment can provide a useful perspective on the degree of consensus or disagreement among relevant experts on various claims. Such assessments are nothing more than a snapshot in time, as science is continuously evolving. When done well, an assessment will reflect the full range of views held by relevant experts, including minority views (see PDF), as well as the connections of scientific understandings to alternative possible courses of action.
Now back to the CRU emails. The emails show a consistent pattern of behavior among the activist scientists to redefine peer review in accordance with their own views of climate science. In doing so, they sought to turn the entire notion of peer review on its head.
The emails show a group of scientists frustrated with the peer review process, seeking to change how it is practiced. How so? The emails indicate concerted efforts to reshape the peer review process by managing and coordinating reviews of individual papers, by putting pressure on journal editors and editorial boards, by seeking to stack editorial boards with like-minded colleagues, by arranging boycotts of journals and other actions involving highly questionable ethics. But we might wonder why these scientists would take such steps to change peer review if, as Schmidt and Mann explained at Real Climate -- "peer review usually does work in the end." Why depart from a process that works? The answer is obvious: the short-term politics of climate change.
The activist scientists decided that the peer review process would work better in service of their political agenda if it used "truth" to determine whose views would be allowed to be published in the literature and reflected in assessments. In this case "truth" simply means the views deemed acceptable among the activist scientists and their close clique of colleagues. In an interview with NPR Real Climate's Gavin Schmidt defended this very backwards view of peer review:
So Schmidt suggests that in order to short circuit the ability of their political opponents to cherry pick and blow out of proportion studies that the activists scientists did not agree with, they saw a convenient short cut: Simply reshape the peer review system such that those papers don't ever appear or go unmentioned in scientific assessments.Journals are supposed to be impartial filters that let good ideas rise to the top and bad ideas sink to the bottom. But the stolen emails show that a group of scientists has decided that's not working well enough. So they have resorted to strong tactics — including possible boycotts — to keep any paper they think is dubious from reaching the pages of a journal.
"In any other field (a bad paper) would just be ignored," says Gavin Schmidt at NASA's Goddard Institute for Space Studies in New York. "The problem is in the climate field has become extremely politicized, and every time some nonsense paper gets into a proper journal, it gets blown out of all proportion."
Most of the papers Schmidt and his colleagues object to challenge the mainstream view of climate science. Schmidt says they may be wrong or even deceptive, but they are still picked up by politicians, pundits and businesses who are skeptical of climate change.
The problem with this strategy, of course, is that many climate scientists (and presumably others inside and outside of the scientific establishment) are unwilling to cede ownership of the "truth" to a small clique of scientists. In fact, peer review exists in the first place because there are no short cuts to the truth, and any such short cut will inevitably fail. Consider that the efforts revealed in the CRU emails to manage the peer reviewed literature went well beyond efforts to prevent so-called "skeptical" papers from being published, but included a focus on papers that fully accepted a human influence on climate, but which offered views that differed in some degree (e.g., here) from those preferred by the activist scientists. The emails reveal activist scientists busy extolling the virtues of peer review to journalists and the public, while at the same time they were busy behind the scenes working to corrupt the peer review process in a way that favored their views on the science and politics of climate change. Here we have a case study in the politicization of climate science by climate scientists.
The clique of activist scientists sees absolutely nothing wrong in what they are doing -- they are after all justifying their actions in terms of "truth" in support of the greater good. And the issue is made even more complex because those who share the political agenda of the activist scientists are ready to join their peer review coup whereas those opposed to that political agenda are happy to try to exploit for political gain the scientists' ethical lapses and failure to appreciate their role in politicizing climate science. So much of the discussion gets wrapped up in these distractions, rather than the issue of the integrity of climate science.
The sustainability of climate science depends upon our ability to distinguish the health of the scientific enterprise from the politics of climate change. The need to respond to climate change (which I support) does not justify sacrificing standards of scientific integrity for political ends. In fact, as the events of the past week show, when standards of scientific integrity are compromised, the political consequences can be double edged.