Picking Cherries and Hot Fudge

In the comment thread to a post on the recent Der Spiegel article about the politicization of climate science a long series of comments has resulted from the following exchange about the IPCC presentation of the so-called Hockey Stick graph. First, Der Spiegel wrote the following (translated from the original German):

But what appeared at first glance to be fraud was actually merely a face-saving fudge

A frequent and thoughtful commenter said:

IMO hiding uncertainty _is_ research misconduct.

To which I responded:

I hear what you are saying, however, in the world of academia, this is just not the case. If it were, most work across most field would be guilty of such charges ;-)

Those two sentences have resulted in several lengthy comment threads, with many comments critically directed at me, fueled by a cheap-shot post by Andrew Montford who has since apologized and asked for clarification, which is fair enough. Here is the clarification.

First, what are we even talking about?

We are talking about the presentation by the IPCC of Figure 2.1. in the Third Assessment Report.

Why does this matter?

From the perspective of policy, it does not matter. Let me be very clear about what I mean by this: My views on climate policy are not sensitive to any of the information presented in IPCC TAR 2.21. The Hockey Stick could be rotated 180 degrees or mirror-imaged or shaken like a jump-rope and it would not alter my views of climate policy one bit. When, back in 2005, I asked Steve McIntyre and some of his critics at Real Climate about why the debate matters for climate policy, there seemed to be a consensus that that time that it did not.

Of course, things are a bit more complicated than this. For many observers, especially those more skeptical, the Hockey Stick debate has come to serve as a proxy for the integrity of all of climate science and the IPCC. It thus serves as a political proxy for larger debates about climate change. For better or worse, it is an important lens through which people view the practice of all of climate science.

What happened?

What happened with respect to the presentation of the IPCC graph is that uncertainties were hidden from view in an effort to present a "tidy story." Steve McIntyre has well-documented the events in a large number of posts, but this one pretty much tells the story.

From the CRU emails we are privvy to some of the inside discussion about this issue (quotes from here). One scientist explained that there are some complexities that should probably be discussed (emphases added):

I know there is pressure to present a nice tidy story as regards 'apparent unprecedented warming in a thousand years or more in the proxy data' but in reality the situation is not quite so simple… [There are] some unexpected changes in response that do not match the recent warming. I do not think it wise that this issue be ignored in the chapter.

Another scientist wanted that "nice tidy story" and protested that if they included the data showing "unexpected changes" then (emphases added):

. . . we have to comment that “something else” is responsible for the discrepancies in this case. . . Otherwise, the skeptics have an field day casting doubt on our ability to understand the factors that influence these estimates and, thus, can undermine faith in the paleoestimates. I don’t think that doubt is scientifically justified, and I’d hate to be the one to have to give it fodder!

You can see the problematic data series in the following early version of the IPCC 2.21 graph below in yellow, with its pronounced decline at the end.
You can also see the "decline" at the end of the series that was at odds with the instrumental record -- it was this divergence which complicated the "tidy story." But as you can see in the final IPCC figure 2.21 that was published (shown above) decisions were made to "hide the decline" and present a "nice tidy story."

Was it Fraud or Something Else?

The facts of what happened here should not be controversial. A group of scientists associated with the IPCC decided to simplify the presentation of paleo-climate data in orger to convey a "tidy message" and to try to avoid the "skeptics" some "fodder" with which to have a "field day." Of course, those plans backfired pretty badly!

The selective presentation of information is a form of cherrypicking, a widely used term that I described in 2004 as follows:

the careful selection of information to buttress a particular predetermined perspective while ignoring other information that does not

The actions by the IPCC scientists to "hide the decline" were a form of cherrypicking.

Semantics aside, anyone seeking to bring charges of formal scientific misconduct against these scientists or the IPCC would have a tough -- I'd say impossible -- case to make. To understand this you need to know what scientific misconduct actually is, and how it is judged. The scientists here were clearly shaping a message, explicitly motivated by a political agenda against the "skeptics" and grounded in a fundamental distrust of policy makers and the public to be able handle the complete information.

As I argue in The Honest Broker, the penalties for cherrypicking are a loss of credibility and legitimacy in scientific and political processes. But make no mistake, everyone is selective in the information that they present when making an argument, scientific or political. In fact, there is no practical way to present all of the information and all of the uncertainties. The question thus is not whether one seeks to manage the presentation of uncertainties -- but rather have you made a argument that is fair, robust and legitimate? Meeting these criteria typically means being open about uncertainties, warts and all. If you are perceived not to have made such an argument then you may see some of the following occur -- your paper could be rejected for publication, your peers might disagree with your views, your policy arguments dismissed, your credibility diminish and so on. Thus, while the selective presentation of information is unavoidable, one cherrypicks at some large risk.

What I find most remarkable about this episode, and others in which the IPCC has been found cherrypicking, is ultimately how very trivial the episodes actually are. Just take a look at the two figures above -- is the science of climate change undermined by that loose yellow thread dangling down in the original graph? Are non-experts incapable of understanding the uncertainties associated with the "divergence problem"? The IPCC has clearly allowed or perhaps even enabled some serious lapses in judgment by its authors by allowing them to risk so much for so little.

This episode is not about scientific fraud -- at least in the way that I understand the concept to be defined in the academy. This episode if one of several --too many -- in which the IPCC was found to be risking its credibility to present a "tidy story." Hopefully, the scientific community has learned that a desire for tidiness should not trump an overarching concern for maintaining the credibility and legitimacy of expert advice, even if that means presenting the science alongside uncertainties and complexities.

But there are lessons here for critics as well. It is reasonable to feel betrayed, angry and upset that the experts tried to play you for a fool. But making wild accusations of fraud and calling for legal sanctions (and worse) simply diminishes your own credibility and represents an ironic sort of overreach.