17 June 2009

Systematic Misrepresentation of the Science of Disasters and Climate Change

Let me start this post by stating that I am a strong supporter of action on both adaptation and mitigation policies related to human-caused climate change. At the same time I have seen some disturbing things take place in the scientific community. And it is just my luck that the area where I have observed the most shenanigans is the area in which I have considerable expertise -- disasters and climate change.

This post summarizes and reviews the systematic misrepresentation of the science of disasters and climate change in major science assessments, partly for my own purposes, but also to explain that there is a pattern of behavior taking place in this community that should be of concern to anyone who cares about the integrity of science, regardless of their position on climate policies and politics.

What I document below includes the following:

1. Reliance on non-peer reviewed, unsupportable studies rather than the relevant peer reviewed literature.

2. Reliance on and featuring non-peer reviewed work conducted by the authors of the assessment reports.

3. Repeated reliance on a small number of secondary of tertiary sources, repeatedly cited such that intellectual provenance is lost.

The questions that I have are, does anyone in the mainstream scientific or media communities actually care? Or is climate change politics so important that we cannot simultaneously worry about standards of scientific integrity?

1. In 2001 the IPCC Third Assessment Working Group II report cautiously claimed in its Chapter 8 that the upward trend in the costs of disasters had a climate component, and supported this assertion by referencing a non-peer reviewed report by Munich Re published in 2000 surveying natural disasters in 1999. That Munich Re report compared disasters in the 1970s to the 1990s and only speculated on issues of attribution. I provided a critique in the following paper, based on a talk I gave at the Smithsonian in 2006 sponsored by the National Research Council.
Pielke, Jr., R.A., 2006. Seventh Annual Roger Revelle Commemorative Lecture: Disasters, Death, and Destruction: Making Sense of Recent Calamities, Oceanography, Special Issue: The Oceans and Human Health, Vol. 19, No. 2, pp. 138-147. (PDF)
The caution in the report did not stop the head of the IPCC John Houghton from making statements about the attribution of increasing disaster losses to human-caused climate change when testifying before the US Congress, or his successor Rajendra Pachauri from making similar statements in 2005 (both referenced in the paper linked above).

Coincidentally, a lead author of the IPCC Chapter 8 that featured Munich Re's non-peer reviewed work included Gerhard Berz, who worked for Munich Re.

2. In 2005 Science published a commentary by Evan Mills, a scientist at Lawrence Livermore National Laboratory, asserting that,
According to the latest Intergovernmental Panel on Climate Change (IPCC) assessment, climate change has played a role in the rising costs of natural disasters.
And Mills attributed at least part of the increase to "anthropogenic climate change."

What source did Mills use to support this claim? Why the IPCC Third Assessment Report Chapter 8 discussed above, which traces its sourcing to Munich Re, 2000. Mills also cites the Munich Re 2000 reported cited by the IPCC, giving the impression that there are multiple sources of support for his claim (even worse he cites a third report that also relies on Munich Re, 2000). Mills commentary is a fact checker's nightmare. Mills commentary is important because it shows up later and repeatedly as offering support for claims of attribution, even though it only offers a secondary and tertiary citation. Presumably the fact that it was in Science in 2005 gives it greater standing than a 2000 Munich Re report.

3. In 2006, the Stern Review report cherrypicked a single non-peer reviewed paper (Muir-Wood 2006) from a workshop I held and used it to generate an estimate of escalating damages due to greenhouse gas emissions. I examined the Stern Review report in depth and found that it also mysteriously inflated damages by an order of magnitude. I summarized the issues with the Stern Review in a peer reviewed paper:
Pielke, Jr., R. A., 2007. Mistreatment of the economic impacts of extreme events in the Stern Review Report on the Economics of Climate Change, Global Environmental Change, Vol. 17, pp. 302-310. (PDF)
In that paper, I conclude:
This brief critique of a small part of the Stern Review finds that the report has dramatically misrepresented literature and understandings on the relationship of projected climate changes and future losses from extreme events in developed countries, and indeed globally.
Richard Tol found it surprising (PDF) that Stern ignored relevant peer reviewed work and engaged instead in a selective reading of the available literature. Despite being unchallenged, my critique is ignored.

4. In 2007 the IPCC released its Fourth Assessment Report and -- surprise, surprise -- it also relied on the single non-peer reviewed Muir-Wood (2006) study cherrypicked from our Hohenkammer workshop as the single study to highlight in its review of this topic.

Further, the IPCC included a graph attempting to show how closely temperature anomalies match up with disaster losses, using a scaling of the axes to suggest a relationship where none has been shown in the peer-reviewed literature. Again it relies on Muir-Wood (2005).

Coincidentally, Robert Muir-Wood, of Risk Management Solutions, Inc., was an author of the chapter of the IPCC report that selectively highlighted his own non-peer reviewed work.

5. The US Climate Change Science Program systematically and repeatedly misrepresented the science of disasters and climate change.

First, the CCSP US extremes report miscited several of my papers in support of claims that they did not make and relied on Mills 2005 as the definitive source on this topic. The disasters and climate change section of this CCSP report is also a fact checker's nightmare.

Second the CCSP draft Synthesis report and final Synthesis report relied on non-peer reviewed work by Evan Mills and ignored relevant peer reviewed research showing different results (in fact all peer reviewed research points in the same direction on this subject).

Coincidentally, Evan Mills was an author of the CCSP Synthesis Report that highlights his own non-peer reviewed work. Mills also apparently consults for companies with an interest in climate policies, and yet this was not dsclosed by the CCSP.

Summary

The information above documents a pattern of misrepresentation of the science of disasters and climate change in the Stern Review report, the reports of the IPCC, an the US CCSP. The pattern of misrepresentation has three common characteristics:

1. Reliance on non-peer reviewed, unsupportable studies rather than the relevant peer reviewed literature.

2. Reliance on and featuring non-peer reviewed work conducted by the authors of the assessment reports.

3. Repeated reliance on a small number of secondary of tertiary sources, repeatedly cited such that intellectual provenance is lost.

The evidence presented here, and in great detail via the links, is unambiguous and unequivocal in support of my claims. Though if you would like to refute them with evidence, please do so in the comments. Until the climate science community cleans up its act on this subject it will continue to give legitimate opportunities for opponents to action to criticize the climate science community.

13 comments:

Roger Pielke, Jr. said...

Via email,

“Perhaps I am missing something? Clearly, the systematic misrepresentation of science is something we want to avoid in policy analysis and in rigorous scientific assessments. But since when are when are these “science assessments” actually just scientific assessments? The CCSP and Stern Review (and arguable the IPCC reports) are political documents.

I am not suggesting that the authors of these reports deserve a free pass. Obviously they deserve to be taken to task for lacking scientific integrity. This is especially true when the authors purport to be engaged in a rigorous scientific assessment. However, why shouldn’t we expect a political document to cherry-pick information that supports the political arguments of its authors?”

Best,
Dave

Roger Pielke, Jr. said...

-1-Dave

I'd have less of problem with the cherrypicking if they advertised themselves as political documents offering up a partial or slanted view. However they are characterized as something different, such as how John Holdren characterized the CCSP report released yesterday: "most up-to-date, authoritative, and comprehensive".

Thus I do think that the IPCC and CCSP and Stern Review reports should be held to a higher standard than, say, the GHF, Munich Re or ABI reports. However, as we've seen, the lines do tend to get blurred, so rather than lowering standards for IPCC and such (recognizing that they are political documents) I'd say in all cases the best approach is to be both honest and accurate, i.e., to have higher standards. I do appear to be in a minority among my peers with this view;-)

The misrepresentations of my work and the work of others are unacceptable in any context and deserve to be called out as such.

newsfan said...

Mr. Pielke,

I'm doing a short series on this report for Examiner.com, and basically (and unintentionally) just lifted most of your post for an article--I apologise, although I did link here. Would you like to participate in a more structured interview for Examiner? We could do it in the comments section of this blog.

Roger Pielke, Jr. said...

-3-newsfan

No worries.

I'm happy to do an interview, send your questions via these comments or by email. Thanks.

Maurice said...

After a few minutes on the new .gov global warming site yesterday it was clear that the authors were very sure of their predictions of dire consequences.

Hmmmm I thought, maybe they’ve figured out the true effect of water vapor and cloud feedback. Searching the site for “feedback” shows that climate science has a goal of:

“Identify deficiencies in cloud formulations and cloud feedback representations in climate models, and improve cloud processes representations. Quantify the magnitude of aerosol indirect effects on clouds and their effects on precipitation and the broader hydrologic cycle.”

After that I stopped reading and filed away one more episode of climate scientists having low credibility.

Not Whitey Bulger said...

"The evidence presented here, and in great detail via the links, is unambiguous and unequivocal in support of my claims. Though if you would like to refute them with evidence, please do so in the comments."

The power of the group is not their ability to refute you. Their power is in their ability of ignore you.

Dean said...

Evan Mills is an energy specialist with the Lawrence Berkeley National Lake. Phil Cooney was a lawyer for the American Petroleum Institute who, while working for the Bush Administration, changed the contents of reports written on government scientists.

I think you've made good points in your post, and I hope that it gets a good discussion and the appropriate adjustments. But the analogy to Cooney is flawed.

Dean said...

Since you mentioned the IPCC in this post, I downloaded and glanced through section 3.8 of the WGI report - Changes in Extreme Events. It seems to see more evidence for extreme precipitation events than temperature events. But that evidence is focused on regions, and tends to average out for entire continents. But this is to be expected since AGW impacts will vary over areas. I'm not referring to losses here, which I know is one of your complaint focuses.

Regarding drought, FAQ 3.3 says:

The Palmer Drought Severity Index calculated from the middle of the 20th century shows a large drying trend over many Northern Hemisphere lnd areas since the mid-1950's, with widespread drying over much of southern Eurasia, northern Africa, Canada and Alaska, and an opposite trend in eastern North and South America.

-----

It also points out that while some areas have not had worse droughts in the sense of less rain, warmer temps and longer dry seasons exacerbate droughts that otherwise would have been considered similar.

I understand your criticisms and hope that they are taken seriously such that either corrections are made or better responses are offered. But I also don't want to lose sight of the broader picture. For example, the IPCC report describes evidence for higher extreme temperature events as "sparse". We are in a phase where absolute evidence is lacking but circumstantial evidence is growing regarding extreme events. Trends are not identifiable and isolatable AGW signals, but the signals are noisy. Some reports overstate the certainly of this circumstnatial evidence. But it is what it is. I'm not saying this as a criticism of you, since are clear exactly what it is you criticize. Just for the larger picture.

newsfan said...

Dr. Pielke,

I'm ready to begin an interview with you here in the comments which will be published on Examiner.com, although I'll happily let you preview the article before I publish it. Here's a link to what I wrote yesterday: http://www.examiner.com/examiner/x-9111-SF-Environmental-Policy-Examiner~y2009m6d17-Global-warming-debate-Climate-scientist-says-he-was-misrepresented-in-Obamas-Climate-Change-report

Before I start, I'd like to invite your readers and commenters to join in with questions--I interviewed another Roger Pielke recently, and some of the questions asked by readers were -ahem- a bit more intelligent than mine.

I will start the interview with a brief bio if you have one available (and can you also provide a picture in jpeg format?) I will then summarise recent events regarding the report.

1. Could you summarise your view of global climate change for us?
2. What forces seem to be driving the changes in sea, air and surface temperatures we have witnessed over the past 30 years?
3. Who in the debate is playing fairly and who is not? (I have not yet gotten a complete answer to this question from your father, Stephen Schneider or Bjorn Lomborg--maybe you'll be the first...)
4. Is climate science such an elite field that experts in other domains cannot offer qualified commentary? I'm thinking of Freeman Dyson and Ivar Giaever, specifically.
5. What do you think the true sensitivity of our atmosphere is to a doubling of CO2?
6. Do you think we are measuring temperatures accurately on land, sea, air or from space?
7. Your father clearly believes that anthropogenic contributions from land use, interruptions to the hydrologic cycle and deforestation are a first order forcing on climatic conditions. What is your opinion?
8. What prescriptions would you offer to zoning regulations, building permits, architectural standards and community siting to make American communities more resilient to the impacts of large scale weather-related events?
9. The government report has, you say, mischaracterised your research. How would you rephrase their comments so that it would accord with your published work?
10. Would you care to offer a figure for average surface temperatures on this planet in 2100?

If, after responding to these questions, you still have time and energy for this, I may follow up with other questions. If your readers do chime in with questions, please feel free to respond to them.

I also recall that some readers were kind enough to rephrase my questions so that they were more relevant--if either you or they feel that's appropriate here, feel free to do so.

Many thanks in anticipation.

Tom Fuller

Roger Pielke, Jr. said...

-9-Tom

I'll be happy to address these, most likely next week. I will let you know that questions 2, 5, 6, 7, 10 are outside of my expertise, so please feel free to send along any substitutes.

Thanks

newsfan said...

Hello again,

Here are replacement questions for 2, 5, 6, 7 and 10. Thanks.

2. If you were a member of Congress, would you vote for the current cap and trade legislation?

5. As a citizen, do you believe that President Obama's energy program takes the right direction, commits the correct level of resources, and is likely to be beneficial for this country?

6. Has the energy of tropical storms increased or decreased over the past 30 years? Are there regular cycles to tropical storm strength, and how does this affect your answer?

7. Has the actual number of tropical storms increased or decreased over the past 30 years? Are there regular cycles to tropical storm numbers, and how does this affect your answer?

10. On a scale from 1-10, where 1 is not at all important and 10 is of the highest importance, where would you rank global climate change in terms of its likely impact on human development? Please feel free to explain.

Tom

Chris said...

I want to say that this is one of the most interesting threads I've ever encountered on a blog. Thanks for doing the interview in public! And Roger, your tenacity regarding scientific integrity in policy forums is tremendously appreciated.

Roger Pielke, Jr. said...

Tom- Thanks. I'll give some thought to these and see if I can get them up on Monday.

Post a Comment