17 June 2009

More on the CCSP Report

I took a look at the public review comments on the CCSP Synthesis report released yesterday that were submitted to the CCSP (here in PDF) earlier this year and there are frequent mentions of my work and how it is at odds with the statements in the report. The following comment from Chip Knappenberger and CCSP response are telling (from p. 299).

Knappenberger:
Citation 27 is to Mills, Science, 2005. The Mills paper has come under intensive criticism, largely from Dr. Roger Pielke Jr—another expert in the field of disaster losses. Pielke Jr. published a response to the Mills paper in Science.

It is inconceivable that the authors of the CCSP report don’t know of the Pielke Jr. criticisms (after all Evan Mills is listed as a CCSP author) and yet it is equally inconceivable that knowing of them, that absolutely no reference is made to them.

The fact is, is that Pielke Jr. concluded “Presently, there is simply no scientific basis for claims that the escalating cost of disasters is the result of anything other than increasing societal vulnerability.” (Pielke Jr., Science, 2005). This is precisely the opposite of the CCSP conclusion. How can such criticism be completely ignored?
CCSP Repsonse (I wonder who wrote it -- I'd bet that is was a particular guy at LLNL, given that its statements that come right out of Mills's response to my Science letter):
The referenced paper is a synthesis of a large literature, and also parallels conclusions from IPCC/TAR/WG2/Ch8 (which should also be cited at this juncture). The referenced criticism took the form of a brief letter from Pielke Jr. to Science, which was answered in detail in the same issue by Mills. It would be beneficial to cite this exchange of letters, and, space allowing, draw out the many factors not accounted for in simplified “normalization” procedures. Considerable effort has been made to normalize historic offices (sic) upward to account for factors such as inflation, but little has been done to quantify the countervaling (sic) factors of improved construction practices, early warning systems, and other adaptive responses that reduce losses.

The referenced exchange of letters in Science has been cited.
Of course, as I documented earlier, in the final version the CCSP did cite my response to Mills to support a claim that I never made. Some improvement. Mills' response to my critique of his 2005 commentary in Science provides no data and does not directly address the substance of my critique, which is grounded in various peer-reviewed studies that I cite ( I provided a critique of Mills' response at the time). Mills' response does have this interesting statement (PDF):
Assuming that only socioeconomic factors— rather than rising emissions—influence losses may yield ill-founded policy recommendations that focus exclusively on adapting to climate change while dismissing energy policy as a legitimate part of the toolkit for responding.
You could not ask for a clearer statement of political necessity driving the need for certain scientific conclusions. The CCSP reports were supposed to summarize the peer-reviewed literature, not to serve as a place to air out unfounded critiques that have never appeared in the peer reviewed literature in order to protect favored policy choices. If Mills thinks that "contervailing forces" hide a greenhouse gas trend in the losses, then he should do the work necessary to support that claim and then publish it. Of course, we have considered that fact in our work on hurricanes and floods, e.g., for hurricanes because the trend in losses matches the trend in hurricane landfall events there is no bias introduced by adaptation, as this would have shown a trend in normalized losses differing from the trend in geophysical events. Regardless, the place for arguing about science is in the peer reviewed literature, not behind the scenes in an assessment report. If there are in fact different points of view to be found in the literature, they a good assessment will report on these differences. In this case, the peer-reviewed literature is unambiguous. Whomever wrote this section just didn't happen to like what it said. Further, if it was Mills writing about his own work it shows how incestuous the process is.

More broadly, there are dozens of peer-reviewed studies not referenced by the CCSP relevant to this subject. The focus on a single study, not peer reviewed and written by an author of the CCSP, should be of concern to anyone, regardless of their views on climate policy and politics.

5 comments:

climatefan said...

Roger,

Who made the decision to invite Mills to write this portion of the Report? Wouldn't it make more sense to invite you to write portions of the report that involve your expertise?

Regards, Bill

moderator said...

Roger,

I have seen the 'review' processes of these reports many times and it simply does not appear to work. Too many cases where criticism is ignored based on flimsy arguments, incorrect arguments or whatever the authors cook up to refute the criticism. I guess that in academic circles it not really something new be critical about the review process, but outside of academics it is highly regarded.

I have been wondering about it for a long time, and again and again keep thinking of the hypothetical situation where one scientist claims 'A', and another one claims 'B'. How are you going to decide who is right and wrong? Are there guidelines how to treat the criticism? How do you ensure that both views are properly addressed in your report? What does that mean, 'properly addressed'? Who decides on how to weigh arguments if you decide to address them both? And by what measures: by vote, by counting how many scientists support a certain view, by something else? What if both sides continue to disagree, and even oppose publication of eachothers views? What if one view gets preferential treatment because of who is on the committee or because of politics? What - in general - safeguards the quality in the review process? What are the procedures and guidelines? I don't see any, but if someone has them ...

I guess that in science this has always been a problem - in general, and for many ages - but that science has also found a workaround in the form of an open debate which is called scientific literature.

Regards, Jos.

Roger Pielke, Jr. said...

Chip Knappenberger offers his perspectives on the handling of his review comments here:

http://masterresource.org/?p=3332

ChipKnappenberger said...

Roger,

Thanks for the link.

As you mentioned, the CCSP authors did make a few changes based upon my comments and those of others. However, as you point out here, and as I do over at MasterResource, the changes, in many cases at least, never really altered the general idea originally laid out by the CCSP--even though, in many cases at least, they should have!

-Chip

Joel Upchurch said...

One of the moderators over at PhysicsForums.com started a thread on the Earth Forum about the CCSP report. Since I had read John Tierney article in NY Times about Dr. Pielke's comments about the report and followed his link over here and read them. I thought the comments were useful and posted a link to Dr. Pielke's comments.

I checked back later and discovered that the moderator had deleted my posting. He said I could only use peer-reviewed sources. I was surprised since the guidelines say: "Controversial claims must be supported by evidence that comes from a scientific, peer-reviewed journal or a similarly reliable source, i.e., unsubstantiated claims are not allowed." From my experience on that forum was this was usually interpreted to mean that comments by actual climate scientists were okay when they were commenting on their areas of expertise.

Once I linked to Dr. Roy Spencer's blog to his explanations about the differences in how the UAH and RSS temperatures are compiled and that was commented on favorably by all.

I pointed out that Dr. Pielke was a leading expert on environmental policy and commenting on the use of his own papers and that in any case the CCSP report itself wasn't peer-reviewed and they replied that it was an official government report and met their guidelines.

I'm sorry Dr. Pielke. You are apparently not a reliable source to comment on your own papers, at least according to one moderator on PhysicsForums.

Here is a link to the original post:
http://www.physicsforums.com/showthread.php?t=320815

Post a Comment

Note: Only a member of this blog may post a comment.