Dutch IPCC Report Generally Useful, But Botches US Hurricanes

The Netherlands Environmental Assessment Agency has issued a report (PDF) that assesses the regional chapters of the 2007 IPCC Working Group II. It finds that there are indeed errors, referencing problems and unsupported claims, but overall judges these issues to be minor. The press release that the NEAA issued is titled, "Key findings of IPCC on regional climate-change impacts overall considered well founded." My reading of the report is that it is considerably more hard-hitting than the press release, with recommendations for significant changes in the IPCC.

Last March, I was contacted by the NEAA during their investigations about the IPCC's presentation of hurricane loses in North America (note that this is a different issue than the global loss issue that I have discussed here before, which appears in Chapter 1 of the WGII report, not a regional chapter). Since I have discussed the (mis)treatment of US hurricane losses in the relevant IPCC regional chapter in depth on this blog (which I shared with and explained to the NEAA) I was surprised to see the NEAA completely botch its treatment of US hurricanes.

At p. 78, the NEAA report discusses the hurricane issue as follows:

Figures TS.15 and 14.1 on the economic damages from hurricanes. (minor)

Figures TS.15 (Technical Summary, page 55) and 14.1 (Chapter 14, page 621) in the Working Group II Report illustrate the statement in the Technical Summary that reads ‘Over the past several decades, economic damage from hurricanes in North America has increased over fourfold, due largely to an increase in the value of infrastructure at risk’. We have a minor comment to make on the figures. Because of a shorter averaging period at the end of the time series (6 years instead of 10), the most recent increase was higher, compared with the increase that would have resulted from applying a uniform averaging period at the end of the series. However, a different choice of statistical method would not have affected the statement. Furthermore, we have found this statement to be fully supported by the underlying material. This minor comment has no consequences for the IPCC conclusions in the various Summaries for Policymakers.

The NEAA is correct that the IPCC presented a misleading graph (the NEAA did not choose to report that the IPCC had invented data for the endpoint of Figure TS.15). But the NEAA is incorrect to claim that the following statement is "fully supported by the underlying material':

‘Over the past several decades, economic damage from hurricanes in North America has increased over fourfold, due largely to an increase in the value of infrastructure at risk’

Either the graph or the statement necessarily must be in error. IPCC Figure TS.15 can be seen at the top of this post. It shows data taken a paper that I was lead author on (Pielke and Landsea 1998) and appends data that did not come from our paper (where it came from is unknown, but it is in error). The data shown in the grpah shows hurricane losses after adjusting for societal change. That is to say, there is no signal of "an increase in the value of infrastructure risk" in the data because our methodology is designed specifically to take out that signal!

Thus, any increase in losses over a period of "several decades" shown in the graph must be due to climatic factors (such as decadal variability), most obviously, the fact that the 1970s saw the least losses over the period covered by our dataset and 2005 had hurricane Katrina. The increase in losses shown in Figure TS.15 from the 1970s to 2005 cannot be due to "infrastructure at risk." Had the graph instead shown non-normalized data, the statement would have been more accurate.

If the IPCC had accurately represented the data, rather than the misleading graph, it would have been clear that over the period of record there has been no increase in normalized losses due to hurricanes in the United States. Here is what the data actually showed (Source: PDF) about US hurricane losses at the time that the IPCC produced Figure TS.15 and wrote the incorrect statement:

Overall, the NEAA report looks to be a valuable effort in assessing the IPCC WGII regional chapters. However, on the topic of hurricane losses in the United States, the NEAA is in obvious error, which is unfortunate because the issue is so straightforward. I cannot judge the other parts of the report. However, it does include some hard hitting advice for the IPCC, including:

• Create a public website for the submission of possible errors found in the published reports;
• Provide stronger underpinning of generalisations of case studies to entire regions or sectors, also making use of regional modelling studies;
• Ensure that statements that attribute impacts to climate change are well founded in scientific research, including systematic observations, modelling and statistics. The climate change component of impacts should be carefully characterised.
• Be careful with phrasing of statements that could be perceived by readers as heightening the projected impacts of climate change;
• IPCC governments should provide financial support for hiring chapter assistants to assist with quality control;
• Assure that the reviews of all draft texts are fully covered by several expert reviewers;
• Strengthen the expert and government reviews of the foundation for and provenance of statements in the summaries;
• IPCC governments should increase their investments in climate-change observations and modelling in developing countries.

The IPCC would do well to listen, but the tone of the press release might be misinterpreted as suggesting that business-as-usual is OK. If so, that would be a shame.