07 December 2009

Do Sloppy Policy Arguments Matter? Part I

The subject of this post is the EPA endangerment finding. But the focus is not on whether or not the finding is appropriate. I am not a legal expert, but I am of the view that the Obama Administration is perfectly justified in advancing the finding, which as I understand it requires a very low scientific threshold -- do specific human activities lead to changes in the environment? And might those changes potentially lead to some degree of harm? I think that both of these thresholds are easily met, though the politics of implementation are likely very difficult. However, the finding itself is not the focus of this post. This post focuses on how some of my research is presented in the EPA endangerment finding and in the EPA response to public comments.

To be clear, nothing in this post should be interpreted as opposing the EPA finding or more generally as opposing action on addressing accumulating greenhouse gases in the atmosphere. This post should be read in the context of a continuing series on the systematic misrepresentation of the science of climate change and disaster losses. Does such misrepresentation matter? Well, it matters to me because it is my research being misrepresented. Here is the latest installment.

On p. 132 of the Technical Summary (PDF) that accompanies the endangerment finding is an image, shown below. The highlighted yellow circle shows a graph with a green line representing dramatically increasing hurricane damage.The endangerment finding does not provide an original source for the graph, but it does point to the IPCC as the source of the overall figure. The overall figure comes from p. 621 of Chapter 14, WG II of the 2007 IPCC report (PDF) and is shown below.

The highlighted text reads:
. . . economic damages, million U.S. dollars (adjusted to constant 2005 US dollars and normalized accounting for changes in personal wealth and coastal population to 2004), and deaths from Atlantic hurricanes since 1900 (data from Pielke Jr. and Landsea, 1998 updated through 2005)
So our work is the source of the graph showing dramatically increasing hurricane damage in the 20th century. The bibliography explains exactly where the data came from:
Pielke, R.A. and C.W. Landsea, 1998:Normalized hurricane damages in theUnited States: 1925-95. Weather and Forecasting, 13, 621-631. with extensions through 2005 at http://www.aoml.noaa.gov/hrd/tcfaq/E21.html
The Technical Summary of WGII, p. 55 (PDF) also shows the hurricane damage graph (TS.15) as follows:

Using an internet archive, I was able to access the NOAA page that presented a non-peer reviewed update of our 1998 paper as of December 2006, which was immediately prior to the release of the IPCC AR4 report. A problem with that data was immediately evident: The NOAA page at that time did not yet have data for the 2005 hurricane season. After publication of the IPCC in 2007 the NOAA page was updated with a value for 2005 of $120B (see it here). So the IPCC must have manually entered a 2005 value, and that value looks to be $100 billion (which is in error), based on my replication of the TS.15 figure shown below (as $120B doesn't match up with the IPCC graph).

There are two problems here insofar as the IPCC is concerned. One is that the IPCC failed to rely on peer-reviewed research in its presentation of the science of disasters and climate change, contrary to its mandate and how it represents its work to policy makers and the public. The second is that the IPCC engaged in its own analysis of data, which also goes beyond its mandate and representation of its activities. In doing so the IPCC introduced a misleading perspective on long-term trends in hurricane damage in the United States, one that found its way into the EPA endangerment finding despite the availability of correct, up-to-date information.

The real story of US hurricane losses can be seen in the following graph, which is peer reviewed (PDF) and has been readily available to the authors of the EPA endangerment finding for almost two years. For that matter it could have been available to the authors of the IPCC, as the paper was completed well before their deadline.

This graph shows clearly that normalized hurricane losses (see the black curve represented an 11-year moving average over 1900 to 2005) have not increased over the period of record. The lack of trends is consistent with the lack of trends in landfalling storms. Our paper concluded:
The two normalized data sets reported here show no trends either in the absolute data or under a logarithmic transformation: the variance explained by a best-fit linear trend line=0.0004 and0.0003, respectively, for PL05, and 0.0014 and 0.00006, respectively, for CL05. The lack of trend in twentieth century normalized hurricane losses is consistent with what one would expect to find given the lack of trends in hurricane frequency or intensity at landfall. This finding should add some confidence that, at least to a first degree, the normalization approach has successfully adjusted for changing societal conditions. Given the lack of trends in hurricanes themselves, any trend observed in the normalized losses would necessarily reflect some bias in the adjustment process, such as failing to recognize changes in adaptive capacity or misspecifying wealth. That we do not have a resulting bias suggests that any factors not included in the normalization methods do not have a resulting net large significance.
In the public comments and responses from the EPA (PDF) a number of reviewers noted that our work was contrary to the implications of the endangerment finding:
Several commenters (1616.1, 3136.1, 4528, 4632R18, 4666, 4670, 4766, and 7020) argue that increasing raw damage totals associated with tropical storm damage in coastal communities are not the result of climate change, but rather are caused by changes in coastal population, wealth of the population, inflation, and poor sediment management practices. A couple of these commenters reference Pielke et al. (2008) as supporting evidence.
The EPA apparently disagrees with the findings in our paper. However, one could argue that the EPA's response was off target (emphasis added in below, see original for the full non-responsive response):
With regard to insured losses from extreme weather events (e.g., coastal flooding), the USGCRP concluded that economic and demographic factors do not fully explain the upward trend in costs or numbers of observed increases in losses. For example, from 1980 to 2004, the U.S. population increased by a factor of 1.3, while losses increased by a factor of 15 to 20 in inflation-corrected dollars. Numerous studies indicate that both the climate and the socioeconomic vulnerability to weather and climate extremes are changing (Brooks and Doswell, 2001; Pielke et al., 2008; Downton et al., 2005), although these factors’ relative contributions to observed increases in disaster costs are subject to debate (Karl et al., 2009). Analyses asserting minimal or no role of climate change in increasing the risk of losses are limited in that they often 1) tend to focus on a highly limited set of hazards and locations; 2) fail to account for the vagaries of natural cycles and inflation adjustments; and 3) fail to normalize for countervailing factors such as improved pre- and post-event loss prevention (e.g., dikes, building codes, and early warning systems) (Karl et al., 2009). The USGCRP concludes that future increases in losses will be attributable to climate change with high confidence as climate change increases the frequency and intensity of many types of extreme weather (Karl et al., 2009). Please see Section 1 of this volume for our specific response regarding the Downton et al. (2005) paper and other studies
Note how the EPA uses Pielke et al. 2008 to suggest that both climate extremes and societal vulnerability are changing. I have shown above that Pielke et al. 2008 cannot be used to support such a claim. In fact, the USGCRP report cited by the EPA concluded that hurricane landfalls have decreased over the long-term.

In Part II, focused on floods, I'll show how Brooks and Doswell 2001 and Downton et al. 2005 also can't be used to support the claims being made by the EPA in the endangerment finding and in the EPA response to public comments (the former paper uses our methods and the second I co-authored).

What we have here is a clear case of extreme sloppiness by the IPCC followed by some very dubious interpretations of the literature by the EPA. Does it matter? Well, it matters to me.


  1. Okay, it's late at night and I could afford to re-read this more carefully, but correct me if i'm wrong —

    are you not saying here, that the EPA, working with IPCC data that the IPCC misrepresented, and from which the IPCC drew a conclusion opposite to your conclusion in the paper they cited, has then used this data to support, in part, the endangerment finding?

    Maybe I'm just a cranky old guy, but "it matters to me" would seem a very understated way of expressing what my reaction would be.

  2. Roger, I agree that the EPA may not have represented the science right, but IPCC is actually quite careful in their assessment. In the TS of WG2 the text related to the figure that you highlight reads (page 55):

    "Over the past several decades, economic damage from hurricanes in NorthAmerica has increased over fourfold (Figure TS.15), due largely to an increase in the value of infrastructure at risk [14.2.6]".

    And that Section 14.6.6 of WG2 (page 626) actually reads:

    "Trends in the number and intensity of extreme events in North America are variable, with many (e.g., hail events, tornadoes, severe windstorms, winter storms) holding steady or even decreasing (Kunkel et al., 1999; McCabe et al., 2001; Balling and Cerveny, 2003; Changnon, 2003; Trenberth et al., 2007: Section"

    And "North America very likely will continue to suffer serious losses of life and property simply due to growth in property values and numbers of people at risk (very high confidence) (Pielke Jr., 2005; Pielke et al., 2005)."

    So although EPA may draw the wrong conclusions, IPCC deserves a bit more credit here.

  3. The alarmist mantra is becoming "fake, but accurate".

  4. -2-Laurens

    Thanks for your comments. And I agree that some of these comments of the IPCC are much better representation of this issue (e.g., those on p. 626) area than others I have taken issue with, e.g.,


    However, they are still wrong in important respects. For instance, consider:

    "Over the past several decades, economic damage from hurricanes in North America has increased over fourfold (Figure TS.15), due largely to an increase in the value of infrastructure at risk [14.2.6]".

    This sentence is referencing a graph of _normalized_ damages. Thus, there is no signal of "value of infrastructure at risk" in the data here. Thus, while the statement is correct with respect to non-normalized data, it confuses the issue by pointing to a graph that shows something very different.

    Also, with respect to the non-normalized data that is not shown, the word "largely" is not correct -- over the long-term "entirely" is more accurate. Over the shorter term, say 1970 to 2005, the increase in damages is not due to largely to an increase of value at risk, but short-term variability in hurricane landfalls, as the 1970s were a very quiet period. And even then the increase is a function of endpoint chosen.

    In some respects, I am actually more willing to cut the EPA some slack as that is a political document whereas the IPCC claims to be among the most peer-reviewed assessment documents in history.

  5. But should EPA not also come up with a balanced scientific view? The EPA technical document says:

    "This document itself does not convey any judgment or conclusion regarding the question of whether GHGs may be reasonably anticipated to endanger public health or welfare, as this decision is ultimately left to the judgment of the Administrator. The conclusions here and the information throughout this document are primarily drawn from the assessment reports of the Intergovernmental Panel on Climate Change (IPCC), the U.S. Climate Change Science Program (CCSP), the U.S. Global Change Research Program (USGCRP), and the National Research Council (NRC)."

    Of course, the analysis is politically motivated, but should EPA not provide the science?

    I think that IPCC failed to come with a single, clear, and balanced conclusions concerning the impact of anthropogenic climate change on disasters. The information is all over the 4AR now. A Special Report is being prepared by IPCC solely on weather extremes, let's see what that says...

  6. -5-Laurens

    "should EPA not provide the science?"

    In principle, yes. In practice the EPA relies on the IPCC and CCSP/USGCRP, the latter which will be a focus of Part II.

    In short, if the IPCC and CCSP/USGCRP don't do their jobs right, EPA cannot overcome this.

    Lets see what that special IPCC report says indeed . . .

  7. Even better than the TC data is fig A snow water equivalent. I liked to see the correlation coefficient for that line.

  8. Roger -

    If we want rational policy then we should have rational support for policy.

    It drives me nuts when "policy wonks" misrepresent the certainty associated with supporting evidence. But at least you can expect that from politicians.

    When the exageration and misrepresentation is endemic to the documents that are provided as evidence then a critic who argues with the certainty claims gets branded as arguing with all of the claims.

    Ultimately that can only lead to a backlash when the exagerated scenarios fail to materialize, and the backlash tends to err in the opposite direction, such as we are seeing now re: "climategate"

  9. Hi,

    Another interesting bit about that graph is that the power dissipation index (PDI) looks like it would a linear regression line with essentially zero slope over the entire period.

    Or even better, throw out the final point for both damages (green diamonds) and PDI (blue dots) and it looks like the regression lines would be almost completely flat.

    So it's that final piece of damages data that really gives the impression that something dramatic is happening.

    As you point out, the more accurate story can be seen from your PLO5 (Pielke-Landsea 2005) data with the 11-year centered averages.

    It would be even better to get that PLO5 curve carried through to 2009, to show that the 2004 and 2005 high-damage years were followed by several relatively low-damage years (2009 has to be close to a record low, doesn't it?), so the 11-year centered average doesn't continue upward (from the black line in your PLO5 graph, which ended in 2000).

  10. The Canadian burned area is incredibly disingenuous.

    As shown by:

    Girardin, M. P., J. Tardif, and M. D. Flannigan, 2006. Temporal variability in area burned for the province of Ontario, Canada, during the past 200 years inferred from tree rings, Journal of Geophyical Research, 111, D17108, doi:10.1029/2005JD006815.

    Canada has had extensive fires in the past, and recent fires doesn't even compare to the burned area of bygone eras.

    Moreover, on a global scale, as shown by:

    Riaño, D., J.A. Moreno Ruiz, D. Isidoro, and S.L. Ustin. 2007. Global spatial patterns and temporal trends of burned area between 1981 and 2000 using NOAA-NASA Pathfinder. Global Change Biology, 13, 40–50.

    "There was also no significant upward or downward global trend in the burned area for any individual month."

  11. I am curious as to who EPA consulted with or contracted to answer the questions. Did they defer to USGCRP to answer questions of science?

  12. of course it matters.

    for two reasons:

    1) sloppy science hidden as good science is a violence against the scientific institution
    2) sloppy science to support policy is a violence against all citizen

    so it matters a lot.

    Thanks for all your post.