05 November 2012

Loss Normalization Methodologies: Technical Thread

UPDATE: Skeptical Science has written a bazillion-word post "responding" to my WSJ op-ed in which they (a) do not contest a single empirical claim made in the op-ed, and (b) demonstrate a complete lack of understanding of what it is a loss-normalization seeks to accomplish.

There have been a lot of questions about the methodologies of loss normalization. As might be expected, there is also some poorly informed understandings of this area of research. Unfortunately (and tellingly), some of the activist blogs on climate purportedly interested in science have banned me from commenting or edit my comments to delete substantive content. This post and the comment thread that accompanies it is for technical questions and discussions related to loss normalizations (and you can see my related research here).

A few points to note to start:
  • Human-caused climate change is real
  • Some measures of climate extremes have changed, notably temperature and precipitation extremes, and have been linked to human forcings
  • However, on climate time scales there has not been detection (much less attribution) of increasing disasters (intensity or frequency) to human-caused climate change
  • This conclusion holds globally and regionally
  • The peer reviewed literature and the IPCC SREX are consistent on this point 
  • If you are looking to see changes in the climate system, then look at climate data, not loss data (climate data can however be used to test the fidelity of loss normalization methodologies)
I may add to this list as needed.

Anyone is welcome to participate here and if you see claims made elsewhere that you'd like to inquire about, just enter a comment. Please do respect the focus of this thread.  Thanks!

42 comments:

Smith said...

Dr. Pielke, is the recent Munich Re report that seems to have resulted in so much discussion similar to a peer reviewed scientific report? Were its methods subjected to independent review?

The reason I ask is that this report is being cited in the blogosphere extensively as if it were equivalent to peer reviewed literature. Ususally reports put out by organizations with a vested interest are viewed with some scepticism, however this one seems to be taken as read by many.

Roger Pielke, Jr. said...

-1-Smith

Thanks. I don't know what review process if any Munich Re employed for its report. However, I do have a copy of the report and can review it myself;-)

Of continuing interest is the fact that research supported by Munich Re and published in the actual peer reviewed literature has arrived at conclusions opposite from those it announces in press releases:

http://rogerpielkejr.blogspot.com/2010/11/mixed-messages-from-munich-re.html

Those in the blogosphere replying on science-by-press release rather than the IPCC and peer reviewed literature have a lot more in common with their mortal opponents then they might care to admit;-)

dbostrom said...

Roger, Tamino's point in the item you cite is fairly crisp: counting losses is not the same as counting natural catastrophes.

Perhaps this is just crossed wires?

Roger Pielke, Jr. said...

On that Skeptical Science "response ... they write:

"Pielke's argument is that the normalized costs associated with hurricane damage have remained relatively flat over time because thus far we have been able to adapt with improving technology"

Clueless.

dana1981 said...

Roger, if you feel that wasn't the point you were trying to make in your op-ed, then maybe you should try, oh I don't know, making an explicit point instead of blowing a bunch of climate denier dog whistles.

And if you're going to draw any meaningful conclusions you should really make an effort to account for improving technology. The fact that normalized economic costs haven't increased as technology has improved dramatically should really tell you something.

Brian said...

I read a story in the LA Times that quoted you (hopefully accurately) and the blog post above. My interpretation of your position is that the hypothesis that climate change will lead to more weather related disasters is not supported by the data. There was a Science article several years ago that reached the same conclusion, specifically about hurricanes. This does not mean the theory is illogical or impossible; there is just no evidence to support it, at least not yet.

I understand the desire to attribute disasters to climate change. Many skeptics say so what if the mean global temperature increases one degree, or the sea level rises nine inches. You accept human caused climate change; do you see a reason to be concerned about that? Do you think the loss data will eventually show a climate component, or is that the wrong reason to be concerned?

Ron Broberg said...

Interested in your response to the 'climate extreme' component that appears to exist in the increasing Federal Crop Insurance indemnity payouts.

http://rhinohide.wordpress.com/2012/11/04/federal-crop-insurance-indemnity-and-extreme-weather

Roger Pielke, Jr. said...

-7-Ron Broberg

Thanks for the comment. There are a lot of important moving parts to crop insurance not covered in that blog post ... perhaps most importantly Congress has dramatically expanded the program via legislation since the 1990s, see:

http://www.cfare.org/conservationcrossroads/Sumner-Zulauf_Final.pdf

It would be hard to detect a climate signal in a 22 year record dominated by policy change.

Roger Pielke, Jr. said...

-5-dana1981

Thanks for visiting. I am assuming that you are the author of the Skeptical Science post.

"climate denier dog whistles"

You caught me ... if you re-arrange some of the letters found in my op-ed they read "Al Gore has a fat carbon footprint" ;-)

Let me be clear -- you guys should be embarrassed to have written that post. It shows that you did not do your homework and haven't the first clue about the research that you are trying to critique.

It is so bad that it is not even wrong, a lot like those folks who argue that climate change is caused by Mars.

If you do have an interest in actually learning a bit about this area of research, I suggest that you slow down a bit and first understand what the concept of "normalization" actually means as you clearly do not.

Thanks!

Joshua said...

dana -


Roger thinks that technological changes, as well as improved forecasting, have had no meaningful effect on damages. Seems a bit hard to believe to me, and highly speculative - but the way to prove him wrong would be to provide hard data that show otherwise.

Roger Pielke, Jr. said...

-10-Joshua

A test for an unaddressed bias in our hurricane loss normalization methodology can be seen here:

http://rogerpielkejr.blogspot.com/2010/11/why-there-are-no-trends-in-normalized.html

Because the trends match up, there is no evidence of a significant bias.

You might also consult various replications and evaluations of the data, such as found in this partial list (from memory) -- Pielke and Landsea (1999), Katz (2000), Schmidt et al. (2010), Nordhaus (2010), Bouwer (2012) among others.

But you are correct, if anyone thinks that they can develop a better normalization, the peer-reviewed literature awaits!

Thanks.

dana1981 said...

I'm not really interested in childish insults, Roger, which seems to be all you're willing to respond with here. If you would like to critique my post with some substance and defend your mistakes, you are of course welcome to comment on SkS.

Roger Pielke, Jr. said...

-12-dana1981

Thanks, but I'll pass.

Do your homework, write a post with some substance, leave your dog whistles behind, and perhaps then let's chat. But I won't hold my breath.

Thanks for stopping by.

dbostrom said...

Roger, off the top of your head can you think of a recent literature review treating loss normalization as it relates to the topic of extreme weather and climate change? I see many publications (though not a great diversity of authors) on this topic, with some variation in findings. I wonder if anybody has synthesized the past 10 years or so of publications to help those of us not in the field see where it's headed? Or is the methodology of normalization as it relates to climate change in stagnation?

Joshua said...

So Roger - just to make sure I understand: we have increased population and property at risk with the same rate of storms and no increase in damages.

And you seem to have argued that improved construction practices has had no preventative effect and you haven't, as far as I can tell, quantified any positive effect from improved forecasting.

Seems like a rather odd configuration of data.

Roger Pielke, Jr. said...

-14-dbostrom

Here you go:

http://journals.ametsoc.org/doi/abs/10.1175/2010BAMS3092.1

You might also look at:

http://onlinelibrary.wiley.com/doi/10.1111/j.1539-6924.2012.01880.x/abstract;jsessionid=B59EBBC2E501A87ED747BC8BFBA02D4D.d04t04

Roger Pielke, Jr. said...

-15-Joshua

The purpose of a normalization is to remove the signal of societal change present in a loss record. If a normalization is successful, what will remain will be a climate signal -- of event frequency and intensity.

For hurricanes this was tested in several ways:

1. By detecting the signal of ENSO variations in the normalized data (see Katz 2000).

2. By comparing long-term trends in hurricane landfall frequency and intensity to trends in normalized losses.

All matches up.

What this tells you is that any remaining factors in the data are collectively not large enough to create a bias.

This dataset has been examined for >15 years and replicated using multiple methods. It has also been evaluated by cat modelers using dynamical models. Karen Clark and Co. recently published an alternative normalization focused on insured losses.

What puzzles me is why anyone would expect to see a trend in normalized hurricane losses given that there are no trends in hurricane landfall frequency or intensity. Any answer to that?

Thanks.

Joshua said...

Roger -

==]] What puzzles me is why anyone would expect to see a trend in normalized hurricane losses given that there are no trends in hurricane landfall frequency or intensity. Any answer to that? [[[===

Increased population and hence property at risk would seem to suggest there'd be an increase in damages unless there was a corresponding decrease in storm frequency.

That is, unless improved construction methodology and improved forecasting lead to mitigation and more resilient property. But as near as I can tell you are asserting that there has been no improved resiliency due to changes in construction, and as near as I can tell you haven't quantified any positive influence from better forecasting.

Seems to me to leave a hole in your analysis somewhere.

Joshua said...

Sorry - my bad. It's too late. I looked again at your question and it was about NORMALIZED loses (obviously meaning controlled for increased property at risk).

Roger Pielke, Jr. said...

-18-Joshua

You'll want to brush up on the notion of normalization, I don't think you get it. Here you go:

http://sciencepolicy.colorado.edu/publications/special/normalized_hurricane_damages.html

Thanks

Joshua said...

So then the answer to your question is that I also can't see any reason why anyone would expect to see a trend in normalized loses if there were no trend in intensity or frequency.

dbostrom said...

Thanks you.

Ron Broberg said...

-8-Roger
"Thanks for the comment. There are a lot of important moving parts to crop insurance not covered in that blog post ... perhaps most importantly Congress has dramatically expanded the program via legislation since the 1990s, see:

Thanks for the link. The most dramatic changes, however, occurred in 1994 when Congress made crop insurance mandatory in certain cases and the program acreage doubled. I am using an indemnity/acre index specifically to account for the variance in acreage. I'll pop a quick post soon showing it.

If you have a more technical comment then "it's hard", I'll be glad to explore it.

Roger Pielke, Jr. said...

-23-Ron Broberg

Thanks ... you'll also want to adjust for the changing mix of crops and their market prices, which are important because federal crop insurance is actually price insurance (not crop insurance).

Presumably drought is the most important factor influencing crops (hail might be a distant second). With drought showing a long-term decline, seems a hard case to make.

Thanks!

Joshua said...

Roger -

Here's how I see it.

Obviously, the point is that some folks don't trust your process of normalization.

However, it does seem that as you say, with a valid methodology of normalization, if known variations (or lack thereof) in frequency or intensity (or power dissipation) show corresponding variations (or lack thereof) in the damages as shown in your data, you have controlled for the societal variables.

Yet, it seems counterintiuitive that improved construction technology and improved forecasting would leave no signal. The explanation you provided through a link, as to why changed construction has no influence, seemed to me to be a pretty poor argument (for some reasons I explained in comments on the other thread). Neither is saying that sometimes building mitigation infrastructure has effects counter to the desired effects.

You have not offered (just from reading your posts) any argument to explain why improved forecasting would have no effect.

But it is problematic to ask you to prove that changed construction and improved forecasting have had no effect.

If someone has hard data that show that improved construction and improved forecasting should result in less damage, then a flat trend in your normalized data would either mean increased frequency or intensity that aren't reflected in your data or a problem with your normalization process. But I haven't seen any such data to quantify a positive influence from changed construction or improved forecasting.

Roger Pielke, Jr. said...

-25-Joshua

Thanks .. quick replies:

1. I am quite familiar with the value of forecasting literature and I am unaware of any claims that improved forecasting reduces damage. Saves lives -- yes. This is a red herring. (Katz and Murphy is the classic volume if you'd like to pursue).

2. We don't have to look at the normalization to assess trends in frequency and intensity. We ca actually look at data on frequency and intensity:

http://rogerpielkejr.blogspot.com/2010/11/why-there-are-no-trends-in-normalized.html

No trends. I'm not sure how this can be any clearer.

Thanks for the exchange.

Ron Broberg said...

24-Pielke

Adjusting for market prices is problematic. Weather is a key component on the supply side and is one of the drivers of prices. On the other hand, it may be that demand via ethanol production has relatively little price pressure on corn prices when compared with climate changes.
http://woods.stanford.edu/sites/default/files/files/Corn-Price-Volatility-Study-Noah-Diffenbaugh-20120806.pdf

chemicalsblog said...

Dear Roger, I've made that point in the past, and even though I became pretty convinced of your approach thanks to reading your (and other researchers') papers, I still think that not looking at mitigating factors leaves a loophole in the normalization method.

Let me begin with stating that I would expect no change in (normalized) damages due to hurricanes, simply because there is no trend in landfalling hurricanes. That is, if it were not for any mitigating factors. Since societies and their economies are constantly developing, I can think of some factors which might actually lead to a decrease in hurricane damages.

First, there's construction. Building codes may not have changed, but what about building practices? Technology keeps developing, and so does the construction business. Are people really building houses the same way today than they did in the 1950s? When I remember correctly you once told me that's actually the case, but how can you be sure?

Second, there's early warning. Hurricanes are relatively well tracked nowadays, and while Galveston had virtually no warning in 1900, today people are being told days in advance what they're facing. So they have time to prepare their houses, protect property, and evacuate.

Third, there is accompanying policies. What about all the flood control measures introduced and extended in the 20th century? New Orleans is the perfect counterfactual, since if they had maintained their levees properly, damages would have been much lower. Ergo, I would expect damages to decrease rather than remain constant.

In other words: Your normalization method accounts for all factors that could possibly increase hurricane damages - especially infrastructure development, e.g. new houses at the beach. BUT it does not account for factors that could decrease damages, which is someting I would normally expect. That's the case here in Germany with storm floods from the North Sea: We had some really bad ones in the 1950s and 60s with huge damages and lots of people being killed. Since then they're not a big problem, even though the 1970s saw the worst series of storm floods ever recorded.

My last point: Finding that there is no trend in normalized damage figures is basically equivalent with finding that there is zero impact of building codes and practices on damages, that there is zero impact of hurricane reconaissance on protected property, and that there is zero impact of co-beneficial policies like flood control measures.

Nils

Roger Pielke, Jr. said...

-27- Ron Broberg

"Adjusting for market prices is problematic"

Indeed. Put an overlay of US ag policy on top of that . . . Thanks.

Roger Pielke, Jr. said...

-28-Nils

"Let me begin with stating that I would expect no change in (normalized) damages due to hurricanes, simply because there is no trend in landfalling hurricane"

Agreed, and that is what we find.

Thanks.

Roger Pielke, Jr. said...

-28-Nils

Also, you may qant to explore research that addresses the issues that you raise. There are answers to be found, for instance:

"The structural damage data compiled by the Metropolitan Dade Department of Building and Zoning after Hurricane Andrew in 1992 show that the long period of low hurricane activity in Dade County during 1950–1990 resulted in lower construction quality. In general, the older housing units constructed after the previous major hurricanes (in 1926, 1945, and 1950) showed less damage than the units constructed during 1970–1990"

http://ascelibrary.org/action/showAbstract?page=196&volume=12&issue=4&journalCode=nhrefo&view=fulltext

Joshua said...

Roger -

==]] 2. We don't have to look at the normalization to assess trends in frequency and intensity. We ca actually look at data on frequency and intensity: [[==

Well, that does seem to be the basic point, doesn't it?

So then the question is why the focus on the data on normalized loses? What function does it serve?

Are you using your work on normalized loses to argue against those who say that ACO2 is going to increase the intensity and/or frequency of storms? It seems to me that given the timing of your posts - coming post-Sandy - they might be intended to serve that function. But then why would you do that when the data about frequency and intensity of storms should stand on its own?


In your paper you linked, you basically say that the implication of your study is that we need to stop exposing an increasing amount of property to risk. Sure. Although that seems pretty obvious to me, and I'm not sure that anyone would disagree. IMO, the problem with that implication is that it ain't going to happen.

My guess is that the reaction to your paper is related to the whole context of people looking at mitigation and adaptation in opposition to each other. Nothing new there. It doesn't surprise me that people react as if you're taking sides in that argument.

It is interesting to me that your work suggests that changes in building practices has had no value. I still don't really byu that result (w/o seeing some quantified explanation for why that is the case) - but I guess that if we're looking for another non-controversial implication from your work, it would be that government enforcing building codes has not been effective, and thus government should regulate building practices more carefully to ensure better outcomes.

Just kidding.

Joshua said...

(#31) Roger -

The data collection period for that report you cited is a poor match with the data collection period for your normalization. Do you have anything more that actually looks at the impact of changed construction practices, the fact that more high rises have been built in high risk areas (which is not fully accounted for by normalizing property value increases), etc? And in particular, do you have any such evidence for periods that better match the data collection period for your normalization analysis?

You keep citing that study as if it validates your conclusion that changed building practices have had zero significant effect over the entire period from 1900-2012. Hmmm.

The correspondence of trends in normalized data and storm activity seem to validate your findings - but it is also possible that you are missing the impact of changed construction practices (or mitigating infrastructure) and that there is a corresponding problem in your normalization methodology. Probably a stretch - but if you had more comprehensive information to quantify why building practices (or mitigation infrastructure, or improved forecasting) has had zero effect it would be more convincing, for me at least. FWIW.

Roger Pielke, Jr. said...

-33-Joshua

Thanks, I won't have further time to engage your many questions. They are good ones and I suspect with a little initiative and research you'll find the answers that you are looking for ... good luck with it, and thanks again for the exchange.

dljvjbsl said...

In comemtn 30 Roger Pileke said

=======================
-28-Nils

"Let me begin with stating that I would expect no change in (normalized) damages due to hurricanes, simply because there is no trend in landfalling hurricane"

Agreed, and that is what we find.

=================

Much has been made in the commentaries about Sandy in the media of the rise in sea level and how that has made things more dangerous. Could collateral effects such as this be used as evidence for an expectation of increased losses even if the frequency and intensity of land falling hurricanes have not increased.

Also in regard to the effect of improved building methods, I can point out Hurricane Hazel of 1954 which like Sandy joined with a cold weather system and re-intensified it then struck Toronto. This caused massive flash floods in the river ravines in the city which caused many deaths and large scale property damage. As a result of Hazel, no development is now allowed in these ravines. One could expect that there would be reduced losses in Toronto during the next occurrence of a hurricane like Hazel or Sandy. However no such hurricane has struck Toronto in Even the last 54 years.

adamwelz said...

Dear Roger

Higher up in this thread you say "I am quite familiar with the value of forecasting literature and I am unaware of any claims that improved forecasting reduces damage. Saves lives -- yes. This is a red herring."

Could you please clearly explain

a) Why improved forecasting does *not* result in lower storm damage, as on the face of it, it would seem that it would: People have time to put up storm shutters, move valuables out of flood-prone basements, and move mobile goods e.g. motor vehicles to safer areas.

b) Why you do not consider lost lives as 'damage'.

Thank you

Adam Welz

Roger Pielke, Jr. said...

-36-adamwelz

Thanks ... great questions.

Please do recall the context of this discussion. The issue here is whether forecasting reduces damage in such a way as to introduce a bias into a normalization methodology. There are other factors at play as well, such as changes in building quality, building styles, building materials, etc.

If forecasts are 100% better today as compared to 1940, and one lives in a wood frame rather than cinder-block house, with large windows and a gabled roof, what is the more important change for adjusted damages? The improved forecast or the more vulnerable construction?

Our work suggests that whatever remaining biases may exist in the normalization methodology, they do not combine to result in a detectable signal. This can be checked mathematically and I encourage you to look at that literature (e.g., such as that which finds an ENSO signal in the normalized loss data).

2. We use the NHC damage database, which includes property damage. See Goklany's various papers for discussion of trends in loss of life related to weather over time.

Thanks!

John Daly said...

Has anyone used Disability Adjusted Life Years lost due to events as a normalization tool? Of course measuring the loss of productive live expectancy due to a disaster would confound the magnitude of the natural event, its proximity to human settlements, and the preparedness of the community affected, as well as the normal health and life expectancy of the people.

n-g said...

Roger - I haven't been able to find a direct criticism by you of the Munich Re report at issue, though I can imagine several possibilities based on what you've said:

(1) It's not part of the peer-reviewed literature and disagrees with the existing peer-reviewed literature, therefore it's very likely to be wrong;

(2) They have a past track record of issuing misleading press releases, therefore it's very likely that their results don't show what they claim to show;

(3) Having read their report in detail, it's clear they made a mistake in their methodology, specifically: ____

(4) Having read their report in detail, it's clear that they are overselling their results, specifically: ____

(5) The study results directly conflict with most (or all) other studies. Therefore it is prudent to assume that the study has an error until the reason for the conflicting results is identified.

Which of these, if any, reflect your opinion of the Munich Re report?

Roger Pielke, Jr. said...

-39-n-g

Thanks, I've made some varied comments on the report in the comments somewhere here, but they not are easy to find (I can't even find them at the moment;-).

I have read it closely (there are only a few pages on this analysis) so here is a concise critique:

Unfortunately, despite the buzz the report does not include any sort of statistical (or other) effort to achieve attribution. What they do say is that they have a record of 22 years, since 1980, when they say their data is good over which the losses that they keep track of related to thunderstorms in the US show a rate of increase faster than US GDP. That is it. They then speculate that this might be due to human caused-climate change.

A few points that make this claim not worth too much more attention:

1. No one in the scientific community that I am aware of -- not even those on the far fringe of attribution -- are claiming changes in thunderstorms (specifically, tornadoes, hail, straightline winds) since 1980 can be attributed to human-caused climate change. I am unaware of any scientific papers making this case, and in fact the SEX says that neither detection nor attribution has been achieved.

2. The 22 years for which Munich Re says it has good data is not even a climate time scale - which the IPCC defines as 30-50 years and longer. Hard to detect secular changes in rare extremes forced by GHGs on 23 year time scales, much less then engage in an attribution exercise, and on top of that link to damage.

3. Their data is not public so their work cannot be checked (actually we have it and a student used it for a recent PhD, more from us coming eventually;-). They have not published this result in the literature. By contrast, our recent paper on tornadoes found a long-term decrease in losses 1950-2011, using public data consistently collected by a single US gov't agency. I am very comfortable with our results and we see no evidence of increasing losses from tornadoes.

Munich Re of course can make any claim they want via press release. I'd note that if Exxon made a claim via report at odds with published research -- such as that glogal warming had stopped -- I doubt it would get the same free pass as Munich Re appears to get.

Science by assertion is not science. Hope this helps, feel free to follow up. Thanks!

n-g said...

Got it, thanks. Sounds reasonable. Personally, I automatically toss into the trash any climate change claim based on regional data spanning only half the relevant natural cycle (PDO/IPO, AMO) if the paper doesn't specifically address that confounding effect.

Robert said...

Re: Changed building practices should reduce losses assertion.

Thanks Dr. P.. As always, readable and concise.

This changed practice point keeps being made with no suggested way to test the hypothesis. That looks suspiciously like a red herring to me.

-31 Roger: One data point Dr. Pielke suggests - comparing data by home age in Miami area implies that there is no correlation. He suggests a cause. Another cause might be that newer, larger homes are proportionately more vulnerable than older homes. I am not familiar with Florida construction, but in Colorado I owned a 40s era home and I can attest that it had a proportionately more and thicker studs than my house built in 1995. I would rather sit out a severe wind storm in that older small house than the newer twice as large house.

Post a Comment

Note: Only a member of this blog may post a comment.