10 January 2012

Bad Economics at NOAA

UPDATE 1/12: See this interesting follow up.

The National Oceanic and Atmospheric Administration is a federal agency that does a lot of excellent work related to weather, climate and the oceans. In fact, it is the primary sponsor of CIRES here at the University of Colorado where I serve as a Fellow. However, NOAA has been publishing information related to disasters that is extremely misleading and scientifically inaccurate.

The graph above shows NOAA's tally of "billion dollar disasters" which NOAA defines as "the 1980-2005 events which resulted in at least $1 billion in overall damages/costs at the time of the event" (emphasis added, source here in PDF).  The bolded part of that sentence is where NOAA's methodology has a serious flaw, as $1 billion does not mean the same thing today as it did in 1980.  In fact, adjusting just for inflation means that $1 billion today would have been the equivalent of $400 million in 1980. And that is not all, because there has been considerable development across the nation since 1980, meaning that there is more property and wealth to be damaged, $1 billion in damage today is actually equivalent to about $170 million in 1980.

Events which would have caused $1 billion in damage today, but did not when they occurred are not included in the NOAA listing. So by focusing on a $1 billion threshold, as $1 billion comes to represent less and less over time, NOAA has built in a strong bias in their analysis which creates the illusion of trend. If the point of the analysis is to say something about trends in weather, it will always be better to look at weather data, not damage data. 

Think of this analogy: Imagine if you had a shopping cart and added up the number products in your basket costing more than $5. In 1980 you would not have very many, and today you'd have a lot. Would a index of "number of products costing more than $5 in a basket" tell you anything about the overall cost of food?  No.

The New York Times recently published an editorial that serves as a good example of how NOAA's data is misleading (Nature however is not fooled):
 A typical year in the United States features three or four weather disasters costing more than $1 billion. In 2009 there were nine. Last year brought a dozen, at a cost of $52 billion, making it the most extreme year for weather since accurate record keeping began in the 19th century.
To be sure, 2011 saw some very extreme events in the United States. But was it the "most extreme year for weather" since the 19th century? Not by a long shot, whether the metric is dollar damage or loss of life.

The NOAA "billion dollar" data has only 1 event from 1980 costing $1 billion, a major drought.  I have quickly compiled a list of other events that would  have certainly resulted in more than $1 billion in damage were they to occur today and a second list of events that lack detailed accounting, but would be worth a further look (note that a rigorous analysis would implement the methodology of economic normalization that we have applied in a range of contexts).
1980 Disasters Greater than $1 Billion in Normalized 2011 Dollars But Not on the NOAA List

Certain additions:

Hurricane Allen - Aug 9, 1980 - $2.0 billion
Grand Island Tornadoes - June 3, 1980 - $1.7 billion
Western Wisconsin Derecho -- July 16, 1980 - $3.8 billion
California/Arizona Floods -- February 13-21, 1980 -$2.0 billion

Other candidates for inclusion but lacking a rigorous quantitative accounting:

Hawaii storms -- January
Hampton Roads "Circus" Blizzard --  January
Texas/Alabama/Louisiana Storms -- May 
Midwest Floods -- June
Panorama Fire -- December
There are thus 4 events that clearly would have been $1 billion events had they occurred in 2011 and I can find 5 other candidates for which data is lacking, but which could possibly have reached $170 million in damage in 1980 (especially if data collection were as intensive as today).  Regardless, of whether the total is 4 or 9 missed events or somewhere in the middle, NOAA's data misses at least 80% of the billion dollar disasters in 1980. Not good. No doubt that a reanalysis of the years 1981 to present would turn up many more such events that failed to meet the contemporary billion-dollar threshold but would certainly do so today.

It is extremely misleading to use economic impacts as the basis for making claims about weather and climate. NOAA should take immediate steps to improve the scientific quality of its tabulation of "billion dollar disasters" lest it find itself accused of misleading the public and decision makers with scientifically unsound information.

10 comments:

Sean Peake said...

But, but, I thought the science was settled? ;-). Good luck with that hope and change thing with the NOAA.

Roger Pielke, Jr. said...

-1-Sean Peake

Thanks, but this post has nothing to do with the science of climate change, and everything to do with a methodologically flawed analysis by NOAA.

Sean Peake said...

Just pokin' the bear tis all

Stan said...

Roger, it has nothing to do with science of climate change (or even climate change). But it has everything to do with the methodology and ethics of some of the people who are big cheerleaders for the CAGW narrative regarding an argument that they try to use to push that narrative.

SC Mike said...

NOAA’s approach is surprising, if not shocking. How can they possibly provide such data without adjusting for inflation? I’ll overlook the point about increased development in dangerous areas and the like, but failure to establish a baseline year for dollar value is inexcusable.

Nice catch.

Mark Bahner said...

"NOAA should take immediate steps to improve the scientific quality of its tabulation of "billion dollar disasters" lest it find itself accused of misleading the public and decision makers with scientifically unsound information."

What accuser would aid The Enemy in such a way?

;-)

Mark

dave said...

Roger, you should be careful about calling other people out on bad economics. Yours is slightly better than NOAA's, but still wrong. You are right about the inflation adjustment. But your second step from $400m to $170m is incorrect. In principle you are right that a second step is needed. If I understand how you did this then essentially you expressed damages today (in present prices) as percent of today's GDP (in present prices) and then multiplied that number with the GDP from 1980 (in 1980 prices), right? That is incorrect. In principle what you are trying to do there is to do something economists call adjusting for the time value of money. But it is not done this way. You need to read up on the net present value principle on how to do this properly. Any half decent intro textbook on something like applied micro-economic policy analysis should cover this.

Salamano said...

I thought this graph says that the dollar amounts were all normalized to 2011 using the CPI... is that not true?

Regardless, the last year in the data should not really participate very reliably in a trend (much like the first year)

Roger Pielke, Jr. said...

-7-dave

Thanks for your comment, but net present value has nothing to do with an economic normalization. If you are interested, please have a look at our 2008 paper on hurricane losses for an overview. Thanks!

Roger Pielke, Jr. said...

-8-Salamano

Right ... they implement the threshold and then apply the CPI. Ug. Thanks!

Post a Comment

Note: Only a member of this blog may post a comment.