[W]e have previously demonstrated that the most extreme surge index events can predominantly be attributed to large landfalling hurricanes, and that they are linked to hurricane damage (20). We therefore interpret the surge index as primarily a measure of hurricane surge threat, although we note that other types of extreme weather also generate surges such as hybrid storms and severe winter storms. . .As I showed in this post, which Gristed commented on, the surge record does not accurately reflect hurricane incidence or damage. Another poor showing for PNAS in climate science.
Last month the Proceeding of the National Academy of Sciences published a paper by Grinsted et al. titled, “Homogeneous record of Atlantic hurricane surge threat since 1923.” In what follows I provide a critique of their paper and offer my argument for why it does not actually tell us much about hurricanes, much less about damage.
The paper looked at 6 tide gauge stations along the US Gulf and Atlantic coasts to develop an annual index of storm surges in the United States. The paper explains why this is important:
[F]rom the economic damage perspective the hurricanes that remain far away from shore in the Atlantic are much less important than those closer to land. Hence in constructing an unbiased record of storms we need to ask what we want to measure. The strong winds and intense low pressure associated with tropical cyclones generate storm surges. These storm surges are the most harmful aspect of tropical cyclones in the current climate (1, 12), and wherever tropical cyclones prevail they are the primary cause of storm surges. A measure of storm surge intensity would therefore be a good candidate measure of cyclone potential impact.My attention was drawn to the paper because unlike other studies and data, which have found no trends in US landfalling hurricane numbers or intensities, Grinsted et al. do find a trend. They write:
We have constructed a homogeneous surge index on the basis of instrumental records from six long tide-gauge records. We demonstrate that the surge index correlates with other measures of Atlantic cyclone activity and that it responds in particular to major landfalling cyclones. The surge index can be used to identify and estimate potential remaining biases in other records of cyclone activity.They also compare their surge index with our record of normalized damage (Pielke et al. 2008). Because their dataset has an upward trend and the normalized loss dataset has no trend they conclude that our dataset is “suspect.”
We detect a statistically significant increasing trend in the number of moderately large surge index events since 1923.
I contacted Dr. Grinsted, who has been extremely responsive in providing data and exchanging views, for which I think him. He and co-authors (along with Kerry Emanuel, who edited the paper for PNAS) were provided a chance to offer comments on a first draft of this blog post -- they have not as yet, though the offer will remain open.
The first thing I noticed about their paper is that their surge dataset contains 465 surge events from 1923 to 2008 (July through November) yet over that same time period there were only 147 landfalling hurricanes (data from NOAA). You can see a comparison of the two datasets in the following graph.
However, the paper says something different:
To estimate the trend in landfalling storm counts, we count the number of large surge events greater than 10 units in 1 y, which is roughly equivalent to hurricane categories 0–5. This threshold was chosen as a compromise between looking at large events and having sufficiently many events to obtain robust statistics. Since 1923 the average number of events crossing this threshold has been 5.4/y …The actual number of hurricanes to make landfall averaged 1.7 per year over 1923-2008. So to claim that their selection threshold is “roughly equivalent” to hurricane landfalls is to provide a very generous interpretation of the term “roughly.” It is like me saying that I am "roughly" 18 feet tall.
The lack of precision in event specification was a point that Dr. Grinsted has admitted is a weakness of the study, as discussed in a news article which covered the paper:
There’s one obvious caveat about the new results: not every hurricane creates a storm surge, since they don’t always hit land. And not every storm surge is caused by a hurricane. “The storm surge index,” Grinsted said, “is sensitive to strong winter storms as well.” And it’s quite possible, he said, that the intensity of a given storm surge could be made greater or less by the angle at which a hurricane hits land.I reject the notion that the 465 surges used in the paper are a "pretty good" stand-in for 147 hurricanes, and the data supports such a rejection. That this claim made it through the PNAS review process does not help this journal's reputation with respect to the quality of recent papers on climate. But I digress.
Surges aren’t, in short, a perfect stand-in for hurricanes, but Grinsted said that they’re pretty good. In cases where they could do so, the team has lined up hurricane data with surge data, and, he said, “there are clear correlations. So while our paper might not explain everything, it is still useful."
To address the discrepancy between surges and hurricanes, during our exchange, Dr. Grinsted performed an additional analysis that did not appear in their paper, which was to look only at the top 150 events in the database, in order to explore whether this subset of their dataset would better capture landfalling hurricanes. This data is shown in the following graph.
The situation with damage is similar -- because it is a further a subset of the 147 hurricanes that cause most damage, far from the 465 surge events of which the vast majority of which are not associated with any damage. However, the top 15 years in terms of surge events from Grinsted et al. account for 46% of the normalized damage from 1923-2008. So there is some valuable information in the surge dataset at the most extreme end of the scale.
It is very important to note that the median date of these top 15 years is 1969, almost exactly the mid-point of the period examined by Grinsted et al. 1923 to 2008, a conclusion which actually supports the finding of no trend in the normalized loss dataset. Thus, a closer reading of the data presented in Grinsted et al. finds that the normalized loss dataset, rather than being “suspect,” is actually pretty robust.
To summarize, Grinsted et al. have created a dataset of storm surges which they have sought to associate with landfalling hurricanes and then further link to hurricane damage. Unfortunately, the connection between the surge dataset and hurricanes is, in their words “rough,” and shown here, tenuous. A further linkage to damage doesn't stand up. However, a closer look at the most extreme years in the surge dataset and its relation to normalized losses does find value. Here the surge dataset actually helps to confirm the finding of no trend in normalized losses.
Thanks again to Dr. Grinsted for engaging.
UPDATE: After writing this post I have been pointed to similar criticisms of Grinsted at al. by Tom Knutson and Gabe Vecchi of NOAA at DotEarth.