13 August 2009

Mann et al. Unsmoothed, Landsea07 Adjusted

UPDATE: I still find this hard to believe, is it possible that Mann has mislabeled his data files such that the smoothed data appears in the annual predictions column in his data file, rather than the raw counts? I find it hard believe that it is otherwise the case.


I was curious how the curve shown in Mann et al. discussed earlier today would look using adjusted data, and thanks to Michael Mann the data is up online allowing a comparison with data adjusted according to work in 2007 by Landsea (i.e., it doesn't include the analysis from Landsea et al. released this week).

I graphed (above) the adjusted data (red curve) along with Mann et al.'s "predicted" historical data (blue curve, based on the Landsea data) both unsmoothed, just to see what it looks like -- using information from these files at Mann's directory:
Statistical Model Predictions of TC Past Activity
Alternative Case (uses Landsea ‘07 adjustment of historical TC
series) (AD 500-1850)
http://holocene.meteo.psu.edu/~mann/Nature09/TCStatModelReconLandsea.dat

Historical Tropical Cyclone (TC) counts
Alternative case [Landsea ’07 adjustments](1870-2006)
http://holocene.meteo.psu.edu/~mann/Nature09/TCLandsea.dat
I now see why Mann claims that the Landsea adjustment does not matter. And he is right, it does not matter.

The Mann et al. historical predictions range from a minimum of 9 to a maximum of 14 storms in any given year (rounding to nearest integer), with an average of 11.6 storms and a standard deviation of 1.0 storms (!). The Landsea observational record has a minimum of 4 storms and a maximum of 28 with and average of 11. 7 and a standard deviation of 3.75. I suspected that a random number generator for hurricane counts since 1870 would result in the same bottom-line results and when I appended a series of random numbers constrained between 9 and 14 from 1870-2006 to the "predicted" values, lo and behold --- 20th century values exceed every other point except about 1,000 years ago.

Mann et al.'s bottom-line results say nothing about climate or hurricanes, but what happens when you connect two time series with dramatically different statistical properties. If Michael Mann did not exist, the skeptics would have to invent him.

34 comments:

clazy said...

So you're saying this is like the hockey stick -- a result that originates in the code, not the actual data?

Dan C said...

Roger-

I was wondering if you could elaborate on your methodology where you "appended a series of random numbers constrained between 9 and 14 from 1870-2006 to the "predicted" values, lo and behold --- 20th century values exceed every other point except about 1,000 years ago".

It sounds like you generated numbers once and reproduced the results, but that alone doesn't mean much, right?

Maybe this is just basic statistics that I'm not grasping at the moment, but isn't the argument that for the data (i.e. the jump 1000 years ago) to be significant to, say, the 95% level, that randomly generated numbers should reproduce the result less than 5% of the time (and thus they need to be generated thousands of times to obtain the percentages)?

Again, my stats skills are not stellar, so if that's not right or if you did something different do let me know :)

Thanks,
Dan

Roger Pielke, Jr. said...

-1-clazy

What I'm saying actually has nothing to do with the code or the mechanism of reconstruction. Simply put, at the two locations listed above have very different standard deviations, which means that the shorter time frame data with the larger standard deviation will appear to represent higher values than those in the longer dataset with smaller SD, even though they have just about the same mean.

Stan said...

It's like Rahmstorf -- the actual temperatures of the last few years weren't worse than they thought; it was the statistical artifacts he created and substituted for reality that were worse than expected.

Or see the Boston Fed study where the researchers cherry picked some of the data and devised their own lending criteria. Using criteria invented by the academics and applied to cherry picked data, they found discrimination. But looking at the actual criteria that the bank used and reviewing all the data, there was no evidence of discrimination.

Beware academics inventing data and using it to make sweeping pronouncements. Reality may not be as much fun, but it does have certain advantages.

Roger Pielke, Jr. said...

-2-Dan C

Random numbers generated from a distribution with SD larger than that in the blue time series will appear to exceed the historical values in every case, and this will be accentuated by smoothing.

Think of it this way. If every number in the long blue time series was, say, 11 (ie., with SD 0), and I append on a red time series centered on 11 with a SD of say 2 you could appropriately say that the shorter time series represents an increase over the line of 11, because a good share of it will be well above it for the shorter time period. Smoothing accentuates that appearance. And in the case of the data used by Mann, there is in fact an increasing trend in storms, with adds a further accentuation.

It is that simple. (Which is why I added the UPDATE above, Mann must have either included data that was somehow smoothed, or there is a missing step somewhere not described. I can't believe it;-)

Dan C said...

@Roger:

Ah I misread what you were doing with the random numbers (including the very quote I used ha), sorry. Yes, that makes much more sense, thanks!

Dan

Tom Fiddaman said...

Still doesn't make sense to me, unless by "appear" you mean some sort of visual impression.

I don't see how you can "appropriately say" that two series with the same mean have different levels, regardless of variance. The eye might be fooled into thinking that, but that's why we have statistics.

It seems plausible that the reconstruction is smooth because the proxies are smooth (i.e. high frequencies are lost in the data).

Roger Pielke, Jr. said...

-8-Tom

You write:

"I don't see how you can "appropriately say" that two series with the same mean have different levels, regardless of variance. The eye might be fooled into thinking that, but that's why we have statistics."

Keep that thought in mind when you re-read my post, especially the part that says:

"The Mann et al. historical predictions range from a minimum of 9 to a maximum of 14 storms in any given year (rounding to nearest integer), with an average of 11.6 storms and a standard deviation of 1.0 storms (!). The Landsea observational record has a minimum of 4 storms and a maximum of 28 with and average of 11.7 and a standard deviation of 3.75."

(hint look at the averages)

Tom Fiddaman said...

I guess I'm still lost in your prose. I was trying to parse comment #5.

So, are you saying in #8 that for all practical purposes 11.6 = 11.7 (given the noise), so claims that red > blue are silly? I'd agree. I suppose if you took the mean of the last century of the blue, it might be somewhat less, but still there's not a lot to go on.

In #5 you seemed to be saying that N(11,2) > 11 in some sense; I was just wondering what sense that could be, given that E(N(11,2))=11.

Richard S Courtney said...

Dr Pielke:

Thankyou. Clear, elegant and undeniable. I trust you have copied your observation to Nature.

But I am astonished that the paper passed peer review when it contained so very basic a flaw. And my astonishment exists even though the paper supports AGW and was for publication in Nature.

Richard S Courtney

Roger Pielke, Jr. said...

-9-Tom

You have it right, and it is a matter of "appearance".

Sylvain said...

I would like to point out that by some miracle Mann's paper will find its way in the next IPCC report while there will be no mention of Landsea.

The IPCC will then claim that they present the state of climate science and that, of course, they don't carry replication of peer-review study.

DeWitt said...

Small numbers such as annual hurricane numbers follow Poisson statistics. The probability that a time series with a mean of 11.6 that follows Poisson statistics would have a range of only 9 to 14 and a standard deviation of 1 is vanishingly small.

Charlie said...

OK, I'll bite.

HOW did he manage to do a historical reconstruction with a standard deviation of only 1.0 storm/yr?

With such spatially limited sampling, how in the world (or the Atlantic) could he come up with a min/max range of only 9 to 14 storms.

It defies common sense.

I must not understand what he is actually claiming.

zinfan94 said...

Charlie... you hit the nail on the head. Dr. Pielke Jr. is re-stating the study to make Dr. Mann's work seem incongruous.

The process used in the Mann paper only can collect hurricane data over multi-decadal periods. The data was collected only at a few sites, so it only gives data over longer timeframes. Looking at year to year (inter-annual) data makes no sense. Here is a quote from a page Dr. Mann set up to address mis-conceptions of the study:

"Finally, the assertion, which focuses primarily on interannual timescales, misses a key point, what is sometimes referred to as the ‘ergodic hypothesis’. The point is that the information gathered at a sparse but representative set of locations over long timescales (i.e., the multidecadal and longer timescales of interest in our study) is increasingly likely to mirror the information contained in a more extensive network. The principal is that what one loses in spatial sampling, one can gain back through greater temporal sampling, to the extent that vagaries of interannual variability are essentially stochastic. Now there are legitimate reasons for why this may not apply strictly in this case (and we’re quite clear about this in the manuscript), but it certainly motivates the view that one should be comparing the records on multidecadal and longer timescales. As discussed above in our response to assertion #1 above, such a comparison shows that the sediment composite record closely tracks the total basin-wide TC record at these longer timescales—both indicate the same multidecadal pattern of variability."

http://holocene.meteo.psu.edu/~mann/Nature09/responses.htm

Roger Pielke, Jr. said...

-16-zinfan94

It was Michael Mann (the PSU prof not the director;-) who created a data file with _annual_ hurricane counts dating to 500. I simply graphed his files in the manner that he presented them.

What is the temporal resolution (in years) of his study methods? Please show us how you arrive at this number. Thanks!

adam said...

Doesn't Mann's SD of 1 also imply (or more simply "exclaim") that the climate was "better behaved" in the past? Less variable? Given that he was working from proxies I can't see any other conclusion.

His Dendrochronology models seem to tell the same story - not just lower temps, but less variability.

I am a "luke warmer" myself but have a hard time keeping quiet when friends evangelize using Mann, so I get attacked as a "denier".

Anyway - the only thing that Mann has convinced me of is that he has identified some weak proxies.

zinfan94 said...

I am not sure is a single temporal resolution; it is likely the longer the period used, say 10 years versus 5 years, the better the accuracy of the calculated results.

Looking for inter-annual variation in the data, simply shows how raw data without knowledge of the process of how it was collected, can be mis-interpreted. Dr. Mann's paper used a process that collected data from sediments at only five sites, with one in Puerto Rico and one on the Gulf Coast, the rest on the East coast. In any given year, the sediment record at any of these sites is unlikely to record even one hurricane event (Cat 2 or higher, seems to be the level of detection). Many hurricanes would likely not pass close enough (Mann estimates a span of 270 km, if I understand correctly) to register in the sediment record.

Only at longer timeframes, does the data make any sense. For example, Puerto Rico is likely to get hit by a hurricane of Cat 2 or higher at least every 10 years or so, and so over 1500 years that is about 150 hurricane events in the sediment records. The Gulf coast site likely contained a similar number of hits. From this information, one can estimate how many Cat 2+ storms made landfall in the Caribbean and Gulf Coast over each 10-20 (or 30) year record.

But even if the sedimentary data show storms one year, and none the next, it simply means the sites weren't hit that year, not that there weren't any hurricanes. The yearly data on the sediments are meaningless until they are accumulated into a longer term record.

You are essentially taking the raw data, that was collected only to measure multidecadal trends and averages, and then saying "Wow, look at the year to year variation... it doesn't make sense."

Anyone who looked at how the data was collected, would expect to see variation in the year to year data, and even for data collected in 5-10 year periods. Only over longer multidecadal periods, can you say anything about the variation of the results.

Your complaint is meaningless.

wattsupwiththat said...

Roger,

A commenter by the name of Paul Klemencic who also sometimes posts as Paul K has said he and/or RC can "demolish" this post.

I told him to take it here and discuss it with you, and if indeed you made a retraction based on that I'd post it on WUWT. I want him to argue with the source, rather than the messenger, since I listed this post as "Quote of the Week".

This is a note for you to expect him.

DeWitt said...

-17-adam,

Small number statistics (Poisson distribution) does not allow "better behavior". Modern annual hurricane counts have been shown to follow Poisson statistics, as one would expect.

Mann's numbers actually work out if he used decadal rather than annual counts. That would then be 116/decade with a standard deviation of (116)^0.5 = 10.8. The table only makes sense if it's actually a ten point moving average. Splicing modern annual data on the end without equivalent smoothing would then be somewhat misleading.

Roger Pielke, Jr. said...

-20-Anthony

Thanks. Paul K (and anyone else) is welcome to comment here.

-21-DeWitt

Interesting.

zinfan94 said...

The paper has this to say about the period used in the smoothing:

"Figure 3 | Long-term Atlantic tropical cyclone counts.
Modern Atlantic tropical cyclone counts (red) compared both with statistical model estimates of tropical cyclone activity based on modern instrumental (AD 1851–2006; black) and proxy-reconstructed (AD 500 1850; blue) climate indices and an estimate of basin-wide landfalling Atlantic hurricane activity (AD 500–1991) derived from regional composites of overwash sediments (green). All series were smoothed (30) at multidecadal (>40-year) timescales."

The paper seems to be using 40+ year data time periods. I can't see how any inter-annual variation could be analyzed from the sediment data from limited spatial collection sites.

zinfan94 said...

I also would like to correct what I posted above about 5 sites... after examining the paper more closely, there were actually five regions, with some regions having multiple sites. The example I used to illustrate the problem with interannual variation was also incorrect. From the data in the paper, the number of "hits" at each site was much lower than I guessed. This makes smoothing the data over longer multidecadal periods even more important.

For those who want to read the paper online, here is a link (you may have to copy and paste the URL):
http://www.essc.psu.edu/essc_web/publications/PDFs/nature08219.pdf

Charlie said...

Zinfan94, were you able to determine how long of half amplitude period is used for the ">40 year smoothing" ?

I couldn't find anything in the text, and when I started looking at the methods, it seemed like there were at least two cascaded smoothings.

DeWitt said...

If the method of analysis does not provide annual resolution, and there is internal evidence that the resolution is decadal at best, then why give annual numbers? There also seems to be too much high frequency variability for a resolution of 40+ years unless the sampling process aliases low frequency signals to high frequencies.

If you take the 1850 to 2006 data and do 11 and 41 point moving averages without end point padding, the modern data doesn't look much like the 500 to 1850 reconstruction. Koutsoyannis has shown that modern hurricane counts have a relatively high Hurst coefficient which would give much more low frequency variation than is evident in the Mann reconstruction.

zinfan94 said...

Charlie... I don't know, I couldn't find that in the information released with the paper, but I am not really the expert on this. Mann has posted once on RC pointing to the pages I have accessed, and he promised an upcoming post on RC on the paper. I think that will be the opportunity to learn more about the processes used in the paper.

My initial response was to be very skeptical of what can be determined from so few sites, and so few boreholes. I guess Mann is following a strategy to test the methods on a limited scale, show some preliminary results, then get the funding for a more comprehensive study. Clearly more sites and boreholes are called for, to improve the confidence in the results.

zinfan94 said...

Here is the one response to a comment from Mann on RC so far (an OT comment on the "Technical issues in science" post):

OT, but if possible, I would like to see some discussion from Mann, on his latest research regarding hurricanes; there have been some accusations made by others that the second paper relies upon the first and that the sediment method used now contradicts a smoothing out of the Medieval Warming period findings previously. Undertsand, I am certainly not making any accusations, but I would like to see more explanation/data from Professor Mann, himself, if possible; thanks.

[Response: Thanks for your interest in the paper. As this is off topic, I'll allow one comment on this (yours), but otherwise don't want to see the thread hijacked by coverage of this topic. At some point in the near future, we'll probably have an article at RC reviewing the various recent developments in our understanding of the linkages between climate and Atlantic tropical cyclone behavior. The criticisms you cite are at best willfully naive. I'm not sure which 'second' or 'first' paper you're referring to, but our recent Nature article on past Atlantic tropical cyclone (TC)/hurricane activity certainly doesn't rely on any other past paper under discussion in this thread. It uses two different methods, one of which uses regional climate reconstructions, another which uses so-called 'overwash deposit' sediment records (these are by the way not records I've used in any other study. they aren't proxies per se of climate, but rather of past hurricane behavior). As for the putative inconsistency with other work, that's a bit of a silly claim since this is the first paper to reconstruct basin-wind tropical cyclone activity (so frankly, there is nothing for it to be either consistent or inconsistent with at the present time). We can see from one of the two approaches used in the study (the statistical modeling approach driven with climate reconstructions) that the peak in activity 1000 years ago arises from a combination of factors. Those factors are La Nina like conditions that appear to have prevailed at the time, and relatively warm tropical Atlantic SSTs. The tropical Atlantic SST pattern closely follows the pattern seen in previously published Northern Hemisphere temperature reconstructions, with the Medieval peak of roughly similar prominence to that described in previous work. The tropical Atlantic SSTs are not as warm as today. It is only the combination of relatively warm tropical Atlantic SSTs and La Nina-like conditions in the tropical Pacific that work together to give a medieval peak in Atlantic hurricane activity that rivals that of today. So any 'inconsistencies' claimed by detractors are either imagined, or manufactured in an intentional effort to deceive readers about what the study actually shows and claims. I would encourage any readers to get their information from the paper (and supplementary information), the various press releases, interviews (including ones I did for NPR and PRI), and a video conference I did for NSF. That can all be found here. In addition, corrections of specific misconceptions about the study (such as some of those described above) are available here. -mike]

Comment by Jacob Mack — 14 August 2009 @ 6:22 PM

Charlie said...

#26 Dewitt: "... modern hurricanes have a relatively high Hurst coefficient which would give much more low frequency variation than is evident in the Mann reconstruction"
It seems like this difference in Hurst coefficient between observed and reconstructed hurricane frequency also exists between observations vs GCM projections and hindcasts for just about every weather metric including temperature.

If we wanted to compare the reconstruction with modern, then we should not be plotting the actual observed hurricane counts, but instead the hurricane counts predicted as in the Mann,Sabbatelli, Neu 2009 GRL paper on undercounts.

Mann's work is truly incredible. In the sediment overwash proxy he has managed to extract an amazing amount of information from the few datapoints (both sparse spatially and in datapoint at each site)

Charlie said...

Just a little question.

I note that in the file http://holocene.meteo.psu.edu/~mann/Nature09/NYAlder.dat
that the - 1 sigma age limit column is transposed with the + 1 sigma age limit column.

Does that impact the analysis, or does his code detect the data error and correct it?

Roger Pielke, Jr. said...

It is worth emphasizing that the Nature press release that Mann has repeated touted as being an accurate description of his paper claims to resolve annual frequencies:

"A surge in the number of Atlantic hurricanes over the past 10 years is not unusual and could be part of a naturally occurring millennial peak, say US researchers1. The finding suggests that any increase in hurricane activity due to global warming would add to the current peak.

The study, published in Nature today, found a peak in hurricane activity across the tropical Atlantic around 1000 AD, that approached levels seen today. In 2005, there were a record 15 Atlantic hurricanes, including hurricane Katrina, which devastated the city of New Orleans in Louisiana. These peaks contrast with lulls in hurricane frequency the study identified before and after the 1000 AD peak, when 8 or 9 hurricanes occurred each year."

http://www.essc.psu.edu/essc_web/research/Nature09.html

Jim Clarke said...

What happens when you try to do science with a little bit of data and a whole lot of assumptions? Well, you can pretty much get any result that you want! Tracking historical tropical cyclones using the 'overwash' method does give you insight to some historical events, but tells you nothing about storms that never make landfall, and nothing about the storms that do make landfall with little surge. (Hurricane Charlie comes to mind as a Cat 4 that produced little surge.)

I haven't read the report, but I assume they compared the overwash in modern times with known storms and extrapolated this precarious relationship back 1500 years. But we are just beginning to realize that there are natural climate cycles that span several decades. Might there also be cycles that span 100's of years that have a big effect on where the storms go? Also, the coast has changed a lot in modern times. I just do not believe you can get an detailed picture of the past through this method. It is kind of like finding evidence of a lost civilization and from these meager clues, describing the culture of the neighboring civilizations.

The second method, the computer model, has even more assumptions, making it another case of garbage in/garbage out (or climate fantasy)!

The nature of climate science means we have to struggle just to get little clues, and then slowly piece the picture together. Mr. Mann has a habit of picking up a few pieces of the puzzle and proclaiming he knows what the whole thousand piece picture looks like. It is not ironic that he proclaims that the whole picture looks just like is ideology says it should look.

This makes him a priest, not a scientist.

Jim Clarke said...

"In 2005, there were a record 15 Atlantic hurricanes, including hurricane Katrina, which devastated the city of New Orleans in Louisiana."

Now that is sensationalism! If New Orleans existed just like today for the past 1,000 years, it would have been devastated at least 10 times by now. Katrina would hardly be the first or the worst. Including that little tidbit in the press release is anti-science, having absolutely nothing to do with the study,

Roger Pielke, Jr. said...

I have sent these questions to Michael Mann:

Michael-

Thanks for this further information and for pointing to the press releases that appear on your webpage as authoritative interpretations of your paper. I have looked at that and have three simple questions that you can quickly answer to help me understand your work and how you have represented it.

First, just to be completely clear, the Nature press release explains that your methodology recovers annual data, and indeed this is what you report on your website.

1. Is this correct? (A yes or no is fine as an answer.)

Here is what the Nature press release says:

"The study, published in Nature today, found a peak in hurricane activity across the tropical Atlantic around 1000 AD, that approached levels seen today. In 2005, there were a record 15 Atlantic hurricanes, including hurricane Katrina, which devastated the city of New Orleans in Louisiana. These peaks contrast with lulls in hurricane frequency the study identified before and after the 1000 AD peak, when 8 or 9 hurricanes occurred each year."


Second . . .

2. If no, then what is the temporal resolution that you claim your methodology? (An number of years is fine as an answer.)

Third . . .

I note that using the Landsea07 dataset that the overall means of the proxy data and the mean of the observational record are just about identical, meaning that a resolution of a bout a decade is needed to support your claim that recent activity is at a historical peak. Indeed this is what the Nature press release says:

"A surge in the number of Atlantic hurricanes over the past 10 years is not unusual and could be part of a naturally occurring millennial peak, say US researchers"


3. Does your methodology purport to reolve frequencies on a 10-year resolution? (Again, yes or no is fine)

Many thanks for your replies,

Roger

Roger Pielke, Jr.
Professor, Environmental Studies
Fellow, Cooperative Institute for Research in the Environmental Sciences
University of Colorado

Charlie said...

The silence speaks oh so loudly.

Post a Comment

Note: Only a member of this blog may post a comment.