11 November 2009

Response to Gavin Schmidt on Klotzbach et al. 2009

UPDATE # 11/8/11: Steve McIntyre, aided by some new information from Gavin, nicely resolves the dispute over amplifications factors and finds Gavin to be in error and agrees with the calculations in Klotzbach et al. 2010.  The Figure above comes from McIntyre's nice illustrative example. 

An update over at Real Climate would be appropriate -- "Science would work a lot faster if authors corrected their own work when errors were found.” -- Gavin Schmidt on Klotzbach et al.  Here is where we corrected our errors -- Gavin, how about you?

UPDATE 2: For an example of how Gavin uses of his comment threads (and why I see little use commenting there) see my submission and Gavin's edits/response. Gavin's reaction helps to explain why such disputes must take place in the peer reviewed literature rather than on blogs, and I hope that Gavin chooses to follow up with a submission.

UPDATE: A predicted below, Gavin uses the comment thread at Real Climate to raise (red herring) issues that he did not feel the need to address in his main post. My father
deals with Gavin's supplementary concerns.

Over at Real Climate, Gavin Schmidt is upset that we did not properly reference the GISS model as a source for a citation in Klotzbach et al. (2009, PDF). He is right to be upset as we failed to make that addition during page proofs. We have submitted a corrigendum as follows:
"A reference in Klotzbach et al. (2009) to McKitrick (pers. corres.) depended upon an analysis of GISS model output placed online by Gavin Schmidt of NASA in early 2009. We regret the oversight."
But Gavin goes too far in his critical comments when he states that we have made an "error" because we did not change aspects of our paper based on informal comments that he provided us in August based on a quick and dirty recalculation he did. For Gavin, not doing what he demands means that we have made an "error" and in typical overheated fashion -- "muddied the peer reviewed literature." His comments are surely going to provoke another round of mud slinging and misinformation about our work, and I rather suspect that this is the point, as in the Real Climate post Gavin encourages his commenters to have a go at us on issues unrelated to his post in the comment thread. Uncool, but not unexpected. Make no mistake, climate science is a contact sport.

We have repeatedly pointed out to Gavin that our results are insensitive to the points that he raises, and we showed this to him based on a recalculation of our analysis using his numbers when he first raised the issue. So were we to have changed our paper it would have said something like:
Informally, Gavin Schmidt raised some interesting points that are worthy of further investigation, however they make no difference to our conclusions. They do however hint at broader problems in reconciling surface and tropospheric tempertures that go well beyond the scope of our paper.
It looks like he completely understands the full insignificance of his comments as he writes at Real Climate that this situation is about "when problems (of whatever seriousness) are pointed out." You can be sure that he put in "of whatever seriousness" because in this instance his complaint is "not very serious." In any case, if Gavin thinks that the issue is important enough, we encourage him to submit something to the peer reviewed literature and explain why this matters. Given that his complaint does not materially affect our results and opens up a Pandora's Box of trouble for the models, I really doubt that he will do so. We also invited him on several occasions to join us in preparing a follow-on paper looking more broadly at the amplification issue, and he declined that invitation. Of course, had he taken us up on our offer (which still stands), he would have not been able to write a misleading post at Real Climate about (ironically, enough) muddying the waters.

Gavin kindly shared an early version blog post in advance of publishing. Below is how the authors of Klotzbach et al. 2009 responded. (Note: In the past Gavin has requested that we do not post his emailed comments, so we have not. He is free to add these emails in the comments if he'd like.) We look forward to further discussing this issue with Gavin in the peer reviewed literature.

Meantime, the bottom line is that regardless of the invective that might be found over in the Real Climate comment thread and among those who are inevitably going to (mis)citing that post, Gavin's comments do absolutely noting to impeach the conclusions of Klotzbach et al. (2009).

Here is the reply we sent earlier today:
Dear Gavin-

Thanks for sending the draft blog post. Below we identify several errors and omissions in it, and we ask you to correct these before publishing to avoid misrepresenting our paper and its conclusions.

But first, let us apologize for failing to add the reference to the supplementary information from Schmidt (2009) as the source of Ross McKitrick's calculations. This referencing oversight was simply overlooked at the proof stage, so thanks for calling it to our attention. We will submit a Corrigendum to JGR-A to rectify this. We also note that you replaced the data McKitrick relied on in our pers. corres. citation with a new dataset on 16 August 2009, which means that the data relied on by McKitrick were replaced on the very same day that you were raising these comments with us. Now that the original data has been removed, that certainly muddies the waters a bit as far as correctly citing the information.

There are also some factual issues with your blog post.

First, you strongly suggest that our paper has "fundamental flaws" but never say what these are. It is misleading at best to intimate that there are "fundamental flaws" when the issues you raise are irrelevant to the analysis in the paper. We'd ask that you specifically identify what you see as "fundamental flaws" if you choose to use such language. Otherwise your post is highly misleading.

Second, we published an exchange with you online in which we showed that our results were insensitive to the amplification provided by McKitrick versus those that you provided. Given that there was no implication to our conclusions, the issue that you raise, while certainly worth further investigation across models, is not central to our paper. At a minimum your blog post should clearly state that we have shown that our statistical results and conclusions are not influenced by this issue, and to be fair you should link to and present our demonstration of this, which you allude to but do not share in the post.

When you write, "a key assumption for this study rest solely on a personal communication from an economics professor" you are simply incorrect. The assumption, as we have shown to you, is not "key." Again, the issue that you raise is tangential to the paper and does not affect its conclusions.

As we understand the information provided to us by McKitrick, he calculated the model ratio based on model grids for which there were also observed data (so he could do an apples to apples comparison with observations and models). We suspect that you are using all land grids from the model. Thus, when we used observations for the surface temperature, we were being more consistent by using the subset of grids Ross developed as compared to those that you presented. At the same time, clearly our paper was not meant to be the final word on model amplification (nor of course was your note to us), since we show only a single illustrative result, and given the wide range of amplifications across the various models this is certainly a subject worth further study. But again, it is not directly relevant to our analysis.

Third, you are incorrect when you write that the choice of amplification factor "leaves one of the key hypotheses of the Klotzbach paper somewhat devoid of empirical support." Again, as we have shown, the conclusions of the paper are robust to choice of amplification factor. We would agree that a closer look at amplification factors over land and ocean as compared with the observational record is worthy of attention, which is why we invited you to co-author a follow-up piece to more fully explore this topic, and to which you declined to participate. The information that you provide suggests that there are even deeper problems in reconciling models with observations, but that subject goes well beyond the scope of our paper. But let’s be very clear, our results are not affected by the issues that you raise.

In sum, we are sorry to have overlooked the reference to the NASA GISS model as the primary source of a personal correspondence reference in the paper, and understand why you might feel aggrieved by this. We will submit a corrigendum JGR-A to present the acknowledgment more fully. We apologize. At the same time we do find it odd that you removed the data that McKitrick relied on from the GISS website and replaced it on the same day that you provided comments to us in August.

Finally, even though we understand why you may be upset about the reference, we see this unfortunate oversight on our part as providing no justification for you to mislead people about our paper or its findings, and we'd respectfully ask that you present information correctly and fully. If you feel that you have substantial scientific disagreements with our paper, we would encourage you to either take up our offer of collaboration on a subsequent piece, or to pursue this through the peer-reviewed literature on your own. As you have correctly emphasized in the past, blog commentary is not a substitute for the peer reviewed literature.

Sincerely,

Phil, John, Dick, Roger and Roger

18 comments:

  1. I have attempted to post up a link to this reply at Real Climate. Let's see if it appears.

    ReplyDelete
  2. Dear Dr Pielke,

    well, I do find it obvious that the personal pride of Mr Gavin was the true motor behind his confusing posting. And my feeling is that he deserved to be cited, indeed, but these a posteriori added comments actually brought him more credit than he deserved.

    Fine. For the rest of us who are not terribly interested whether Mr Gavin's contribution to the data analyses in this context is modest or extremely modest, is there something we shouldn't miss in these exchanges? ;-)

    ReplyDelete
  3. @Dr. Pielke

    "I have attempted to post up a link to this reply at Real Climate. Let's see if it appears."

    Looks like RC's posted your link. The tone of RealClimate's - and especially Dr. Schmidt's - inline responses perpetually astound me.

    I would hope that any further responses of yours are published at RC. If they aren't (stranger things have happened in this sphere), you have a great forum in your own blog to communicate your own thoughts, and I know you make it available to anyone else who wishes to do so.

    But just to see if the An Inconvenient Comment project has any utility, if you'd allow me to draw attention to the forum again after your own generous plug, I'd also like to make people aware that any attempted, but rejected, comments at the RealClimate story in question, or any others, are welcome at AIC.

    I'm the only admin at the moment, and the internet's going to be inaccessible to me for the next couple days at least, so for the time being, it's open season over there. We'll see what happens..

    ReplyDelete
  4. Gavin did let my link through. Good for him. Then he responded as follows:

    [Response: Way to go Roger! Reduce everything to a triviality of attribution and completely ignore the substance. Just so we're clear, 'puzzled' and 'surprised' doesn't equal 'angry' and the issue is not who should get the credit but whether the answer is right. If you are happy putting your name on clearly incorrect work, go right ahead. Readers can judge your credibility accordingly. - gavin]

    We await Gavin's submission on this to the peer-reviewed literature.

    ReplyDelete
  5. Gavin wrote: "Reduce everything to a triviality of attribution and completely ignore the substance."

    Roger wrote: "We await Gavin's submission on this to the peer-reviewed literature."

    If Gavin wants a shoulder to cry on I'm sure SteveMc can fully sympathise! Deliciously ironic!

    ReplyDelete
  6. Isn't the major issue that the amplification factor over land according to the GISS model is close to unity, and that this is erroneously reported as 1.2 in your paper?

    That sure was a more important issue that giving Gavin credit for the modeling work. Do you implicitly claim the amplification issue to be insignificant to the main message of your paper?

    Bart

    ReplyDelete
  7. -6-Bart

    First, it should be absolutely clear from the post that we have shown explicitly (i.e., we do not just claim) that the amplification issue is insignificant to the main message of our paper. Please do read the paper and you'll understand why.

    Second, as we said above, we understood McKitrick's calculations to be more consistent with our purposes (he looked at model grid cells for which there was also observational data, Gavin did not, looking instead at all land).

    Whether Gavin's results hold up or not, I don't know. Since his analysis don't affect our results I'm happy leaving it to him to explore in the peer reviewed literature. From what I've seen of discussions of model amplification ratios there is a pretty large spread across models (John Christy sent me some info on this that I'd be happy to post up here in the comments). Should Gavin open up this discussion, I can assure you that it will raise all sorts of interesting questions (e.g., consider what Gavin's new numbers do over the oceans).

    But again, this issue is not relevant to the conclusions of Klotzbach et al. I can keep saying it, but you can easily look at the paper and see why this is so.

    ReplyDelete
  8. How petty is Gavin Schmidt?

    This discussion will give you an idea:

    http://rankexploits.com/musings/2009/politically-motivated-nofollows/

    See comment Comment#23332.

    What, are we in junior high school? ;-)

    ReplyDelete
  9. As you have correctly emphasized in the past, blog commentary is not a substitute for the peer reviewed literature.

    This is what's commonly known as "hoist by his own petard!" As DaveJR says--"Deliciously ironic!"

    ReplyDelete
  10. Way to go Roger! Reduce everything to a triviality of attribution and completely ignore the substance.

    Umm ... red herring, anyone? I don't get attribution to be an issue here. Seems like Gavin's trying to muddy the waters still further.

    Shocker, that.

    ReplyDelete
  11. What, are we in junior high school? ;-)
    I think gavin's decision to nofollow links to your blog is the blog equivalent of decreeing you have cooties.

    ReplyDelete
  12. -11-Lucia,

    What? Roger has cooties?

    Does the government have a vaccine ready yet?

    Just kidding Roger...

    ReplyDelete
  13. Re: Blogging is no substitute for peer review

    When journals like Science allow the Hockey Team to effectively keep their data from sceptical eyes, perhaps blogging is in fact superior.

    ReplyDelete
  14. I submitted the following as my final comment at Real Climate. I see that Gavin's commenters rose to his bait.

    ------------------------
    My last comment here:

    Our paper depends upon a warming trend accompanied by increasing greenhouse gas concentrations. It is perfectly consistent with anthropogenic greenhouse warming (in fact it depends on it), and may suggest a mechanism for better reconciling divergent temperature trends at the surface and lower troposphere (See Urs Neu above at #43). The details will be worked out in the literature and our work is certainly not the last word. That is how the peer review process works. I hope that Gavin does submit a comment as that is how science works.

    Given the above, it is strange to see the paper characterized here at Real Climate as a "denier" paper with "fatal flaws", and this goes uncorrected by Gavin. What is it that we are denying? Gavin, do you see our paper as a "denier" paper? Or does it have "fatal flaws"? If not, why don't you correct those comments here on your blog?

    Also, the invitation to Gavin to collaborate on a subsequent piece remains open, however, so far he has declined the invitation. Surely that would be a good opportunity to work together rather than through blogs, which this post shows are not a particularly good way to advance understandings.

    Finally, Gavin, what is the deal with putting no-follow links to my site? Care to explain your reasoning there? ;-)

    Thanks all for the comments.

    ReplyDelete
  15. Realclimate seems to be an odd mix of some good science - the 'baby' - acting as cover for an extreme ideological pre-commitment to severe AGW - the 'bathwater'.
    To preserve the former while balancing out the latter, what is perhaps needed is an independently run sister-realclimate that publishes the sort of replies the Team prefers to suppress.
    A service http://rogerpielkejr...
    could perhaps provide?

    ReplyDelete
  16. And here is how my comment appeared, along with Gavin's edits and commentary. Note how he responds to comments of mine that he edits out:

    ------------------------------
    My last comment here:

    Our paper depends upon a warming trend accompanied by increasing greenhouse gas concentrations. It is perfectly consistent with anthropogenic greenhouse warming (in fact it depends on it), and may suggest a mechanism for better reconciling divergent temperature trends at the surface and lower troposphere (See Urs Neu above at #43). The details will be worked out in the literature and our work is certainly not the last word. That is how the peer review process works. I hope that Gavin does submit a comment as that is how science works.

    [Response: Science would work a lot faster if authors corrected their own work when errors were found. - gavin]

    [edit]

    [Response: I'm not in the least bit interested in your bruised feelings because of some manufactured slight that you perceive in the comments. In any case, commenters are responsible for their own words and publication in no way implies that they reflect the views of me or RC as a whole. I'm pretty sure that goes for your blog too. - gavin]

    Also, the invitation to Gavin to collaborate on a subsequent piece remains open, however, so far he has declined the invitation. Surely that would be a good opportunity to work together rather than through blogs, which this post shows are not a particularly good way to advance understandings.

    [edit]

    [Response: That would be great, except... oh, I don't know,... maybe the fact that you've recently called me a liar and a thief and accused me of 'baiting' our readers to be mean to you, all the while completely ignoring the pretty mundane substance of the complaint here in lieu of pretending this is just about me wanting a citation in your paper? Nah, that couldn't be it.

    If I might offer a little advice, you would do well to try and emulate your co-authors, particular your father and Richard McNider (above), who provide good object lessons of how people can disagree over substance and yet discuss issues without getting personal and without misrepresenting the other person's statements. Try it and see. Who knows where that might get you? Thanks! - gavin]

    ReplyDelete
  17. Peter: Realclimate seems to be an odd mix of some good science - the 'baby' - acting as cover for an extreme ideological pre-commitment to severe AGW - the 'bathwater'.
    Lately it's become a political shill site for the carbon cap and trade folk. The science on that blog was always uneven, but now it is either microscopic or gone entirely.

    ReplyDelete
  18. How exactly did some people decide it was in any way ethical or acceptable to selectively edit comments reasons other than offensive language, with no explanation? This practice seems far worse than outright rejecting a comment. While Dr. Schmidt's most recent treatment of Dr. Pielke's comment may only be a 5 or 6 or so on a scale of 10, I'm sure worse has happened on RC in the past, and probably will happen in the future...

    "[edit]

    [Response: I'm not in the least bit interested in your bruised feelings because of some manufactured slight that you perceive in the comments. In any case, commenters [edit] reflect the views of [edit] RC as a whole. [edit] - gavin]

    Also, the invitation to Gavin to collaborate on a subsequent piece remains open, however, so far he has declined the invitation. Surely that would be a good opportunity to work together rather than through blogs, which this post shows are not a particularly good way to advance understandings.

    [edit]

    [Response: That would be great [edit], you [edit] provide good object lessons of how people can disagree over substance and yet discuss issues without getting personal and without misrepresenting the other person's statements. [edit] Thanks! - gavin]"

    ReplyDelete