02 September 2011

Retraction, Remote Sensing and Due Process

[UPDATE 9/6: Gavin Schmidt explains that "Wagner's resignation is a recognition that he should have done a better job to prevent that [i.e., "the politicisation of the situation"].  There in a nutshell is the disease that afflicts climate science -- the view that scientific work in the peer review process should be evaluated according to expectations of political impact.]

[UPDATE 9/4: A reader writes in to remind me of my own related posts here and here and here.]

[UPDATE 9/4: Pielke Sr. has some thoughtful comments here.]

[UPDATE 9/3: The circus continues:
Kevin Trenberth received a personal note of apology from both the editor-in-chief and the publisher of Remote Sensing.
Why in the world would Trenberth need to be apologized to? Simply bizarre.]

The blogosphere is all atwitter over the news that the editor of the journal Remote Sensing has resigned to atone for what he believes to be a failure of his oversight of the journal by allowing what he asserts is a fatally flawed paper by climate skeptics to pass peer review and to be published.

The editor explains in an editorial published today that the paper in question "is most likely problematic" with respect to "fundamental methodological errors" and "false claims" and consequently "should therefore not have been published."

I am in no position to evaluate the substantive claims of errors and false claims in the paper, but I do agree with the folks over at RetractionWatch who call the resignation "curious" and ask if the editor feels as he does, "why not simply retract it?" In fact, if a paper has "errors" and "false claims" then a journal editor has an obligation to retract a paper (while of course giving the authors proper due process). In this case, the fact that the editor is unwilling or unable to retract the paper suggests that his resignation is probably the best course of action.

It is important for the new editor and editorial board of remote sensing to initiate retraction proceedings for the paper in question -- in other words the charges levied by the resigning editor need to be properly adjudicated. This is both in fairness to the authors (and the rest of us observers) but also good for science.

If the charges of "error" and "false claims" are upheld the paper should certainly be retracted.  If the charges are not upheld then the authors have every right to have such a judgment announced publicly.

Absent such an adjudication we are left with climate science played out as political theater in the media and on blogs -- with each side claiming the righteousness of their views, while everyone else just sees the peer review process in climate science getting another black eye.

38 comments:

EliRabett said...

You have, perchance actually READ the resignation. Clearly Wagner is extremely bothered by the uses the AUTHOR (well one of them) have made of this paper, which lifts it well out of the let's have comments category.

Amusingly your dad has wandered right into this one over at Climate Abyss.

Roger Pielke, Jr. said...

-1-Josh Halpern

I do not believe that post-publication discussions of a scientific paper in the media or on blogs should be used as the basis for subsequently re-evaluating the scientific merit of that paper within the scientific peer review process. Maybe I'm just old fashioned.

omniclimate said...

Hello Eli - so what did Wagner do wrong, exactly, that warranted his disappearance without the paper being retracted? Perhaps he didn't divine what people would have written about the paper outside of peer-review? Are journal editors going to be provided with crystal balls from now on?

omniclimate said...

Roger - if "post-publication discussions of a scientific paper in the media or on blogs" can now "be used as the basis for subsequently re-evaluating the scientific merit of that paper within the scientific peer review process", it just means that blogs and the media are now to be considered on-par with peer-review as ways to evaluate the scientific merit of a paper.

IOW all people that support Wagner's resignation are telling the world that the old complaint against skeptics "your article hasn't been subjected to peer-review!" is not valid any longer. A blog or an interview will suffice.

Methinks only Gavin could come up with such a spectacular own goal.

Roger Pielke, Jr. said...

-4-omniclimate

Thanks ... of course post-publication discussions will shape how people think about scientific claims. And often such discussions have more traction in the media and in political discourse. Nothing unusual about that.

What we should be concerned about is that the climate science community seems willing to substitute such judgments for the peer review process. Consider Peter Gleick's claim:

"Now, the question remains, will Fox News, Drudge, the Heartland Institute, and others that covered the initial report of this paper show the honesty and courage that Professor Wagner has shown and cover the fact that the paper is “fundamentally flawed?”"

Gleick explains, "this is also the way science works: someone makes a scientific claim and others test it. If it holds up to scrutiny, it become part of the scientific literature and knowledge"
http://www.forbes.com/sites/petergleick/2011/09/02/paper-disputing-basic-science-of-climate-change-is-fundamentally-flawed-editor-resigns-apologizes/

I hate to break it to Peter but the Spencer paper remains part of the "scientific literature" -- if he wants it removed he should be demanding a retraction proceeding. Similarly if I were Spencer I'd be demanding a retraction proceeding from the journal as well to have the editor's allegations evaluated within the peer review process.

The scientific community blurs the court of peer review and the court of public opinion at some risk. It may seem a fine thing to do in the context of climate, but what about an issue like MMR?

Stan said...

Since when does an editor without any background in the science at issue have the power to decree that a paper is "fundamentally flawed"? Gleick's comment is more than just stupid. It shows that he is so determined to try to score political points that he is willing to sacrifice his own reputation to do so.

Gerard Harbison said...

This latest fracas in the symptom of a scientific culture in decline, and, alas, I don't think it's unique to climate science (although in climate science the politics seem to be honed to a fine edge).

I've given affirmative reviews to dozens of papers where I thought the basic idea was wrong headed. I never thought it was my role as a referee to filter the content of scientific communication; but merely to make sure that the math was correct, the proper citations were made, and the data were not collected, interpreted or presented fraudulently. Whether the paper was right or wrong is for science, as a process to cast judgement on. It's beyond my pay grade.

Bad papers don't get refuted, they get ignored. Had Spencer and his coauthor's antagonists had any confidence in the process, the would simply sit back and let the process work out. That they did not bespeaks fear in the strength of their own position.

Ideally, science is a marketplace of ideas. Beyond some basic quality control, it's best to let the market do its work.

AMac said...

Dr Wagner's resignation letter states that he resigned on the strength of this issue: "The problem is that comparable studies published by other authors have already been refuted in open discussions and to some extend also in the literature (cf. [7]), a fact which was ignored by Spencer and Braswell in their paper and, unfortunately, not picked up by the reviewers."

We know what peer-reviewed paper Dr Wagner has in mind, ref. [7]. That is Trenberth et al, "Relationships between tropical sea surface temperature and top-of-atmosphere radiation," Geophys. Res. Lett. 37: L03702 (2010).

We don't know the identities of the "open discussions" where Spencer and Braswell's ideas were definitively refuted. The letter does cite non-peer-reviewed websites on other matters, but not these key discussions.

I hope they are not posts at partisan websites that aggressively moderate comments, thereby stifling free and open discourse. That would be a curious basis for this resignation. :-(

Roger Pielke, Jr. said...

Here is a great comment from a scientist posting over at Bishop Hill:

"This is truly bizarre, and just shows how profoundly warped the climate science community has become. I make no judgement here on the correctness of the paper, but editors just don't resign because of things like this.

Nobody resigned at Science when they published that utter drivel about bacteria replacing phosphorus with arsenic; they just published seven comments (IIRC) back to back with a rather desperate defence from the original authors.

Nobody resigned at Phys Rev Lett when I trashed a paper (on the evaluation of Gaussian sums) they had selected as one of the leading papers of the month: indeed nobody has formally ever accepted that I was right, but remarkably all the later papers on this subject follow my line.

I have been up to my neck for over a year in a huge row with Iannis Kominis about the underlying quantum mechanics of spin sensing chemical reactions, and either his papers or mine (or just possibly both) are complete nonsense: but nobody has resigned over Koniminis's paper in Phys Rev B or mine in Chem Phys Lett.

Sure, my two controversies above never hit the popular press, but the arsenic stuff was discussed all over the place, far more than Spencer and Braswell.

What sort of weird warped world to climate scientists inhabit? How have they allowed themselves to move so far from comon sense? What is wrong with these guys?

Sep 2, 2011 at 9:51 PM | Jonathan Jones"

n.n said...

So, in failing to regulate and legislate, they have resorted to appeals to emotion. I wonder if they will follow through with an executive order.

Whether the allegation is true or not, they have manufactured a perception, which will be used to marginalize others by association. With time, the details will be lost, but the perception and association will remain.

In any case, the action taken by Wagner would suggest that this was not about scientific enterprise, but about politics and personal/special interests.

I really hate conspiracy theories; but, since I lived through one (until its collapse), I cannot summarily dismiss the possibility. I am all too familiar with the tactics by which competing interests engage each other.

At some point we strayed from the enlightened path, and returned to a perverse natural order. Then again, maybe humans were never enlightened, and only through mutual assured destruction are we capable of realizing "harmony" in our relationships.

hro001 said...

Well, I suppose the alarmist/activist "climate scientists" had to do something to give Bob Ward material on which to opine ... otherwise he might have been forced to justify Al Gore's recent descent into disastrously desperate "dialogue"!

So the "in-crowd" - lacking any scientific rebuttal of S&B that could hold up to serious scrutiny - decided that Wagner, the new kid on the journal editing block, should be the multi-purpose diversionary scapegoat.

Apart from the flurry of MSM and blog coverage this has generated, look at the precedents his "resignation letter" has established:

1. An unsubstantiated "most likely problematic" will be sufficient for critics to point to in lieu of any substantive arguments against S&B's work.

2. An establishment of the "principle" that if the results of a paper are such that there is any indication that the "modellers'" prognostications should be found wanting then the former must be tarnished at all costs (including that of the reputation of the reviewers should their names ever come to light)

3. While the alarmist/advocates have long adhered to the "principle" that no critique of their work should be taken seriously until accepted in a "peer reviewed" journal, it would seem that this lofty "principle" is henceforth to be considered as a one-way street.

Mind you, it is interesting (albeit highly irrelevant to S&B or the currently non-existent "published" rebuttals thereto) the impact of the press release regarding of S&B's paper should be of such concern.

Particularly when one considers the recent controversy over the IPCC's May SRREN SPM release. The fanfare accompanying this particular work emphasized that the SPM had been "reviewed line-by-line" and "approved" by all 194 governements.

The Guardian's Damian Carrington had made much ado of this in July. Yet the IPCC's own numbers indicate that less than 50% of the "national delegations" actually participated in this "line-by-line review" and "approval" process. [Pls. see http://hro001.wordpress.com/2011/08/08/of-ipcc-reports-and-press-releases-in-which-they-hide-the-declines/

Oh, well, I guess when it comes to "climate science" there's a double standard for press releases, as well.

EliRabett said...

To move this away from climate, consider how NASA attempted to leverage the arsenic eating bacteria paper and the firestorm that started. The public gets fascinated by the spin and the experts get enraged.

This has become all too common, when someone attempts to mount their hobby horse on a thin paper with the help of the local press office or some think tank.

Mike Smith said...

Roger, this comment from WattsUpWithThat was highlighted over at Bishop Hill where I saw it. I think it makes an important point about modeling and data. http://wattsupwiththat.com/2011/09/02/breaking-editor-in-chief-of-remote-sensing-resigns-over-spencer-braswell-paper/#comment-734895

omniclimate said...

Eli - so how many resigned over the arsenic bugs?

Climate science is a "special" science. And this awful story can only reinforce the point.

Stan said...

Roger,

Why did you cut the quote of Gleick's short? You left out this really extraordinary claim ... "safe until someone can put forward a more compelling theory that satisfies all of the observations, agrees with physical theory, and fits the models."

"Fits the models" ??!! - these guys are insane.

Rich Horton said...

It seems fairly obvious that the real problems for Wagner are political (as some have noted above) and not scientific. S & B have not expressed the proper "climate consciousness" so they (or at least their work) must be sent to Siberia.

dagfinn said...

The lack of precision implied by the expression "comparable studies" is mind-boggling. I thought what would have to be refuted would be actual, specific arguments or data use in the Spencer and Braswell paper. Apparently not.

The dictionary tells me that "comparable" means "capable of or suitable for comparison". How does that mean you can refute one by refuting the other?

bernie said...

Wagner's resignation letter appears to have unleashed a raft of unintended consequences. Peter Gleick's argument about "fits the models" is too funny for words given the substance of S&B2011.

dagfinn said...

-15-Stan

Well yes, it does seem to follow from Gleick's statement that if you were to find that the models don't agree with the observations, your study is necessarily wrong.

Stan said...

Roger's dad has posted an e-mail exchange with Gleick. http://pielkeclimatesci.wordpress.com/

Gleick appears to be very impressed that Real Climate disagreed with the paper and thus, the editor was correct to resign.

dagfinn said...

The climate science community is looking more and more like the kind of dysfunctional social system in which behaviors that seem perfectly reasonable on the inside look totally absurd from the outside. Religious cults are only the extreme example of this. It happens in families, businesses and other organizations too. What's so weird is the fact that it's out in the open. Usually, some attempt is made to hide it since there is some vague understanding that it's perceived differently by the rest of the world.

TheLastBrainLeft said...

"Gleick appears to be very impressed that Real Climate disagreed with the paper and thus, the editor was correct to resign."

Hardly. Even if Real Climate had issues with the paper, they should have allowed the scientists battle it out. The good thing about science is that the more you talk, the more the truth comes out. By resigning, it's basically a tacit admission that Wagner was acting not on the merits of the paper in question, but to political pressure placed on him by those who refuse to accept dissent on man made climate change.

laogai said...

"The lack of precision implied by the expression "comparable studies" is mind-boggling."

Probably a reference to the RealClimate article where they claimed that their refutation of Lindzen also applied to Spencer, although in fact Spencer had made exactly the same point in his paper so they actually wound up confirming what Spencer had said. There's a comment somewhere on WUWT (the viral thread) giving details.

ABC NEWS WATCH said...

Dessler's article now available from papers in press section of GRL

http://www.agu.org/journals/gl/papersinpress.shtml

dagfinn said...

Gavin at RealClimate seems to be trying to change the rules of this game, and I'm trying to understand his reasoning.

"[Wagner] clearly feels as though he, and his fledgling journal, were played in order to get a politicised message to the media...

Many bad papers are published (some of which are egregiously worse than the one in question here) and yet very few editors resign over the way the process was handled."

That would seem to imply that the primary reason given by Wagner for his resignation is not the primary one after all. The "politicised message" is the problem. Or is Gavin insinuating that Spencer and Braswell deliberately and dishonestly--for political reasons--submitted a paper they knew was "bad"?

How often do editors resign as a result of the political use or abuse of published papers? I've never been involved in scientific publishing, but in my naive thinking, if I were the editor, I would say the politics is none of my business.

http://www.realclimate.org/index.php/archives/2011/09/resignations-retractions-and-the-process-of-science/

dagfinn said...

Gavin is also saying that retracting the paper would be "a really big step". So is the resignation of the editor just small step?

Marlowe Johnson said...

Roger,

I'm confused. Why does Remote Sensing need to initiate proceedings to get the paper retracted? Should all papers that contain errors and/or unsupportable claims be retracted?

Once again I find your 'tastes' on this issue astonishing. Not one word spent admonishing Spencer and Braswell for attempting to game the peer review system by publishing in a journal that was inappropriate given the subject matter. No comment on Spencer's misrepresentation on his blog or the Forbes article. No comment on the failure to select appropriate reviewers. No comment on failure to acknowledge other relevant work.


At least with the Wegman affair your response was appropriate, if a bit underwhelming 'if he plagiarized he should be fired'

Frontiers of Faith and Science said...

Anyone who has bothered to be informed about what actually happened in Climategate understands exactly what happened and why Trenberth required an apology.

Roger Pielke, Jr. said...

-27-Marlowe Johnson

Some replies:

"Why does Remote Sensing need to initiate proceedings to get the paper retracted? Should all papers that contain errors and/or unsupportable claims be retracted?"

Any paper for which the editor claims has false claims and methodological errors, and thus was mistakenly published, should be evaluated for retraction -- if nothing else than fairness to the authors.

On your other comments, I could care less about Spencer's paper or the Forbes article or his blog, sorry (I actually haven't read any of the three). This post is about procedural issues in peer review, not the substance of the paper or its use for political ends.

Thanks!

markbahner said...

"Why does Remote Sensing need to initiate proceedings to get the paper retracted? Should all papers that contain errors and/or unsupportable claims be retracted?"

Does the editor think the paper is incorrect in its conclusions? If the editor thinks that, don't you think he has an obligation to his readers to get a paper published in his journal that he thinks is correct, rather than simply resigning, so that the readership of the journal may never see what is correct?

markbahner said...

"Any paper for which the editor claims has false claims and methodological errors, and thus was mistakenly published, should be evaluated for retraction -- if nothing else than fairness to the authors."

I think the "fairness to the authors" should come below the "fairness to the readers."

If a journal publishes a paper that is incorrect in its conclusions, then the Editor in Chief should make sure that a paper that is correct is published in the journal.

Otherwise, the readers are left with no record (in the journal in question) of what the truth really was.

Rather than Wolfgang Wagner having performed some sort of noble and highly moral act in resigning, in my opinion his resignation is a dereliction of his duty to his readers.

Marlowe Johnson said...

Me
" I find your 'tastes' on this issue astonishing. Not one word spent admonishing Spencer and Braswell for attempting to game the peer review system by publishing in a journal that was inappropriate given the subject matter. No comment on Spencer's misrepresentation on his blog or the Forbes article. No comment on the failure to select appropriate reviewers. No comment on failure to acknowledge other relevant work."

You:
"I could care less about Spencer's paper or the Forbes article or his blog, sorry (I actually haven't read any of the three). This post is about procedural issues in peer review, not the substance of the paper or its use for political ends"

I'm tempted to say 'own goal' but I'll leave it to readers to decide. On a related note, what exactly is it in your father's post that you find 'thoughtful'? By your own admission you haven't read much of the relevant material...

Roger Pielke, Jr. said...

-32-Marlowe Johnson

Sorry, you are too cryptic for me ... care to be more direct?

tonylearns said...

Roger, having read Gleick's, Your fathers and RC's responses, it seems very clear that politics makes this a rather special case. This paper is quite relevant to the hot potato issue of Human induced climate change. Yet is was not published in either a journal specializing in that field or in one of the other major journals that publishes papers on the issue. The editor admits ignorance of the subject, which in itself is a rather unusual situation. Normally it would be that case because the paper is too specialized for the editors expertise to be relevant, but in this case, it is because the subject is so far removed from the normal subject matter of the journal. Now if it had been submitted to those other journals and rejected, and this was the first that would publish, that raises a number of questions. Was it rejected because the science was good but these standard journals are either all corrupt or too rigid and incapable of accepting anything that will contradict CO2 forced climate change? But if this was the case would the authors not reference the papers that their work is contradicting and point out the errors in that work (unless of course that charge is a lie)? Apparently there are scientific responses to this article that are being released now or soon in various journals, so the brouhaha about this resignation should be limited to politics of the situation. According to a post by Dressler about his response, the mistakes in both this paper and Lindzen Choi 2010 are of a monumental nature. Such that it should be evident to scientists familiar with the basic physics involved, such as your self, to determine rather simply if this paper is exceptionally valid, totally invalid or has both valuable and incorrect elements.
Clearly the publicity around the paper both by the authors and by those with a political agenda apposing the idea of ACC was a large factor in the resignation. It is also quite possible the the stigma of "allowing" an anti ACC paper to be published, whether accurate or fatally flawed, could have been a factor of potentially large consequence. But as I said this is a hugely political issue that has alarmists on both sides decrying the other side to be responsible for global catastrophe, either due to climate change or political, social, economic devastation. Ignoring that reality and treating this case as if it is just a question of appropriate treatment of the peer review process seems rather naive. Is there evidence that this editor was pressured to resign in order to impugn the paper? Does this editor have a publicly established bias and history of acting in a way that would indicate fear of retribution or humiliation if he did not respond in some dramatic way to protect his reputation? In reading about this issue I see that retractions are extremely rare in all science fields. I am just guessing here, but it only seems to happen when there is strong evidence of fraud, or when there is such overwhelming evidence against the methodology or the math or science as to make it worthless. Retracting the article would cause a tremendous political effect, and in any case is unlikely to occur immediately after being printed, So that seems like an irrelevant issue at this point. I don't see how the authors can demand a retraction proceeding. has that ever occurred? the idea of fairness to readers or authors also seems rather unimportant. Should the paper turn out to be a valuable contribution to understanding climate dynamics, his resignation will be an embarrasment, should it turn out to be bad science, a principled action. I am not clear which you think is the scenario that will play out. And if you don't know, why you are questioning his action without knowing?

Roger Pielke, Jr. said...

-34-tonylearns

Thanks for your comments, a few quick replies ...

1. For all the sound and fury, I am of the view that the great alarmist vs. skeptic war means very little in the policy process (see The Climate Fix), though admittedly the blogosphere thrives on it.

2. I am also of the view that the science in Spencer's and Dessler's is not particularly significant (but for opinions more informed than mine see Real Climate, Briggs, McIntyre, Pielke Sr., Curry), and is certainly irrelevant to actual policy (again, see TCF).

3. I also think that the fidelity of the peer review process is a very important topic in its own right, and it is not trumped by the politics of climate (see The Honest Broker).

Thanks

tonylearns said...

Roger,
I am not sure I understand any of your three points.
#1 have read references to your books but don't see how that realities of the current politics of climate change "means very little in the policy process". I am certainly aware that numerous industries and collaborations with academia are functioning outside of the "sound and the fury" of climate change, but I doubt that is what you are referring to.
2. From what I have seen so far, there appear to be two narratives that are rather mutually exclusive regarding the science of Spencers paper and Dressler and other commentaries. Either Spencer's paper is worthless for reasons A, B, or C, or Realclimate's, and Dressler's criticisms are worthless for reasons D, E, and F. Being as politicians are so polarized by the issue, especially as the next big election gains steam, they husband their rationalizations to use as ammunition against the enemy that holds the opposite view.
3. the point of my comment is that the politics of climate does indeed trump peer review, in the sense that the issue has become so politicized that publication in peer review itself is considered to be the litmus test for credibility. Believers in ACC use this as strong evidence that opposition to ACC has no scientific validity, and those opposed to ACC use this as strong evidence of corruption.
For these reasons I do not think it is useful to look at the resignation as isolated from the tremendous political pressure that surrounds this issue, and applying traditional standards of scientific protocol will have very limited usefullness.

barn E. said...

RE: UPDATE 9/6: Gavin Schmidt explains that "Wagner's resignation is a recognition that he should have done a better job to prevent that [i.e., "the politicisation of the situation"].

From a Canadian perspective I can think of only 2 reasons a team would pull their goalie; and neither happen when you're winning. Just saying . . .

-barn

dagfinn said...

-36-tonylearns

#2 This game of declaring each other's arguments worthless is typical of the climate war, and typically both sides are wrong. However, Spencer is not totally dismissive of Dressler's critique.

Post a Comment

Note: Only a member of this blog may post a comment.