22 August 2011

Ink Blots, Ambiguity and Outcomes in the Real World

A fundamental problem with climate science in the public realm, as conventionally practiced by the IPCC, is the essential ink blot nature of its presentation. By "ink blot" I mean that there is literally nothing that could occur in the real world that would allow those who are skeptical of scientific claims to revise their views due to unfolding experience. That is to say, anything that occurs with respect to the climate on planet earth is "consistent with" projections made by the climate science community. Some scientists go further and argue that climate science cannot be shown to be incorrect based on experience because its projections are probabilistic. The result is that  people tend to see in climate science other things than those that can be resolved empirically -- which fosters politicization and tribal behavior.

The ink blot nature of climate science would be a non-issue if it were a field like philosophy or cosmology in which people were debating non-empirical claims for academic interests. But climate science -- or at least a very visible part of that field -- has set forth on an evangelistic path in trying to convince the unconvinced of their views among politicians and the general public.

But the ink blot nature of climate science leaves climate scientists in a position of arguing from authority or demanding that people need "trust us."  The typical mode of engagement with skeptics by many visible climate scientists is to argue how right they are (and wrong/evil the skeptics are) -- but what skeptics need instead is to hear what it would mean for climate scientists to be wrong. If one cannot be wrong, then experience cannot be used to adjudicate claims. (I am aware of various debates that have occurred about using variables such tropical tropospheric temperature trends, ocean heat content, water vapor feedback, etc. in an effort to falsify claims of climate science.  While I am by no means an expert on these debates my understanding is that the climate science community argues that uncertainties/variability are so large as to make such claims not inconsistent with their views, taking us back to square one.)

Consider by way of example how the field of economics handles such situations.  Ratings agencies issue ratings associated with the likelihood of default for lenders, and utilize a language very similar to that found in the IPCC. For instance, here is how S&P defines their credit ratings (PDF):
In our view, likelihood of default is the centerpiece of creditworthiness. That means likelihood of default--encompassing both capacity and willingness to pay--is the single most important factor in our assessment of the creditworthiness of an issuer or an obligation. Therefore, consistent with our goal of achieving a rank ordering of creditworthiness, higher ratings on issuers and obligations reflect our expectation that the rated issuer or obligation should default less frequently than issuers and obligations with lower ratings, all other things being equal.

Although we emphasize the rank ordering of default likelihood, we do not view the rating categories solely in relative terms. We associate each successively higher rating category with the ability to withstand successively more stressful economic environments, which we view as less likely to occur. We associate issuers and obligations rated in the highest categories with the ability to withstand extreme or severe stress in absolute terms without defaulting. Conversely, we associate issuers and obligations rated in lower categories with vulnerability to mild or modest stress.
It is generally understand that the ratings agencies were wrong in their estimates of the likelihood of default among mortgage based securities. How do we know this?  Well, the global economy melted down for one.

I am aware of no one who has claimed that the ratings of subprime mortgages cannot be judged wrong simply because ratings are based on likelihood estimates. But this is exactly where some climate scientists would find themselves, if they were arguing about economics rather than climate.

It is not just the subprime crisis either where experience matters for the evaluation of expectations. Recently, Paul Krugman discusses the recent S&P downgrade of the US as follows:
When assessing the downgrade, the question of track record comes up. As I understand it, countries that defaulted in the past were almost always downgraded well before the default happened; but in all such cases, the markets were already signalling big trouble before the rating agencies moved.

The question should be, in cases when the markets aren’t signalling worry but the agencies downgrade anyway, how often are they right?

The answer, I believe, is never — not for Japan 2002, not for various European countries in the late 1990s, not for Canada 1994.
Until the climate science community steps out from behind academic parsing and hiding behind uncertainties, it will continue to be an ink blot, and one that many people evaluate using political and other non-scientific criteria.

There are two ways for the climate science community to move beyond an ink blot (if it wishes to do so). One would be to advance predictions that are in fact conventionally falsifiable (or otherwise able to be evaluated) based on experience.  This would mean risking being wrong, like economists do all the time. The second would be to openly admit that uncertainties are so large that such predictions are not in the offing. This would neither diminish the case for action on climate change nor the standing of climate science, in fact it may just have the opposite effect. 

The default will be the status quo, which means climate science as inkblot -- and the associated arguments from authority, "trust us" and politicization that comes along with it.

25 comments:

Joel Upchurch said...

If you are in an area where your predictions can't be falsified, are you actually engaged in Science? A scientific hypotheses should lead to predictions that either support (not prove) the theory or falsify it. You might say Science consists of a huge number of hypotheses that have proven to be false and a small body of theories we have not disproven yet. Climatologists aren't the only people with this problem. String Theory doesn't seem to be leading to testable hypotheses either.

Fred said...

The much bigger problem for Climate Science is that it has been taken hostage by the Eco-Greenie Industry that uses extreme fear mongering as its chief method of operation.

After awhile all those fear mongering predictions - aka lies to raise funds and keep the R&D Gravy Train rolling, add up and when they don't happen - where are the 4.5 Billion humans that are supposed to have died by 2012 due to global warming, the thousands of drowned polar bears, the ice free Arctic, the flooded subways in Boston, the rapidly warming atmosphere that isn't, where's the missing heat and the warming oceans . . .

People remember these things. They are also smart enough to wonder why everything to do with AGW causes only bad things to happen when every Minnesotan knows a warmer summer is a nice thing.


When Climates Science hitched its caboose to the AGW Fear Mongering Gravy Train you went all in with the Great Scam. Hysterical AGW has sold a lot of newspaper advertising, made a lot of Eco Rent seekers into Green Carper Baggers and been a bonanza for Greenpeace, Sierra Club and WWF fund raising scams.

That's your biggest problem. Thousands of fear mongering "The Sky is Falling" predictions and none of them are coming true.

Us ordinary folks can see a naked emperor when one walks down the street and the EcoGreenie zealots latest Emperor of Fear Mongering is clearly butt naked.

Apologies to anyone who images Al Gore as that emperor.

Gerard Harbison said...

There are a several reasonable predictions the IPCC could make right now:

The Arctic will be substantially ice-free for some length of time in at least n summers by year 20xx.

The average global temperature over the period 2015-2020 will fall in the range x - y degrees.

The mean sea level in 2020 will be x mm above where it is today.

They problem is, while I believe AGW is a major driving factor in all three predictions, many of the skeptics will simply argue these are straightforward extrapolations from current trends (which they ascribe to other factors) and will argue they are not tests of AGW.

Mark B. said...

It took a while to get to the word falsifiable. Commenter Joel hits the nail on the head with his first sentence. The 'scientific consensus' is that if it's not falsifiable, it's not science. Now different fields and sub-fields of science have different degrees of loyalty to falisfiability, but none dare reject it entirely on principle.

Multi-decadal climate model forecasts are falisfiable - in an ultimate, but not proximate sense. That is, when we're all dead, they will be tested and either verified or rejected. But since policy prescriptions require action NOW, and models cannot be tested NOW, they are, for all practical purposes, not falsifiable.

Matt said...

"The second would be to openly admit that uncertainties are so large that such predictions are not in the offing. This would neither diminish the case for action on climate change nor the standing of climate science, in fact it may just have the opposite effect."

I suppose that politically, or psychologically, this might be true, or perhaps if we're looking at this using the precautionary principle. In effect, saying that we can't predict anything (which is really what they are saying now when you follow the logic to its conclusion!) cannot possibly but diminish the case for action.

Roger, I'll grant your arguments about decarbonization being worthwhile for other reasons, but as I've said before, if you take away the urgent justification of AGW, that can only make any course of action based upon it less urgent. Unless one is an advocate of the precautionary principle, in which case I advise just staying in bed.

Roger Pielke, Jr. said...

-3-Gerard Harbison

Ah, but these are not probabilistic statements.

Matt said...

-3- Gerard,

You've hit upon a key point, which is really the old saw about correlation not being causation. While many of those events certainly sound like Global Warming, it's not necessarily anthropocentric. Given the massive uncertainties in the outputs of climate science, there's a lot of reason to doubt the anthropogenic nature of the warming. And even within the anthropogenic component, CO2 may not even be a major player.

models-methods-software.com said...

Have the initial projections of the path of hurricane Irene been proven to be Right or Wrong, Correct or Incorrect, Consistent With or Not Consistent With reality, Robusty or Flakey, Certain or Uncertain, Close or But No Cigar, True or Untrue, Accurate or Inaccurate?

Maybe all of the above?

MIKE MCHENRY said...

Can someone tell if this is wrong? If you plotted Probability vs Temperature change the highest probability would be for CO2(+ other GHG's) alone with no feedback's.

Gerard Harbison said...

Roger: ice-free in n summers in m years is effectively a probabilistic statement. You could just as easily express it as 'a probability of n/m in any one year'. I was trying to do what you suggested (and I agree with); formulate a concrete prediction that stems from AGW.

Sirius said...

Roger, your post is very interesting from an epistemological point of view. Alternative 1: "Advance predictions that are in fact conventionally falsifiable (or otherwise able to be evaluated) based on experience."  Alternative 2: "[...] admit that uncertainties are so large that such predictions are not in the offing." The first alternative is theoretically (and so materially) impossible in any reasonnable short term (unless we can all leave, as epistemic agents, at least 100 years from now); the second alternative is impossible to sustain by warmists, since that contracticts their claim that we are actually experiencing AGW. So actually, AGW theory does not belong to science at all but, at best, to the universe of nervous green philosophical speculations or (not exclusively) "carrément" (i.e. frankly) SF litterature. But mayby I have exaggerated something somewhere.;-)

Stan said...

Wow! Huge leap in that next to last sentence. We can't prove we are right, but you still have to give up your rights and sacrifice your children's chances for an improved standard of living. Because we have faith we are right, we think.

Roger, I don't think you can make that case morally. Political action merely requires sufficient power. But moral political action requires something more than simply sufficient numbers to force action down people's throats.

I don't see any way to make a moral case for govt action without being able to make some kind of scientific, evidence-based showing. Especially when the current probabilities are bogus numbers pulled from certain people's backsides.

Paul Baer said...

Well, it seems like we're in a bit of a pickle. The physical evidence that is available today, and for (say) the next decade or two, isn't likely to be definitive to those who are concerned about Type I errors; and predictions whose falsification would be convincing to those worried about type II errors have to be far enough in the future to be, for practical purposes, unfalsifiable.

Under the circumstances, we seem to be left with examining the indirect evidence and reasoning that is used to back up the assessment of risk. The people writing the IPCC chapters thought this was what they were doing. However, when the stakes are so high, the standards for "traceable accounts" are necessarily much higher, and I think that there is little disagreement that the IPCC has failed to address this adequately. But that doesn't make addressing it properly an easy task; there are no uniformly accepted methodologies for aggregating and documenting expert judgments on complex issues.

This is where we need "post-normal" scientific practices to address the post-normal scientific situation. (I realize that this is just baiting some people, but others, including Roger, will I hope see that it is a plausible framing for the necessary process changes.)

bernie said...

Surely the charge to those who claim AGW and that AGW has severe consequences is that they need to make a prediction that constitutes a severe test of their theory (viz. Popper). This is not easy, but this does not remove the requirement to provide such a test. Moreover the test does have to differentiate an AGW cause from all other available explanations.
Arguably in the field of public policy, the precautionary principle might suggest action before such a severe test is available ...but that would definitely require a very robust comparison of costs and benefits from a full range of available ameliorative policies. Banning CFCs fits this situation. It is less clear to me that banning DDT could be as readily justified.

Roger Pielke, Jr. said...

-13-Paul Baer

Thanks, and yes I think that this is right, though I do think that there are potential areas for shorter term predictions that might bear fruit -- such as ocean heat content changes.

On the need for better managing uncertainty in the context of post-normal situations -- Amen ;-)

On the other hand, perhaps we could just should louder that the other guys are stupid or malign ... (joke);-)

Pirate said...

I think climate scientists and the associated NGO's acknowledged way back that in order to "sell" AGW to the public, one needs to tell the people how climate change can be seen for real in your own environment - not by something as abstract as global ocean heat content but actual weather events. In other words, climate scientists had to provide long-term weather forecasts.

The hurricane predictions and super El Ninõs in the US, the warmer winters, droughts and heatwaves predicted by the Met Office in Britain.... well, we know the story there. Same thing happened here in Finland. The Greens had a big demonstration in 2007 in Helsinki, with the slogan "We want our winter back - stop climate change". They got it back in spades.
http://yle.fi/uutiset/news/2010/12/record-breaking_snow_depth_predicted_in_helsinki_2232218.html

As far as the public is concerned, the credibility of AGW suffered a huge blow. So, I think short term predictions directed to John Q. Public are out.

thierryp said...

"Quantum Mechanics can only make probabilistic predictions about measured variables. If you measure a variable V and obtain any value x , it will be always fine with QM provided that x belongs to the set of authorised values which is generally infinite anyway.
Yet QM makes a perfectly deterministic prediction of a function Psi (x,t) which defines unambiguously the probabilities of measure of any variable in a given experimental set up everywhere and for all times.
Unfortunately a function is defined by an infinity of values therefore a single measure doesn't tell anything.
Verifying QM consists to make the same experiment in an identical experimental set up over and over."

(E.Schrödinger in his article about the measure problem in QM)

I have quoted from memory but have the article at home. While some words may be different, the spirit is accurate.
If there are scientists who pretend that they can predict probabilities of the dynamical states of the climate then they have not read and/or understood Schrödinger or any other QM physicist.
If they told Schrödinger that they have a theory which predicts the climate probabilistically but can do only one measure for any given experimental set up (or initial/boundary conditions what is the same thing), then he would teach them that what they are doing is no science.

marlow said...

Is the measured increase in CO2 emissions - coupled with our theoretical understanding of the GHE - not enough all by itself to justify the kinds of benign measures that have been proposed by Roger and his colleagues? Are there are any good reasons for *not* following their suggestions?

New Face in the Mirror said...

--- But the ink blot nature of climate science leaves climate scientists in a position of arguing from authority or demanding that people need "trust us." --

I think this is also called "religion".

Maurice Garoutte said...

According to the scientific method, a properly formed hypothesis must be falsifiable by experiment or observation.

Climate Change cannot be falsified because it is consistent with any event.

Therefore Climate Change is not in the realm of science.

I don't believe in Climate Change, I'm a Methodist.

Mark said...

Is the measured increase in CO2 emissions - coupled with our theoretical understanding of the GHE - not enough all by itself to justify the kinds of benign measures that have been proposed by Roger and his colleagues? Are there are any good reasons for *not* following their suggestions?

No for the first. Yes for the second.

Take Australia. It has mega-tons and mega-tons of coal. To use anything else for electric power generation will pose a significant economic burden on the country. They really need some more proof than "it might be a good idea" before they double the cost of electricity.

I doubt China will consider any plan to drastically cut its carbon use as "benign", regardless of how necessary.

n-g said...

Climate science makes lots of predictions that can be and have been verified. "Prediction" doesn't just apply to future events, but also (and in science, mostly) to things yet to be observed about the present. In that regard, climate science has a strong record.

My comments: http://blog.chron.com/climateabyss/2011/08/roger-pielke-jr-s-inkblot/

Unknown said...

The experience that most caught my attention was a RealClimate thread discussing model falsification. Someone had calculated the 95% probability range for the IPCC ensemble of models, and suggested the real world data point lay outside that range. Others suggested the calculation was in error, the range was wider, and the real world still fell within the 95% band.
My reaction was very negative. It appeared the IPCC argument was "our work is very weak, the error bands are very wide, you cannot prove us wrong, nah nah nah nah". Would it not be more reasonable to force models or model assemblages to respect the real world data point? Yes it is only one possible outcome, but it is the one we have. Then explore the bands of possible outcomes with the centroid of the outcomes being the real world?
That could be the basis of a risk calculation, the basis of precaution, more reasonable than a band that just barely captures, at the lower end, the real world realization. I recognize there is plenty of noise in the real world trends, and plenty of noise in the calculation of what is actually happening. However, it seems to me that one (imprecise) data point deserves more respect than it receives from the IPCC.

Roger Pielke, Jr. said...

-23-Unknown

Here you go:
http://sciencepolicy.colorado.edu/prometheus/archives/ipcc+55.png

Frontiers of Faith and Science said...

One factor in the AGW movement is the human need to impose order on ambiguity.
AGW signals are at best ambiguous.
Believers impose the order on them that is based on their existing beliefs.
Think of believers, in another belief context, seeing images of Mary in a tortilla.
That is what makes inkblots so powerful. They are not revealing much about ink, but a lot about the person looking at the inkblot.

Post a Comment

Note: Only a member of this blog may post a comment.