30 August 2010

Report of the IAC Review of the IPCC

The InterAcademy Council Review of the IPCC has been released. The report is remarkably hard hitting with constructive and far-reaching consequences.  In the report's own words:
If adopted in their entirety, the measures recommended in this report would fundamentally reform IPCC’s management structure while enhancing its ability to conduct an authoritative assessment.
It is an excellent, thoughtful report.  While the report focuses on procedural questions and does not address any questions of scientific content, its recommendations have far-reaching substantive implications, such as for how to deal with uncertainty.  The report also directly addresses difficult subjects such as conflict of interest, policy advocacy and tenure of the IPCC chairman.

This post is simply a summary of the report's recommendations, and I will soon follow it with a bit more analysis.  Here then are the report's "key recommendations" that are highlighted in the Executive Summary, follwed further below by the recommendations found in the body of the text:
Governance and Management

The IPCC should establish an Executive Committee to act on its behalf between Plenary sessions. The membership of the Committee should include the IPCC Chair, the Working Group Co-chairs, the senior member of the Secretariat, and 3 independent members, including some from outside of the climate community. Members would be elected by the Plenary and serve until their successors are in place.

The IPCC should elect an Executive Director to lead the Secretariat and handle day-to-day operations of the organization. The term of this senior scientist should be limited to the timeframe of one assessment.

Review Process

The IPCC should encourage Review Editors to fully exercise their authority to ensure that reviewers’ comments are adequately considered by the authors and that genuine controversies are adequately reflected in the report.

The IPCC should adopt a more targeted and effective process for responding to reviewer comments. In such a process, Review Editors would prepare a written summary of the most significant issues raised by reviewers shortly after review comments have been received. Authors would be required to provide detailed written responses to the most significant review issues identified by the Review Editors, abbreviated responses to all non-editorial comments, and no written responses to editorial comments.

Characterizing and Communicating Uncertainty

All Working Groups should use the qualitative level-of-understanding scale in their Summary for Policy Makers and Technical Summary, as suggested in IPCC’s uncertainty guidance for the Fourth Assessment Report. This scale may be supplemented by a quantitative probability scale, if appropriate.

Quantitative probabilities (as in the likelihood scale) should be used to describe the probability of well-defined outcomes only when there is sufficient evidence. Authors should indicate the basis for assigning a probability to an outcome or event (e.g., based on measurement, expert judgment, and/or model runs).

Communications

The IPCC should complete and implement a communications strategy that emphasizes transparency, rapid and thoughtful responses, and relevance to stakeholders, and which includes guidelines about who can speak on behalf of IPCC and how to represent the organization appropriately.

Here are additional recommendations found in the body of the report:
The IPCC should make the process and criteria for selecting participants for scoping meetings more transparent.
The IPCC should establish a formal set of criteria and processes for selecting Coordinating Lead Authors and Lead Authors.

The IPCC should make every effort to engage local experts on the author teams of the regional chapters of the Working Group II report, but should also engage experts from countries outside of the region when they can provide an essential contribution to the assessment.

The IPCC should strengthen and enforce its procedure for the use of unpublished and non-peer-reviewed literature, including providing more specific guidance on how to evaluate such information, adding guidelines on what types of literature are unacceptable, and ensuring that unpublished and non-peer-reviewed literature is appropriately flagged in the report.

Lead Authors should explicitly document that a range of scientific viewpoints has been considered, and Coordinating Lead Authors and Review Editors should satisfy themselves that due consideration was given to properly documented alternative views.

The IPCC should adopt a more targeted and effective process for responding to reviewer comments. In such a process, Review Editors would prepare a written summary of the most significant issues raised by reviewers shortly after review comments have been received. Authors would be required to provide detailed written responses to the most significant review issues identified by the Review Editors, abbreviated responses to all non-editorial comments, and no written responses to editorial comments.

The IPCC should encourage Review Editors to fully exercise their authority to ensure that reviewers’ comments are adequately considered by the authors and that genuine controversies are adequately reflected in the report.

The IPCC should revise its process for the approval of the Summary for Policy Makers so that governments provide written comments prior to the Plenary.

All Working Groups should use the qualitative level-of-understanding scale in their Summary for Policy Makers and Technical Summary, as suggested in IPCC’s uncertainty guidance for the Fourth Assessment Report. This scale may be supplemented by a quantitative probability scale, if appropriate.

Chapter Lead Authors should provide a traceable account of how they arrived at their ratings for level of scientific understanding and likelihood that an outcome will occur.

Quantitative probabilities (as in the likelihood scale) should be used to describe the probability of well-defined outcomes only when there is sufficient evidence. Authors should indicate the basis for assigning a probability to an outcome or event (e.g., based on measurement, expert judgment, and/or model runs).

The confidence scale should not be used to assign subjective probabilities to ill-defined outcomes.

The likelihood scale should be stated in terms of probabilities (numbers) in addition to words to improve understanding of uncertainty.

Where practical, formal expert elicitation procedures should be used to obtain subjective probabilities for key results.

The IPCC should establish an Executive Committee to act on its behalf between Plenary sessions. The membership of the Committee should include the IPCC Chair, the Working Group Co-chairs, the senior member of the Secretariat, and 3 independent members, including some from outside of the climate community. Members would be elected by the Plenary and serve until their successors are in place.

The term of the IPCC Chair should be limited to the timeframe of one assessment.

The IPCC should develop and adopt formal qualifications and formally articulate the roles and responsibilities for all Bureau members, including the IPCC Chair, to ensure that they have both the highest scholarly qualifications and proven leadership skills.

The terms of the Working Group Co-chairs should be limited to the timeframe of one assessment.

The IPCC should redefine the responsibilities of key Secretariat positions both to improve efficiency and to allow for any future senior appointments.

The IPCC should elect an Executive Director to lead the Secretariat and handle day-to-day operations of the organization. The term of this senior scientist should be limited to the timeframe of one assessment.

The IPCC should develop and adopt a rigorous conflict of interest policy that applies to all individuals directly involved in the preparation of IPCC reports, including senior IPCC leadership (IPCC Chair and Vice Chairs), authors with responsibilities for report content (i.e., Working Group Co-chairs, Coordinating Lead Authors, and Lead Authors), Review Editors, and technical staff directly involved in report preparation (e.g., staff of Technical Support Units and the IPCC Secretariat).

The IPCC should complete and implement a communications strategy that emphasizes transparency, rapid and thoughtful responses, and relevance to stakeholders, and which includes guidelines about who can speak on behalf of IPCC and how to represent the organization appropriately.