Honest Brokering and Biosecurity Advice

Over the weekend Nature reported that Michael Osterholm, a member of the US government's National Science Advisory Board for Biosecurity, has accused the Board and its staff of biasing its advisory process in a recent high profile case. The situation raises important questions about expert advice and how it is structured to ensure quality, authority and legitimacy all at the same time.

Nature writes:

A closed meeting, convened last month by the US Government to decide the fate of two controversial unpublished papers on the H5N1 avian influenza virus was stacked in favour of their full publication, a participant now says. Michael Osterholm, who heads the University of Minnesota’s Center for Infectious Disease Research and Policy in Minneapolis, is a member of the National Science Advisory Board for Biosecurity (NSABB), which was tasked with evaluating the research. In a letter to Amy Patterson, associate director for science policy at the National Institutes of Health in Bethesda, Maryland, and sent to other members of the NSABB, Osterholm writes that the meeting agenda and presenters were “designed to produce the outcome that occurred“. The letter was leaked to Nature by an anonymous source.

Nature also put the letter online (here as DOC). Here is a passage from that letter which details Osterholm's concerns:

I believe that the agenda and speakers for the March 29 and 30th NSABB meeting as determined by the OBA staff and other USG officials was designed to produce the outcome that occurred. It represented a very “one sided” picture of the risk-benefit of the dissemination of the information in these manuscripts. The agenda was not designed to promote a balanced reconsideration of the manuscripts. While I don’t suggest that there was a sinister motive by the USG with regard to either the agenda or invited speakers, I believe there was a bias toward finding a solution that was a lot less about a robust science- and policy-based risk-benefit analysis and more about how to get us out of this difficult situation. I also believe that this same approach in the future will mean all of us, including life science researchers, journal editors and government policy makers, will just continue to “kick the can down the road” without coming to grips with the very difficult task of managing DURC and the dissemination of potentially harmful information to those who might intentionally or unintentionally use that information in a way that risks public safety. Merely providing a “minority report” in the final findings and recommendations of the meeting does nothing to address the fundamental issues of how the risk and benefits were determined, described, and considered at the meeting.

Let's see if we can disentangle some of the issues here. First, what kind of science advisory body is the NSABB?

Even though it has the phrase "science advisory" in its title, the Board is more accurately described as a "policy advisory" body. The Board is not focused on rendering expert judgments on scientific questions -- what I have called "science arbitration" -- but rather it is tasked with a much broader mission of making recommendations on action.

Here is how the NSABB charter describes its functions:

The NSABB is a federal advisory committee chartered to provide advice, guidance, and leadership regarding biosecurity oversight of dual use research, defined as biological research with legitimate scientific purpose that may be misused to pose a biologic threat to public health and/or national security.

The NSABB is charged specifically to:

• Recommend strategies and guidance for enhancing personnel reliability among individuals with access to biological select agents and toxins.
• Provide recommendations on the development of programs for outreach, education and training in dual use research issues for scientists, laboratory workers, students and trainees in relevant disciplines.
• Advise on policies governing publication, public communication, and dissemination of dual use research methodologies and results.
• Recommend strategies for fostering international engagement on dual use biological research issues.
• Advise on the development, utilization and promotion of codes of conduct to interdisciplinary life scientists, and relevant professional groups.
• Advise on polices regarding the conduct, communication, and oversight of dual use research and results, as requested.
• Advise on the Federal Select Agent Program, as requested.
• Address any other issues as directed by the Secretary of HHS.

As a committee that makes policy recommendations the committee can go one of two ways in offering policy advice.  One way would be to make recommendations that describe a preferred course of action -- that is, to advocate that a particular decision be made.  Where there is disagreement among the committee a minority position might be included in a report. However, there is no commitment to surveying or evaluating a wide range of options.

A second way to handle policy advice would not be to recommend a particular course of action but lay out the various options available for action, along with the risks and benefits associated with each fork in the road -- the honest brokering of policy alternatives. Here there would not be a single course of action favored and thus there would not be a need for majority or minority views. The key criteria of a report produced from such a process would be whether or not the scope of choice is adequately represented and if each alternative has been fairly and comprehensively evaluated in terms of risks and benefits.

In his letter to the NSABB, Osterholm suggests that the US government, the entity to which the NSABB provides advice, had an interest in a particular outcome from the advisory process:

It has been two weeks since the NSABB meeting of March 29-30 where the Board was requested by the USG to reconsider our previous decision recommending the redaction of both the above referenced manuscripts before publication.

In such a situation -- characterized by a controversial issue in which the decision maker requesting advice has a vested interest -- the integrity of the advisory process is protected by approaching advice from the standpoint of an honest broker. This serves several functions.

• Clearly delineates advice from decision
• Places responsibility for the decision with decision makers
• Allows for a full consideration of all sides of risks and benefits associated with different courses of action
• Brings controversy, uncertainty, ignorance and values out into the open
• Protects the advisory body from charges of bias in its deliberations 

In his letter, Osterholm provides a compelling list of reasons and justifications explaining why he thinks that the full ranges of risks and benefits were not explored by the NSABB.

For his part, Osterholm expresses a desire to have had a "disinterested subject-matter expert" involved in the process, based on his concerns about a lack of diversity of perspective among the committee related to the subject being considered. He explains:

The subject matter experts that addressed this issue at the meeting have a real conflict of interest in that their laboratories are involved in this same type of work and the results of our deliberations directly affect them, too. The same can be said about the attendees and outcome of the February World Health Organization consultation. In short, it was the “involved influenza research community” telling us what they should and shouldn’t be allowed to do based on their interested perspective. Such a perspective is very important and should be included in this discussion, but it shouldn’t be the only voice.

While looking for "disinterested" experts makes sense in such situation, alone, it cannot address the issues that Osterholm raises.

In this case there are two routes that the Board can follow. One is to explicitly take on the role as a partisan in the advisory process, making its best case for a particular course of action. This provides authority and cover for decisions, but leaves the advisory process vulnerable to legitimate complaints about procedure and substance.

A second route would be for the Board to explicitly consider a range of decision alternatives and then carefully evaluate the risks and benefits associated with each course of action. If the Board does not have experts or advocates representing each of these perspectives, then it should seek to have them represented in some manner.

At a minimum bodies like the NSABB which are expected to provide policy advice should explicitly discuss and clarify how exactly they are functioning. Do they seek to reduce choice or expand it? They can't do both at once, and leaving the issue murky can lead to exactly the situation that the NSABB finds itself in today.