RMS Responds to the Sarasota Herald-Tribune

Several colleagues shared this letter submitted by RMS to the Sarasota Herald-Tribune's recent articles about reinsurance and catastrophe models.  I offer several comments below the letter.

Letter to the Editor

Your article ‘Florida Insurers Rely on Dubious Storm Model’ (November 14) contains some key inaccuracies about why and how RMS derived its medium-term hurricane risk model, and how these models are used by insurers.

Most fundamentally, catastrophe models deliver probabilistic forecasts not deterministic predictions. A probabilistic activity forecast means that, on average, a certain number of hurricanes can be expected over a period of time. The actual number experienced in a particular period will be just one sample from a broad distribution of possible outcomes.

There is widespread agreement within the scientific community that the number of intense North Atlantic hurricanes has increased since the 1970s, and that since 1995 overall hurricane frequency has been significantly higher than the long-term historical average since 1900. The question is, how much higher is the frequency and how will it impact hurricanes making landfall in the U.S.?

Given the lack of scientific consensus on this subject, we have attempted to answer this question by gaining the perspective of expert hurricane climatologists. The scientists were deliberately kept at a distance from the commercial implications of their recommendations. In our annual reviews of medium-term activity rates (the next five years), we have worked with a total of 17 leading experts, representing a broad spectrum of opinions. The process has evolved year on year, including the introduction of an independent moderator to oversee the elicitation in 2007. More recently we have employed a range of forecast models and methodologies subject to peer review in a scientific publication. Even when the experts involved and the scientific forecasting models have changed, the results of the five-year forecasts have remained remarkably consistent.

If RMS had been estimating medium-term activity rates during the 1970s and 1980s, the medium-term view would have shown lower activity than the historical average of activity. It should also be noted that 2010 has been another very active year for North Atlantic hurricanes. Fortunately none of these made landfall in Florida.

As an independent catastrophe risk modeler, the aim of our models has always been to provide the best unbiased estimate of risk to help the insurance industry and policy-holders to recognize and manage, and where possible, reduce the risk through the application of risk mitigation programs and initiatives. There is no commercial advantage for us to overstate the risk.

Pricing insurance risk involves a complex set of decisions. Models help determine the key drivers of risk, allowing insurers and reinsurers to understand their exposure to catastrophic loss. Other market conditions, such as the worldwide shortage of capital after the high-loss years of 2004 and 2005, also have dramatic influences on the availability and pricing of insurance.

We welcome review and debate of the timeframe over which catastrophe models should be used to characterize hurricane risk.

However, this debate should be based on a balanced and constructive view of the facts.

Sincerely,

Hemant Shah
President & CEO
Risk Management Solutions (RMS)

I have two responses to this letter.

First, Shah is correct that the RMS outlook does not offer deterministic predictions.  However, the measured language in the letter to the editor is contrary to how RMS characterized its outlook at the time.  Consider this excerpt from a peer-reviewed paper describing its prediction methodology published subsequent to its issuance of its 2006-2010 prediction (PDF):

The medium-term perspective is more specifically defined here as a window covering the next 5 yr. There are both scientific and business reasons for choosing the 5-yr horizon. The variance of predictions over 5 yr is smaller than that of seasonal forecasts, in part because of the way that the variations accompanying the state of the El Ni˜no are implicitly accounted for, as 5 yr nears the average period of one ENSO cycle. Predictions at longer timescales, such as 10 or 20 yr are also found to be less skillful, given the observed multidecadal variability. Five yr also bound most business applications within the insurance industry, whether it is planning for capital allocation or for transferring financial risk through Catastrophe Bonds, for example.

There is no discussion of uncertainties or probabilities associated with the prediction in the paper, and the term "prediction" is used throughout.

Second, Shah states:

Even when the experts involved and the scientific forecasting models have changed, the results of the five-year forecasts have remained remarkably consistent.

This is indeed remarkable.  So remarkable that after participating as an RMS elicitator in 2008 I looked into it and found that the results have little to do with choice of experts, but rather, the methodology employed by RMS.  Of course the results changed little.  When RMS did change its methodology a bit, the expected losses dropped a bit, and RMS suspended its elicitation process.

Along with its peers, RMS is an important company.  They do work that potentially helps make the global reinsurance and insurance industry do its work with a closer connection to empirical science. It is precisely because RMS is so important that it merits close attention.  Like ratings agencies, RMS and other catastrophe modelers are too important to a range of public outcomes to be left to govern themselves.  As much as cat modelers may not welcome greater external attention and accountability, as a result of their success and importance, that time has come.