Disaster Recovery: Plan Now, or Face Regret Later

A widening array of solutions is giving carriers more options to backup and safeguard their critical data.

If it is at all possible to find a silver lining in the spate of natural and manmade disasters afflicting the United States since the turn of the century, it is that they remind companies to prepare themselves so that the next disaster causes minimal damage. And yet, in spite of the devastating effects of 9/11, Hurricane Katrina and Hurricane Rita, the California wildfires and many other disasters, circumstantial evidence suggests that many insurance firms do not have strong and steadfast business continuity plans in place.

MISPLACED CONFIDENCE

Too often, insurers “logically” deduce that they are out of harm’s way and only act after it is too late. For instance, one carrier was talking with the Bothell, Wash.-based technology provider AMS Services about signing a deal for recovery support, but kept dragging its feet, recounts Nanci Taplett, the vendor’s public relations manager. The insurer procrastinated until suddenly, an AMS salesperson received a call from the insurer crying: “We have to move now. We have been in a flood and have lost everything.”

While the United States has been fortunate to avoid major disasters in past months, the result is that some firms may again be relegating disaster recovery plans to the back shelf.

One reason disaster recovery solutions are not a higher priority is that they can be expensive, especially when a firm builds a data center, with a staff, to which they can fail-over critical business operations.

Yet, cost is simply an excuse for some carriers. “The technology has become less expensive and more efficient,” says Stephanie Balaouras, senior analyst, Forrester Research Inc., a technology and market research company based in Cambridge, Mass. “So the opportunity to use the technology is there.”

Simply having a modicum of foresight and barebones technology, such as a few telephones working in other locations, can be very helpful in at least ensuring good client service, observes Linda Trevino, operations manager for Talon Insurance Agency, a firm based in Port Arthur, Texas, that has recently weathered hurricanes, tropical storms and fires. She recalls how several insurers were so unprepared for Hurricane Rita that employees merely left a note on the headquarters’ front door as they left town, noting that they were gone and ‘to contact this number’ (numbers that were not even those of company agents).

Companies can husband resources by focusing on those systems that must be most rescued or duplicated, rather than backing up their entire IT infrastructure. “It is possible for you to focus on very specific areas that must be up and running immediately and where you can’t afford any downtime,” says Andrew Barnes, senior VP of corporate development of Neverfail Inc., an Austin, Texas-based provider of disaster recovery solutions. Doing this can set back an insurer tens of thousands of dollars, rather than millions.” Insurance companies themselves have to make sure that IT dramas do not become crises,” explains Barnes. “And they become crises if the disaster recovery plan is out of date or doesn’t recognize the importance of communication and collaboration.”

THE BLUE CROSS APPROACH

Reducing downtime costs, and meeting demands to stay online and competitive are chief reasons as to why firms are enhancing disaster recovery strategies. Though banks and securities firms have greater recoverability requirements than do insurance firms, regulations are spurring some insurers, such as Blue Cross and Blue Shield of Minnesota (Blue Cross), to take a pro-active approach. “We are driven by things like HIPAA and Sarbanes Oxley, which don’t directly require certain recovery strategies,” says Kenneth Alwin, the business continuity program manager for the firm, which is based in the St. Paul suburb of Eagan. “But compliance with those requirements, in our interpretation, means that we have had to do a better job of evolving our recovery capabilities.”

Indeed, in an effort to improve operations following disasters, numerous insurers have enhanced their business continuity plans, such as Blue Cross. In 2005, the company began to move to a self-recovery strategy after having used a vendor-based hotsite approach. That year, Blue Cross built a secondary data center to increase replication, redundancy and resiliency. Then, earlier this year, it began migrating its mainframe with the same self-recovery strategy.

Since launching its new strategy, Blue Cross has seen dramatic improvements in recovery time—it has cut its distributed systems recovery time objective (RTO) to four hours—and greatly diminished exposure to data loss because its data is replicated between the company’s two sites in near real time.

Additionally, it slashed network costs by $1.5 million in one year by consolidating network contracts. It whittled down its number of primary telecom vendors to four from 20.

“When you look at things like 9/11, the Minneapolis bridge collapse last year, Hurricane Katrina and a lot of the regional disasters, it really hits home that disasters can happen anywhere,” says Alwin.

REDUNDANCIES AND ONLINE SERVICE

In order for business continuity strategies to be successful, companies need to assess what systems they need to make redundancies for. Generally, an insurer needs to make a redundant copy of an agency management or benefits management system, or whatever system it deems contains critical information.

Blue Cross, for one, feels it has hedged its risks appropriately. The company’s corporate campus is in the Metropolitan Twin Cities area, but has call center operations in two other Minnesota locations (one of which is located three and one-half hours from headquarters).

“We can transfer workloads for both call center operations and claims between our different facilities, and we have fully redundant systems between those,” Alwin says.

Moving data to the Internet is an obvious preventative mechanism for disaster recovery, and some vendors hail the benefits of Web-based and online disaster recovery solutions.

“With that combination an agent can suffer a pretty substantial disaster and then they can be back in business very quickly the next day,” says Bill Bunker, senior VP of product management and marketing for AMS Services. “All they really need is a Web browser and, boom, they are up and running.”

CRUCIAL STEPS

Though insurers might disagree on the best business continuity strategy, there are some critical steps that each should take, from conducting a business impact analysis and threat assessment, to putting together a disaster recovery plan and effectively implementing the technology.

Do a business impact analysis. An insurer must first institute a business impact analysis, identifying its most critical business processes. More specifically, a company must map all the dependencies for those business processes so it knows exactly what to recover, then decide what its sensitivity would be to a downtime or data loss.

Perform a local threat assessment. Second, a firm should undertake a local threat assessment, pinpointing the set of threat events within its locale that it needs to protect itself against. For instance, is your firm in a high-risk area for hurricanes or winter storms? Or are you located next to a power plant? “Probably the most common cause of downtime is power outages,” notes Balaouras. “So don’t forget all the mundane stuff that is actually pretty easy to solve with preventive measures.” Insurers should establish their data centers in areas having the lowest possible threat profile.

Formalize the plan. A firm should document its disaster recovery plan. This entails formalizing its plan via documentation (software can assist with this). It should include details on what redundancies the firm must have in place. The most important systems to copy are those that store information that the insurer will need to access post-disaster, says Bunker. Insurers also should assess how much they want to rely on the Internet for backup.

Test, test, test. To ensure that a disaster recovery strategy is successful, it must be updated regularly. To update the plan, testing must be fairly frequent. Blue Cross typically runs one large-scale, fully integrated systems test (or “exercise”) per year. It then conducts a series of smaller component or environment exercises. For the latter, it might conduct five to six exercises annually on a particular environment (say the call center, telecom or Web portal).

Promote a cultural shift. It is not enough for just a few staffers to be aware of the firm’s disaster recovery plans; most of the company has to be involved. “You have to educate your senior executives because they are the ones who sign off on the investment,” says Balaouras. “And they should understand what the strategy is and whether it actually meets their business requirements.” These executives, not IT officials, must decide how fast the company needs to get back in business.

Additionally, all heads of strategic business units, or of applications, need to be involved directly in disaster recovery. This means participating in both the development of the plan and the testing. One option: conduct a “table top” exercise, where executives sit around a table with colleagues who helped put together the disaster recovery plan. The officials can then “war game” the response to a disaster by walking through the plan as the disaster is happening, and discussing roles, responsibilities and activities.

Balaouras recommends that results from tests to be reported to different business units and senior executives.

ACT NOW

Insurance firms have increasingly followed those steps, protecting themselves following the recent series of terrorist incidents, environmental changes, flooding, forest fires and other disasters.

Nevertheless, approximately 27% of enterprises do not have a recovery site in the event of data center site failure; 23% of enterprises never test their disaster recovery plans; and 40% test their plans merely once per year, according to Forrester.

Just as this article attempts, awareness must be incessantly cultivated, compelling insurers to act now rather than later.

Daniel Joelson is a freelance writer based in Alexandria, Va.

For reprint and licensing requests for this article, click here.
Analytics Data and information management Policy adminstration
MORE FROM DIGITAL INSURANCE