How Most Companies Plan for Disasters
For several years Business Continuity Planning has been the exclusive domain of the Disaster Recovery Planner/Coordinator (DRC). Every year the DRC performs a Business Impact Analysis (BIA) to determine the employee base's tolerance for application outages caused by a catastrophic disaster. With the BIA results, the DRC begins the process of working with the IT operations department to identify the most cost-effective ways to meet the RTO and RPO targets identified during the BIA.
Most of the DRC’s time is spent preparing for the annual disaster recovery test. This planning can take months of time working with various stakeholders but the result is nearly always the same – the company takes a group of its best technologists to an off-site location (3rd party or otherwise), executes the recovery plan as written using a pristine set of full backups, validates the recovered applications do in fact start and then celebrate their success. All within the prescribed recovery test window and only for those applications deemed testable during the planning.
The DRC compiles a list of findings that are distributed to the stakeholders, and obvious gaping holes are addressed and the checks are put into the appropriate boxes to be shared with necessary auditors and reviewers.
The entire situation’s most frustrating part is that nothing fundamentally changes from year to year and the only people that can truly feel satisfied are those that have to answer the auditors’ questions – “Yes, we successfully conducted a disaster recovery test.” Unfortunately, a number of serious issues are either ignored or avoided because everyone has more important things to be doing. And that is the true dichotomy, the smartest people in your company spend collectively less than one week a year contemplating how to recover from the wrong type of disaster. And in this short-sightedness, many issues remain unaddressed.
Complexity, as applications have become more sophisticated, the number of dependencies between separate systems have grown rapidly and the nature of these interconnections have also become more sensitive to failure. Most bet-the-business applications are now interconnected to the point where they appear as one business system.
Ecosystem expansion, most companies have embraced some form of 3rd party software as a service (SaaS) application such as salesforce.com. As these applications become more ingrained in the customer environment it becomes increasing harder to identify the interdependencies and their impact post-disaster. The very nature of what constitutes a disaster if often overlooked. Clearly, an act of God that destroys a computer room and renders a company’s application inaccessible is a disaster but why aren’t malicious human activities that deliver the same results considered with the same degree of concern.
For many companies, the Disaster Recovery Coordinator’s role is rapidly reaching an end. Companies don't want to spend monthly planning for an annual recovery test using irrelevant or obvious data gathered during a time-consuming BIA.
All this means companies are looking for new ways to address the availability of their critical applications. We will address those in the next blog.