February 17, 2016
Natural and man-made disasters are inevitable. Every business is susceptible to significant disruptions, security breaches and data loss that can affect a brand’s image, productivity and their bottom line.
You can’t avoid every fire, hurricane, tornado, earthquake or terrorist attack. But data center disaster avoidance strategies, effective testing, and disaster recovery planning can minimize the impact of any natural event, explosion, malware, theft or hardware malfunction that threatens to compromise data and harm a business. Without the right systems and audits, companies risk closing without the ability to reopen! Being prepared is the key.
It’s tough to operate a business if you can’t access essential data like:
- Customer databases
- Reports for customers
- Information about suppliers
- Tax records
- Sales documents
Many businesses certainly do invest in IT and do protect their data. Some, however, have much room to improve in terms of where they back up data, how often they test their backups, and the ways they report on the reliability of their systems (for their own standards, to meet the expectations of other businesses, and to comply with government regulations). MarketsandMarkets, a market research firm, way back in 2013 projected that the global Disaster Recovery as a Service and cloud-based Business Continuity market would grow from $640 million to $5.77 billion by 2018.
The February 26, 1993 World Trade Center (WTC) bombing was among the most notable disasters that served as a wake-up call for companies that they need to back up their data.
After the September 11, 2001 terrorist attack at the WTC, businesses lost considerable data. But many managed to recover data and resume operations in different locations. Geo-Diversity with data centers is a smart strategy.
Noting lessons learned from the 1993 bombing, Morgan Stanley Dean Witter analyst Charles Phillips in 2001 told InformationWeek: “It was probably one of the best-prepared office facilities from a systems and data-recovery perspective.”
In 2002, the New York City comptroller’s office estimated that it would cost $55 billion to replace property, including $6 billion for computers, furnishings and cars in the WTC and surrounding buildings. The estimate included $24 billion for the loss of life – 2,753 people in the WTC area (including what they would have earned). The economic impact was estimated at $123 billion.
Preparation for any disaster is vital, but companies sometimes fall short of their potential when it comes to IT
Highlights from the report include:
- Increasingly, respondents aren’t confident that they’re prepared for a data disaster (16% are confident compared to 23% in 2010).
- Only 39% of the companies surveyed run a full data recovery test at least once a year.
- More of the businesses (15%) rely on the Cloud for data protection
- For mission critical systems, more than half of the respondents use technology like replication.
- Power failure continues to be the leading cause of data disasters.
- One in three respondents had a data disaster in the last five years (an increase from one in five in a 2010 report).
The Disaster Recovery Preparedness Council gave companies low marks in its 2014 benchmark survey, “The State of Global Disaster Recovery Preparedness.”
Nearly three out of four companies worldwide earned either D or F grades based on their disaster recovery preparedness. For example, 60% of the companies that took the survey don’t have a fully documented disaster recovery plan. And 40% indicated that the disaster recovery plan they use wasn’t effective when they referenced it during their worst disaster recovery event or scenario. About 25% lost all of most of their data centers for hours or days at some point over a year.
Although many respondents test their disaster recovery plans, 23.3% don’t. And when they do conduct tests, 65% don’t even pass their own tests, according to the Disaster Recovery Preparedness Council survey.
In a 2012 survey, BUMI (Backup My Info!), a managed service provider, found that 30% of CEOs believe business continuity is a critical reason that data should be backed up. Although 98% expected their companies to have access to data at least 24 hours after a disaster, 23.8% said they never check the restore process.
Data loss for any period of time takes a toll. Emerson Network Power teamed up with Ponemon Institute to produce the “2013 Cost of Data Center Outages” report that included perspectives from 450 data center professionals and an analysis of 67 data centers.
The study quantified the costs of unplanned data center outages at a little more than $7,900 per minute, a 41% jump from 2010.
The study noted that costs include detection, containment, recovery, productivity losses and other expenses. More than 50% of respondents believe the outages could have been prevented. Causes included:
- UPS battery failure (55%)
- Accidental EPO/ human error (48%)
- UPS capacity exceeded (46%)
- Cyber attack (34%)
- IT equipment failure (33%)
- Water incursion (32%)
- Weather related (30%)
- Heat related/CRAC failure (29%)
- UPS equipment failure (27%)
- PDU/circuit breaker failure (26%)
Some data center disasters don’t involve the loss of data, but the financial hit (and damage to reputation) can be enormous when there is a security breach or if key information is mistakenly shared with the public. Data center security has become big business and is vastly important to companies today.
Here are a few examples from the last several years of bad actors, cyber-thieves and employee accidents:
- Heartland Payment Systems in 2009 discovered that hackers were able to steal data related to 130 million credit and debit cards.
- The Transportation Security Administration in 2009 mistakenly revealed airport passenger screening practices on a federal web site (including technical settings related to explosive detectors and X-ray machines).
- Health Net Inc., an insurer, in 2011 lost track of unencrypted server hard drives that included the names of patients, addresses, medical information, social security numbers and more.
- Data breaches in 2014 affected several retailers, including Target (credit and debit cards and contact information for 110 million people and Home Depot (credit and debit cards and contact information for 109 million people).
Coping with Natural Disasters
Between 1994 and 2013, there were 6,873 natural disasters worldwide that killed more than 1.3 million people, according to the Centre for Research on the Epidemiology of Disasters (CRED) and its Emergency Events Database EM-DAT. Disasters included storms, floods, earthquakes, tsunamis and more.
Vulnerable to natural disasters at any moment, businesses can ill-afford not to back up data and test data recovery plans.
The financial effects of data disasters vary year to year based on the number and types of events. One global measure is EMC’s Data Protection Index, which tracks data protection strategies and the relative preparedness of businesses.
In 2014, EMC’s Data Protection Index calculated that enterprises worldwide lose at least $1.7 trillion each year because of data loss and unplanned downtime (based on interviews with more than 3,000 IT decision makers in 24 countries).
Some businesses just don’t put enough thought into a data crisis that can easily emerge.
In 2012, software maker Sage North America released the “The Small Business Disaster Preparedness Study,” which found that 62% of businesses lack a formal emergency/disaster preparedness plan.
Although the respondents do back up their financial data, only 39% store the information off-site or through the Web.
When given options to explain why they don’t have a formal disaster or emergency preparedness plan, one-third selected this answer: “I’ve never had an issue before/disasters are rare in my area.”
Another 30% preferred this option: “I haven’t really thought about it” and 27% chose “I don’t think it’s important for my business.”
Tips for Effective Disaster Recovery Planning for Data Centers
Company executives, including IT managers and directors, need to understand the risks and how much they’ve prepared for a disaster. The following questions can help you evaluate or improve your process:
How seriously do you regard natural and man-made disasters? Are they events that you’re ready to handle, or are they viewed as worst-case scenarios that get little attention until it’s too late?
- How much data loss can you tolerate?
- How much time would you need to recover?
- For the most critical applications, have you defined a set of Recovery Time Objectives (RTO) and Recovery Point Objectives (RPO)?
- How do you value unplanned downtime based on different areas of the business?
- Do you have a disaster recovery plan? If so, what areas of the business contributed to it (not just IT)?
- How much of the company’s data is vulnerable if employees have a habit of saving documents to their desktops?
- How often do you test your data recovery procedures?
- Is there a schedule that you follow to test batteries, generators and emergency power sources?
- How reliable is the data that you back up? Is it current? Is it accurate?
- How difficult is it to retrieve the data?
- Do you automate backups?
- Are you using best practices in your data center?
- How much support you have from senior executives as you continually identify ways to avoid and respond to unplanned outages?
- Does your business leverage sever virtualization to assist with disaster recovery?
- How secure is your firewall to block intruders before they reach your network?
- How will you handle business continuity challenges? What tasks must be done right away and which ones can be deferred?
- Where will employees work? Do you have other offices that can accommodate workers? Will they work at home? Do you have reciprocal agreements with other businesses if your company must temporarily close its doors?
Reliable disaster recovery plans are critical for businesses that should be proactively protecting their data. Without effective backup systems and regular testing, companies will face unnecessary costs and could miss out on vital sales if disasters strike, crippling companies so severely that they can’t provide the services or produce the products that customers and investors expect.
Are you prepared for the next storm, fire, earthquake or computer virus that could wipe out key data? Altus can help you to be prepared. We offer disaster recovery testing and disaster avoidance services for data centers. Please contact Altus to get this important conversation started for the safety and health of YOUR data center. Learn more about Altus and our data center services to keep your company healthy and prosperous in the IT Age.