BEST PRACTICES in DISASTER RECOVERY TESTING EDITOR’S LETTER 1 Following a Disaster Or Other Outage

Handbook VIRTUALIZATION CLOUD DEVELOPMENT APPLICATION NETWORKING ARCHITECTURE STORAGE CENTER MANAGEMENT DATA BI APPLICATIONS RECOVERY/COMPLIANCE DISASTER SECURITY 1 EDITOR’S NOTE 2 Best Practices in Disaster AVOID THESE COMMON Recovery Testing DR TEST MISTAKES DR testing is frequently put off or overlooked entirely. However, 3 IMPROVING BC many surveys show that IT pros are not confident in their ability PLAN EXERCISES to recover in a timely manner following a disaster. 4 DISASTER RECOVERY COSTS: MAKE TESTING, PLANNING COST-EFFECTIVE EDITOR’S LETTER 1 Technology is No Substitute for Good Backup Strategy

Every year, we read a crop of surveys that indicate organizations are not con- Home fident in their ability to recover data after an outage. There are a variety of reasons Editor’s Letter why IT people lack confidence in their DR plans, but many simply lack confidence in the backup/DR technologies they rely on. Avoid These The thing is, it shouldn’t really be about that. The technology is just a tool to Common DR Test Mistakes achieve a goal, and it’s too easy to just say ‘Oh well, it doesn’t work.’ That’s not going to fly where near 24/7 uptime is expected and organizations are subject Improving BC to regulatory compliance mandates requiring that data is protected and easily Plan Exercises accessible. Disaster Technology isn’t a substitute for a good backup strategy. And testing is the Recovery Costs: only way to find holes in your strategy. Too often organizations think of DR as Make Testing, Planning Cost- a ‘set it and forget it’ exercise when DR really needs to be ongoing. Also, many Effective people believe that all DR tests have to be elaborate. But, while full-scale DR tests should be conducted, there is also a lot to learn from smaller table-top exercises conducted frequently to test specific aspects of your plan. Whatever the size or shape your test takes on, though, the goal is to find vulnerabilities. It is important not to conduct tests under perfect conditions—conditions will be far from perfect

2 BEST PRACTICES IN DISASTER RECOVERY TESTING EDITOR’S LETTER 1 following a disaster or other outage. It’s important for tests to simulate reality. This Handbook offers a Q&A with independent DR expert Jon Toigo that cov- ers a variety of DR testing topics from how frequently tests should be conducted to exactly what tests should entail. You will also find an article about conducting cost-effective DR tests and a piece about how and why testing should be built Home into your DR plan. n Editor’s Letter Andrew Burton Avoid These Senior Site Editor, SearchDisasterRecovery.com Common DR Test Mistakes

Improving BC Plan Exercises

Disaster Recovery Costs: Make Testing, Planning Cost- Effective

3 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 Avoid These Common Mistakes in DR Tests

Disaster recovery planning is useless if you can’t restore business operations Home following an interruption. So, setting up a DR plan and forgetting about it until Editor’s Letter something happens isn’t an option. The only way you can be sure that your disaster recovery measures will be adequate is to test your plan regularly. Avoid These Why, then, do so many organizations neglect to test their plans or perform Common DR Test Mistakes inadequate tests? Because good DR testing can be time-consuming and difficult. In this Q&A, independent disaster recovery expert Jon Toigo discusses some Improving BC of the most common mistakes organizations make when performing DR tests, the Plan Exercises variety of types of tests you should consider running, how frequently tests should Disaster be conducted, and the variety of technologies involved in DR testing. Recovery Costs: Make Testing, Planning Cost- In every disaster recovery methodology, testing Effective is a huge component. What makes it so important? I could recite lots of platitudes, but as a practical matter, testing is key to change management in disaster recovery planning. From this standpoint, testing helps to identify “gaps” in the recovery capability that you have developed to respond to a disruptive event and to assure continuous or near-continuous business

4 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 operations. You need to test periodically to identify changes in personnel, business processes, applications and technology infrastructure—things that change constantly in the real world—that may impact the strategies you have developed to accomplish the three core objectives of DR: data restoration, application re- hosting, and user re-connection. Following the test, you take the information Home acquired to improve the plan. Then you re-test. Editor’s Letter The more important value of testing is rehearsal. Plans tend to be complex instruments with lots of moving parts that require multiple recovery teams to Avoid These work in concert. The more you can rehearse, the better your team members will Common DR Test Mistakes understand their individual roles and the interdependencies between what they are doing and what others are doing. That is very important since it enables the Improving BC teams to work in a mostly independent way and to perform tasks in a reasonably Plan Exercises dependable way even in the face of a great irrationality—a disaster. Disaster Recovery Costs: What are the components of testing as it is conceived today? Make Testing, Planning Cost- I think we are doing testing wrong today. Basically, tests are conducted on sched- Effective uled days, once or twice a year, or sometimes quarterly. We take teams off site and hold an event where we test how we will recover data from a backup or mirror, how we will restore applications in a minimum equipment configuration, and how we will reconnect application hosts to a network so that users can get to their work with adequate if not optimal performance levels and security.

5 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 This is the way testing has been done for a long time, but it has the limitation of being non-linear. We break strategies down into tasks, and we test tasks out of order—in a non-linear fashion. That way, the failure of one test will not prevent us from undertaking the other scheduled tests. Such an approach obviates the teach- ing value of testing. The human brain doesn’t readily reorganize non-linear tests Home into a coherent end-to-end understanding of the strategy, the team member’s role Editor’s Letter in the strategy, or the interdependencies between the roles and activities of different team members. That pretty much demolishes the rehearsal value of testing. Avoid These Add to this one other criticism: The act of formal test preparation tends to Common DR Test Mistakes skew the outcome of the test. We pull the right backup tapes in advance, ensure that all necessary tools and equipment are present, and so forth—things that make Improving BC the test less and less like a real-world application of procedures in the face of an Plan Exercises actual disaster. That also limits the efficacy of traditional testing. Disaster Recovery Costs: In an ideal world, what would a DR test entail? Make Testing, Planning Cost- In my perfect world, we would have defined strategies for data protection and Effective restore that can be tested in real time, on an ad hoc basis, using either simulated or “live” procedures. Backups to tape should be subjected to read/write verifica- tion to ensure that the data replicated is the right data and that it can be restored when needed. Data replicated to disk also must be verified routinely and not as a part of some formal test event.

6 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 Ideally, we would select strategies for application re-hosting and for network reconnection that also avail themselves of testing at any time during the normal operating day, and without disrupting normal operations. Geo-clustering holds out the promise of such a strategy, as do, to a certain extent, server and storage virtualization techniques. Again, you should be able to confirm that system and Home network recovery capabilities are up to the task without waiting for a formal test Editor’s Letter event to find out. If we could accomplish these goals, formal testing would come down to a Avoid These much simpler set of tasks having to do with logistics—how the disaster would be Common DR Test Mistakes identified, who would be contacted and in what order, how customers would be notified, how teams would travel to a recovery facility, how externalized services Improving BC (e.g., “clouds,” hot sites, vendors tasked to drop-ship recovery supplies, the phone Plan Exercises company, user facilities, etc.) would be activated, how the order of recovery tasks Disaster would unfold. That could be tested in a very linear fashion and at much less ex- Recovery Costs: pense than traditional testing entails today. Make Testing, Planning Cost- Effective What are the biggest mistakes you see companies make when performing a DR test? First and foremost, fewer than 50% of companies with plans test them at all. That is a huge mistake. If you are going to go to the trouble of defining a continuity capability and provisioning your strategies with people and resources, you ought to test the result to make sure that your theory of operation matches the reality of

7 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 a disaster event. These days, with budget dollars in short supply, too many firms are skipping the test—or at least postponing it into oblivion. Not a good idea. When companies do test, something I am occasionally brought in to observe, the same sort of problems tend to arise. The short list includes:

Home ■■ Failing to bring the right tapes or other recovery media to the test Editor’s Letter ■■ Too many cooks: management wants to show their support and attends the test, Avoid These getting in the way and wasting precious time. If they want to be part of the test, Common DR Test Mistakes give them a clipboard and make them observe and take notes.

Improving BC ■■ Lack of standards for test data collection: taking notes on random scraps of paper Plan Exercises makes correlating notes with tested activities, and producing summary reports, Disaster a difficult process Recovery Costs: Make Testing, Planning Cost- ■■ Failure to perform post-mortem interviews with test teams while the experience Effective is still fresh in their minds—that’s where you obtain some of the best data for procedure refinement

■■ Tendency to shape results to confirm or validate planning, ignoring failures: there is no such thing as a failed test. Tests produce information that can and

8 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 should be used to improve procedures so that recovery activity can be accom- plished within the specified timeframe. Yet, many planners try to hide “failures” to prevent management from losing faith in the project.

Why is it important to break mirrors as part of your DR test? Home Mirroring is the process of replicating data between two disk stores, usually across Editor’s Letter a local storage interconnect. Its cousin, replication, involves the same process but across distance using a WAN. Mirroring is usually viewed as synchronous, while Avoid These replication is asynchronous (distance induced latency creates data deltas or dif- Common DR Test Mistakes ferences between the state of local and remote data stores). While mirroring is regarded as a synchronous form of data copy, and pretty Improving BC close to real time if done correctly, a lot of folks outsource the mirroring process Plan Exercises to their array vendor. They copy after write rather than during write. In effect, Disaster data is written to disk on array #1, then it is copied by on-array software to an Recovery Costs: identical array #2 located nearby using a high speed/high bandwidth link. This Make Testing, Planning Cost- approach drives up the cost of mirroring by locking you into the same vendor’s Effective gear on both the primary and mirrored array. But the real problem is one that a little software company, 21st Century Soft- ware, has been showing off in its presentations for a couple of years. They show actual screen shots of multiple customers who thought their hardware was mirroring the right data, only to discover that NO DATA was actually being mirrored.

9 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 This usually happens when the volumes in the array that hold data are reorganized by the vendor service tech or by a storage administrator, but the DR coordinator is not informed of the change. Without periodically breaking the mirror, you might not notice the problem… until an actual disaster occurs. So, why don’t folks break their mirrors and test? Simple: It is a hassle to quiesce Home applications, flush caches to volume 1, replicate to the mirrored volume 2, then Editor’s Letter shut everything down to do a file by file compare between the two. It takes time, both operator time and time from production application work, and there is no Avoid These certainty that systems will restart and mirroring will resume properly. Common DR Test Mistakes Remote replication has some of the same problems, but it may be possible to test for data deltas without quiescing the replication process. There, you have a Improving BC ton of other issues related to latency and to jitter. You need to be vigilant that you Plan Exercises have a data store to which you can recover. Disaster Recovery Costs: How frequently should DR tests be conducted? Or should Make Testing, Planning Cost- it be determined by changes to your infrastructure? Effective Depends what you mean by test. Formal test events, as we noted previously, entail a lot of activities that planners would be smart to design into the strategies themselves. We should be able to validate data protection and recovery on an ad hoc basis—every day! With geo-clustering, we could simulate or perform failovers to remote kit and networks on a simulated or actual basis at any time of the day or

10 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 night. That leaves logistics testing, which only needs to be done two to four times a year, and most certainly following significant changes to business processes or infrastructure. Table-top exercises and plan walkthroughs are great testing vehicles that can be used much more frequently than formal off-site test techniques. They have Home the merit of being much less expensive to conduct and have fewer logistical re- Editor’s Letter quirements, and they give teams a chance to interact with each other as they walk through documented procedures for accomplishing their work. Avoid These Just keep in mind that these tests must also be conducted in an objectives- Common DR Test Mistakes driven way. You may not be testing to a stopwatch, but interactions need to be kept sufficiently formal and professional to exercise the concept of a strategy or Improving BC procedure in an effective way. Plan Exercises

Disaster How can storage virtualization help DR testing? Recovery Costs: Storage virtualization entails the establishment of a software controller ahead of Make Testing, Planning Cost- all vendor equipment. Generally speaking, you negate the “value-add” software Effective installed on heterogeneous storage hardware and use the “uber-controller” to serve value-add functionality across all disks in all cabinets. Among the services usurped from proprietary hardware controllers and pro- vided instead, and on a more universal basis, by the storage virtualization uber- controller are data protection functions like mirroring, replication, continuous

11 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 data protection, snapshot copies, etc. These services are centralized in the storage hypervisor software layer, away from hardware, so they are not limited or con- strained by the brand names on the various storage arrays—data can be replicated across spindles in any of the heterogeneous storage rigs, which are represented as virtual volumes. If your remote storage environment is also virtualized, storage Home hypervisors typically feature the ability to target remote volumes with writes, Editor’s Letter simplifying WAN replication. Perhaps the coolest part of this scenario is the ability to manage, allocate, and Avoid These test data protection services from one place—a single pane of glass. That simpli- Common DR Test Mistakes fies the testing and validation of data protection processes; in some cases, obvi- ating the need to quiesce applications to test mirrored volumes for consistency. Improving BC Plan Exercises If you use a “hot site” for DR, how does the testing process Disaster typically work and does it differ widely between providers? Recovery Costs: With a hot site, as with some of the current generation “cloud” or managed hosting Make Testing, Planning Cost- solution providers, you arrange for a certain amount of test time over the course Effective of the year as part of your subscription agreement. The quantity of test time pro- vided, the roles that will be played by service provider personnel, the specifics on how tests need to be scheduled, as well as the minutiae of security, logistics, and communications, will vary from provider to provider. I believe that too little is being done to validate the measures that service

12 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 providers are deploying to safeguard your applications and data—especially when it comes to platform and infrastructure cloud service providers. They all claim to provide Tier 1 data centers and comprehensive professional operations capabilities, but very few clients actually ever travel to the cloud provider sites and confirm that they are getting anything more than some old servers racked in a bent-up Home cage at the end of a row in some musty old managed hosting facility. Editor’s Letter What else should people consider before choosing the cloud as a backup target? Avoid These Cloud storage could be an effective backup target, and I don’t paint all cloud pro- Common DR Test Mistakes viders with the same brush. But the facts need to be closely considered. First, there is usually one charge for writing data to the cloud storage target, and Improving BC other charges for getting the data back. You need to know what all of the charges Plan Exercises actually are. Disaster Second, most cloud storage targets provide adequate bandwidth for transport- Recovery Costs: ing changed data to the cloud once the original or full data backup has been copied Make Testing, Planning Cost- (often a teeth-pulling experience). Following a disaster, you need your data—per- Effective haps close to all of your data—restored. Chances are good that the WAN connection to the cloud is inadequate to this task. Remember: it takes more than a year to move 10TB of data across a T-1 (DS-1) WAN link. While MPLS networks may provide bigger pipes, using them typically means that your backup data is inside the boundary we might consider to be a

13 BEST PRACTICES IN DISASTER RECOVERY TESTING DR TEST MISTAKES 2 minimum safe distance demarcation between your production data center and recovery environment (about 50 kilometers). Those constraints need to be taken seriously. I personally would never use a cloud backup service provider whose facility was closer than 50 kilometers or one that could not provide me my data back on tape. Home

Editor’s Letter Do cloud providers offer anything to facilitate testing? It really varies from service to service. Most do not want to touch your data as a Avoid These function of legal liability avoidance, so they do little to validate your backups or Common DR Test Mistakes replicas. —Jon Toigo

Improving BC Plan Exercises

Disaster Recovery Costs: Make Testing, Planning Cost- Effective

14 BEST PRACTICES IN DISASTER RECOVERY TESTING BC PLAN EXERCISES 3 Improving BC Plan Exercises

Planning and conducting business continuity (BC) plan exercises is one Home of the most important activities in a business continuity program. Editor’s Letter Conducting one or more BC plan exercises annually is a key component of a business continuity management system (BCMS). Exercises should be scheduled Avoid These and integrated with other BCMS activities, such as plan updating, emergency team Common DR Test Mistakes training, policy reviews and audits, business impact analyses (BIAs), risk assess- ments (RAs), and awareness programs. Improving BC A BC plan exercise is not the same as a disaster recovery test. For instance, Plan Exercises you don’t actually failover in a BC plan exercise. That’s what you do in a typical Disaster technology disaster recovery test, which addresses the recovery of IT systems, Recovery Costs: data, databases and so on. This is strictly business continuity. Make Testing, Planning Cost- When planning a BC exercise, the following are priorities: Effective

1. Decide specifically what you plan to exercise, e.g., the entire plan or parts of the plan such as incident response procedures or the evacuation plan.

2. Secure a location to conduct the test that is away from any possible interruptions,

15 BEST PRACTICES IN DISASTER RECOVERY TESTING BC PLAN EXERCISES 3 and encourage exercise participants to turn off their mobile devices if possible so they can concentrate on the exercise. If possible, conduct the exercise outside the participants’ offices in a less conspicuous location. If this is not possible, it may make sense to schedule the exercise outside of normal work hours or perhaps over a weekend. Home

Editor’s Letter 3. It may be useful to invite participants other than the exercise developer(s) and representatives of the department(s) or activity being exercised, such as staff Avoid These from IT, operations, risk management, human resources, legal, quality assur- Common DR Test Mistakes ance and internal audit, but this is not mandatory. A corollary to this is to have the “right” participants in the exercise. This means inviting people who have Improving BC a true stake in protecting their department, as well as the company. Inviting Plan Exercises senior management to an exercise is often avoided because the fear is that a Disaster senior manager may get too involved (e.g., try to take over the exercise) and Recovery Costs: other exercise participants may reduce their level of participation in deference Make Testing, Planning Cost- to the executive. Effective

4. It’s not necessary to complete a “successful” exercise. Completing a successful exercise doesn’t necessarily mean that the plan ran perfectly, the emergency team is fully prepared or that employees are ready to respond. It’s far better to identify flaws in the exercise logic and supporting activities now, rather

16 BEST PRACTICES IN DISASTER RECOVERY TESTING BC PLAN EXERCISES 3 than later (e.g., during an incident), when the flaws could result in serious consequences. You should also assign someone as a timekeeper and scribe, so that a record of the exercise can be produced. This is important from an audit perspective and also for regulated organizations like banks or firms that are scrutinized by Home government agencies, such pharmaceutical companies and the U.S. Food and Editor’s Letter Drug Administration (FDA). And, it’s a good practice for all exercises. While not usually a priority, consider launching a surprise exercise in addi- Avoid These tion to scheduled exercises. This is perhaps the best way to determine if your Common DR Test Mistakes emergency teams are really prepared to respond to a business-threatening incident. Some advance planning (e.g., warning) is advised, especially if your exercise Improving BC affects other departments, such as IT or facilities. Also, if other departments, Plan Exercises such as IT, have scheduled exercises the same time as your surprise event, it Disaster may be prudent to reschedule. Of course, in real life, there will be no advance Recovery Costs: warnings or courtesy calls alerting you and others of an impending disaster. Make Testing, Planning Cost- Effective Well-planned and conducted BC exercises are important investments in a company’s long-term success and survival. Knowledge of regularly scheduled exercises can also enhance the firm’s reputation and competitive position, especially since more organizations today require data about a prospective vendor/partner’s business continuity and disaster recovery activities. —Paul Kirvan

17 BEST PRACTICES IN DISASTER RECOVERY TESTING MAKE DR AFFORDABLE 4 Disaster Recovery Costs: Make Testing, Planning Cost-Effective

For many, the biggest inhibitor to implementing an effective disaster recovery Home (DR) plan is cost. The shock that is often associated with the price of a proposed Editor’s Letter DR solution becomes something that organizations strapped for funds are simply unwilling to swallow, so they may end up stopping a disaster recovery project in Avoid These its tracks. Common DR Test Mistakes The question then for many data storage administrators is often, “Okay, how much DR can we buy for this amount of money?” rather than, “Here are the Improving BC capabilities we need, let’s find a way to make it affordable.” The goal for every Plan Exercises company trying to compose a cost-effective disaster recovery plan should be to Disaster understand the range of cost-saving options and the associated tradeoffs, and Recovery Costs: revise their disaster recovery strategy based on the choices deemed most Make Testing, Planning Cost- acceptable. Effective In this article, learn about where you can look for efficiencies that will not compromise disaster recovery. Learn about where you can find opportunities for savings in areas like disaster recovery testing and new technologies that will reduce disaster recovery costs. Here are some areas to explore to realize more cost-effective disaster recovery:

18 BEST PRACTICES IN DISASTER RECOVERY TESTING MAKE DR AFFORDABLE 4 1. Eliminate idle assets. One of the greatest cost contributors to disaster recovery is the expense of maintaining assets that largely sit idle waiting for a disaster to happen. For years, it was not uncommon to see servers and data storage sitting unutilized in a disaster recovery facility. Today, very few organizations can allocate funds for such unused capacity. Devising a plan that allows sys- Home tems to be multi-purposed to any degree can dramatically improve the disaster Editor’s Letter recovery cost structure. One of the most common approaches is to leverage test and development Avoid These environments as backups for disaster recovery. The challenge here is determin- Common DR Test Mistakes ing how long these functions can be unavailable during a disaster situation.

Improving BC 2. Standardize. The more different types of widgets that are deployed, the greater Plan Exercises the number of distinct types of resources are needed for disaster recovery. Disaster Likewise, the number of different configurations and system software variants Recovery Costs: of the same platform make DR design and testing more costly. Limiting vari- Make Testing, Planning Cost- ants of platforms and other infrastructure components and defining standard Effective configurations reduces complexity and unnecessary costs.

3. Automate. Beyond standardization, the use of automation, where possible, can simplify the testing process, improve reliability and drive efficiencies by reducing the number of hours necessary to complete tasks. Automation

19 BEST PRACTICES IN DISASTER RECOVERY TESTING MAKE DR AFFORDABLE 4 opportunities exist in areas like system deployment, data replication, and host or application failover.

4. Virtualize. Perhaps the greatest potential opportunity to drive down DR planning and testing costs today is offered by virtualization. Server virtualization Home mitigates the idle assets problem and can greatly assist in efforts to automate Editor’s Letter the disaster recovery process, not only reducing costs, but offering the potential for improved levels of service. Avoid These Storage virtualization also has a role to play, but other than adding a com- Common DR Test Mistakes mon management layer, it’s simply another way to offer things like data migra- tion, data replication and snapshots. Improving BC Plan Exercises 5. Operationalize. All too often the DR process is treated as an exception, some- Disaster thing that exists somewhere on the fringes of IT rather than as part of the day- Recovery Costs: to-day IT operations. Make Testing, Planning Cost- By better integrating disaster recovery into the core functions of IT and Effective thinking about DR as part of application development, architectural design and operational planning activities, more efficient DR solutions can be implemented. Creating after-the-fact or one-off solutions that must be force-fitted and then exist as exceptions are costly and become difficult to manage and maintain.

20 BEST PRACTICES IN DISASTER RECOVERY TESTING MAKE DR AFFORDABLE 4 6. Document. A traditional weakness in the realm of disaster recovery is the availability of current and comprehensive documentation. What does this have to do with cost? Besides the real risk of extended downtime and delays in the event of a disaster, poor or missing documentation can contribute to cost over- runs in disaster recovery testing and can require extended time from senior Home resources where, with proper documentation, more junior personnel could get Editor’s Letter the job done.

Avoid These 7. Simplify. Complexities that drive up costs have a way of creeping into orga- Common DR Test Mistakes nizations if steps aren’t actively taken to avoid them. Previously mentioned factors like standardization and virtualization can go a long way towards sim- Improving BC plification, but there are other ways to simplify. For example, in many cases, Plan Exercises customers implement discrete point solutions, such as one-off host-based Disaster replication or backup solutions, to support the recovery of a specific applica- Recovery Costs: tion, which is often based on application vendor recommendations. In extreme Make Testing, Planning Cost- cases, this can result in multiple data backup or replication technologies that Effective each must be supported and managed by IT. While unique application requirements must be considered, supporting multiple recovery solutions can become a DR management nightmare.

8. Compartmentalize testing. Disaster recovery testing can be a highly disruptive

21 BEST PRACTICES IN DISASTER RECOVERY TESTING MAKE DR AFFORDABLE 4 event that impacts day-to-day operations, raises overtime costs and generally increases anxiety. While large-scale DR testing is essential, consistent compartmentalized testing of networks, servers, storage and, to some degree, application components can help ensure recoverability while helping to reduce the disruption and avoid unplanned expenses related to failed or delayed DR Home tests. Editor’s Letter 9. Optimize data. When it comes to reducing or avoiding DR costs, less can be Avoid These more. Technologies such as data deduplication and thin provisioning reduce Common DR Test Mistakes underlying data storage footprints and can also significantly decrease bandwidth requirements needed for data replication—thereby making DR signifi- Improving BC cantly more affordable. Another often overlooked data optimization practice Plan Exercises that can reduce DR data footprint and traffic is data archiving. It’s important to Disaster consider that from a data perspective, disaster recovery is primarily concerned Recovery Costs: with currently active data sets, but the reality is that often the majority of data Make Testing, Planning Cost- sitting on today’s storage arrays is non-current, historic data that has accumu- Effective lated over time. A program to purge unneeded data or move it offsite to a cloud or secondary repository could greatly reduce the data storage capacity required at a DR location and may likely even help speed up the recovery process—not to mention the savings associated with freeing up expensive primary storage at the primary location.

22 BEST PRACTICES IN DISASTER RECOVERY TESTING MAKE DR AFFORDABLE 4 In the end, disaster recovery is a type of insurance that people invest in, and then hope that they never need. Regardless if you never think you’ll use your DR plan, it’s important to recognize the necessity of being properly protected. The key is to buy enough, but not too much, and the items listed above represent areas to consider when making long-term planning decisions. Determining the right Home approach requires taking the time to fully understand the problem before diving Editor’s Letter into specific technical solutions. Once the problem is understood, however, the technical nuances of different solutions become critical and can impact disaster Avoid These recovery costs substantially. The good news is that the cost of the core technolo- Common DR Test Mistakes gies that support DR are more affordable than ever. Ultimately, the challenge is that cost-effective disaster recovery is more a matter combining the right tech- Improving BC nologies with the right policies and processes, and this is where organizations Plan Exercises frequently come up short. —Jim Damoulakis Disaster Recovery Costs: Make Testing, Planning Cost- Effective

23 BEST PRACTICES IN DISASTER RECOVERY TESTING ABOUT THE AUTHORS JON TOIGO is CEO and managing partner of Toigo Partners International. He has penned thousands of computing articles and columns, and has written 15 books on computing. He has chaired the Data Man- Best Practices in Disaster Recovery Testing agement Institute since 1992. is a SearchDisasterRecovery.com Home e-publication.

PAUL KIRVAN, CISA, FBCI, works as an in- Rich Castagna | Editorial Director Editor’s Letter dependent business continuity consultant/ Andrew Burton | Senior Site Editor auditor and is secretary of the Business Ed Hannan | Managing Editor Avoid These Continuity Institute USA chapter and mem- John Hilliard | Associate Site Editor Common DR ber of the BCI Global Membership Council. Test Mistakes Sonia Lelii | Senior News Writer He can be reached at [email protected]. Linda Koury | Director of Online Design

Improving BC Neva Maniscalco | Graphic Designer Plan Exercises JIM DAMOULAKIS is CTO at GlassHouse Jillian Abbott | Publisher Technologies, a leading independent pro- [email protected] Disaster vider of storage and infrastructure services. TechTarget Recovery Costs: 275 Grove Street, Newton, MA 02466 Make Testing, www.techtarget.com Planning Cost- © 2013 TechTarget Inc. No part of this publication may be trans- Effective mitted or reproduced in any form or by any means without written permission from the publisher. TechTarget reprints are available through The YGS Group. About TechTarget: TechTarget publishes media for information technology professionals. More than 100 focused websites enable quick access to a deep store of news, advice and analysis about the technologies, products and processes crucial to your job. Our live and virtual events give you direct access to independent expert com- mentary and advice. At IT Knowledge Exchange, our social commu- nity, you can get advice and share solutions with peers and experts.

24 BEST PRACTICES IN DISASTER RECOVERY TESTING