Data Backup for Clcs

Data Backup for CLCs

Author: Justin Finighan, IT consultant NACLC, Director Finrea Pty Ltd. Version: 1.1 10 November 2011

This document describes how you can ensure that data at your centre is being correctly secured. Data security is critical to the day-to-day functioning of your Community Legal Centre. Many CLCs are not able to confidently say that they have an adequate data backup and disaster recovery plan in place. Following the steps in this document will assist you to ensure that your data is adequately protected.

Feedback and suggestions for improvement to this document are welcome. Please forward any comments to Email: Justin Finighan.

1 Why backup? Imagine arriving at work on Monday morning, and you find that a burst water pipe in the ceiling has poured water on your server, and the (only) external hard drive that was sitting on top. Both are beyond repair. The phone rings, and it is a client with a family law issue who needs an appointment. You cannot do a conflict check, so you refer them on. Because you have lost all your CLSIS data, you find yourself referring clients for the next 12 weeks, while the data is painstakingly re-entered from paper records (well, those that were not destroyed by the water leak). Even after 12 weeks of data re-entry, your problems are only just beginning. The data re-entry is impossible to complete to its original condition, and you are now being sued by one client for being out of time (your reminder system for limitation dates went the way of your CLSIS data), and another for breaching conflict of interest (some paper records were unreadable, so the other party data was lost). Of course, all the documents you had written on behalf of clients are lost too, and only some of the hard copies are useable after the good soaking experienced by your file cabinets. Your electronic employee records are lost, and you have to reconstruct all the QuickBooks accounts from a copy the bookkeeper made on a USB at the end of last financial year.

You get the picture. Without a thorough and tested backup regime, you risk bringing your centre to its knees.

With a good backup, you are up and running Tuesday morning, and the soaked paper files are your only nightmare. All you need is a telephone line, a computer, and a dry room, and you look like a CLC again.

2 Who is responsible for your backup? The short answer is “You are”. Many organisations assume that their IT service provider is taking care of their data backup. This is a dangerous assumption. From a technical point of view, your IT service provider will be responsible for setting up your backup systems. Responsibility for day-to-day management and monitoring of those systems must lie within the centre, and your IT service provider must provide the means for you to do this.

If you are going to make an external person or organisation responsible for your backup (for example your IT support company), then you should have a service agreement that specifies exactly the level of data security and monitoring you require. You will need to be prepared to pay a premium for such a service – it is a very big responsibility for an external organisation to take on, and it is an expensive service to provide. If you are not paying your IT people for such a service, then they are probably not providing it!

of 8 3 Why we lose data The causes of data loss are many and varied. Some of the more common ones are briefly described here. There are specific backup strategies that can be implemented to avoid each of these data loss scenarios.

3.1 Hardware failure This can include server failure, breakdown, failure of backup devices, networking equipment. If your server contains the only copy of your data, then total data loss for the organsiation is possible.

3.2 Software failure Server operating system failure, infection by viruses and malware, backup software failure, application software failure, database corruption.

3.3 The backup stops working This is a common cause of total data loss. In many instances, backup systems are tested a few times after initial setup, then simply assumed to keep working. Consistent backups require a number of systems to continue to work correctly. If any one of these breakdown, you can find yourself with a blank media set, or a media set that only contains old data.

3.4 Essential data not included in the backup Part of the setup of every backup system includes a specification of what should be backed up. In many cases, the backup selection list does not include all essential data. One example of this is the non-inclusion of the backup files created by the CLSIS backup process. This leads to a high risk that the CLSIS database will be unrecoverable in the event of a server failure. See “CLSIS Backup – a special case” below.

3.5 Theft Servers and backup devices, including media, can be stolen. Without off-site storage, all of your disaster recovery options can disappear in a single theft.

3.6 Fire Off-site storage is the only way to avoid total loss of all disaster recovery options. If you can trust an on-site fire proof safe, then you may have enough protection to do without off-site storage. Few commercial companies are satisfied with this level of risk.

3.7 User Error Probably the most common cause of data loss is accidental deletion. Such accidents can sometimes go undetected for some time, so maintaining sufficient backup history is essential.

3.8 Sabotage Deliberate unauthorised removal of data by employees or volunteers. You may consider this unlikely, but I can cite at least a dozen instances of this occurring in community organisations, including CLCs, over the last two years.

of 8 4 Key principles of a good backup and recovery strategy 4.1 Backup every day How often should you backup? The answer to this question is quickly given by asking another question. How many days’ data entry would you like to redo? Even a single day’s data re-entry would be difficult. If you have a good backup system in place, it is very little extra work to make it run every day.

Backup every day!

4.2 Spread your risk The aim of backing up your data is to spread your risk across multiple points of failure. If you have no backup, the risk of losing all your data is at least equal to the risk of the failure of a single drive. By copying your data to a single external hard drive, the risk of total data loss is at least equal to the risk of both drives failing at the same time, a very remote possibility.

On the surface, this looks like a sufficient strategy. Some centres have adopted the strategy of backing up to a single external hard drive. However, this is not as safe as it appears. A severe electrical fault can destroy both the server and the external hard drive. In the case of theft, both are likely to go. In addition, unless your external hard drive is of sufficient capacity, AND your backup software is configured to maintain data history, you cannot keep adequate data history in your backup.

The industry standard is to backup to multiple media, whether that be tape or hard drives. How many media? The most robust solution is to backup up to sufficient media to allow off-site backup of long term data, and to ensure that data history is maintained. Under the backup media rotation scheme discussed below, 16 tapes or drives are required to maintain full data history over a year, with one additional media required to be added per year.

4.3 Minimise down time In the event of total loss of your server, you need to be able to get up and running as quickly as possible, and at least cost. Backup of your server system data should allow for fast recovery of the entire server. If you only backup your data, and not your server system, then recovery can be a lengthy process. See “What to backup” below.

4.4 Vigilance Backup is not “set and forget”. In the same way that you would not just stop collecting the mail each day, you must view backup monitoring as a daily task, even if you have not seen an error for months. Very often, backup systems that were setup months ago, and were working perfectly, have begun to fail. A common story describes how the staff diligently replaced the backup media every day, kept copies off-site, only to find that the backup media is all blank, and the last successful backup was 6 months ago. Backup functionality must be checked daily.

4.5 Maintain data history It is essential to be able to go back in time, to be able to access your data as it was some time in the past. Data deletions may not be noticed for months, database corruptions can take a long time to become apparent. Legal and accounting records may need to be viewed as they were many months ago.

Keeping a month’s history is easy, and keeping history forever is a simple extension of the same concepts. At first, this may seem to be impracticable or overly expensive. However, by applying the principle that the further you go back, the fewer copies of data you require, data history can be maintained with very little additional cost per annum. See the section “Backup Media Rotation Schedule” for a method of keeping data history for years.

of 8 4.6 Prove your backup The adage is “you don’t have a backup until you have done a restore”. Unless you can actually get your data back, your backup is useless. You need to regularly check the integrity of your backup by performing a restore. This should be checked at least quarterly, and built into your regular IT maintenance schedule.

To test your restore, choose any file at random, move it to a different folder, then restore that file from the backup (you may need to ask your IT provider to do this for you – it will be a good test of the system they have setup for you).

4.7 Don’t skimp on your investment A good backup system is expensive, but much cheaper than data loss. Make sure you adequately invest in a good quality backup system that will cover you for all the scenarios outlined above. You can expect a solid backup system to require around $1,800 to $3,000 to implement, maybe a little more for large centres. If you are being quoted much more than this, you may be paying for something you don’t need!

Backup software can be expensive, but like much other software that CLCs use, very cheap licences can be obtained through Donortec (see www.donortec.com.au for details).

5 What data should be backed up? 5.1 Working data Your working data includes documents, databases, CLSIS backup files, accounting data and all other data that you accumulate as part of your day-to-day business. As a minimum, all this data must be included in your backup software selection list.

5.2 System data System data includes your server operating system(s). This data is required to quickly restore your systems in the event of the failure or loss of the server. If system data is not available for restoring failed servers, the return to normal network operation can be lengthy and costly.

Many CLCs have no system software backup. This is usually because system data is large, and centres have insufficient storage capacity to back up entire server systems. Until recently, the cost of backup media with sufficient capacity to store system data made system backups expensive to setup. Now, with the low cost of external hard drives, full system backup is achievable for all centres.

6 CLSIS Backup – a special case CLSIS backup requires special attention. Many CLCs have been advised by their IT service providers that a separate procedure for CLSIS backup is unnecessary because they have ensured that the live CLSIS data is included in the daily backup of the server data. Some IT people have even turned off the internal warning in CLSIS so that the annoying “system required backup” message does not appear.

This advice is incorrect.

The ONLY safe way to backup CLSIS is to run the CLSIS backup process from within CLSIS itself. This process creates a set of files that should then be included in the daily backup selection list. If this procedure is not followed, there is a significant risk that the CLSIS database will not be successfully restored.

of 8 If you see the “System requires backup” message when you start CLSIS, then your CLSIS data is NOT being backed up correctly. Staff at many CLCs admit to seeing that message “all the time”. It is likely that some CLCs have not had a proper backup of CLSIS for months, or even years. Remember, no CLSIS means no conflict checks, which means no service from your CLC until all the data is re-entered from paper files!

7 What do you need to do? 7.1 Ask your IT provider to comprehensively document your data backup and disaster reco very It is essential that you have a record of all aspects of your data backup system. This should include:

 Backup software being used

 Backup hardware specification

 The backup selection list

 Media rotation scheme being used

 Data restore procedures

 Disaster Recovery Plan – how your systems will be recovered in the event of hardware failure or total loss of servers.

7.2 Have your IT provider show you how to check your backup each day Regardless of whether or not your IT service provider is monitoring your backup, this is a task that you must undertake within your centre. It is simply too important to delegate.

Backup should be checked at the same time as the backup media is changed. See “Media change record” below for a sample form you can use to record backup checking and error reporting.

Your IT service provider can setup a simple meted of verifying the data backup each day. This should include instructions about how to identify errors, and when you should call IT support to rectify a backup problem.

7.3 Use a media rotation schedule A good media rotation schedule is essential to prevent data loss due to accidental deletion, or data corruption errors that do not become apparent for a period of time. It is also important to be able to see what the data looked like at some time in the past. For example, it may become apparent that a client has been deleted from your CLSIS database, and you need to retrieve details of that client from an old version of the database. If you are maintaining history through your backups, you will be able to see how the data looked prior to the client being deleted.

The Backup Media Rotation Schedule described at the end of this document will enable you to maintain a data history for the whole year using 16 tapes or disk drives. Each additional year of history will only require one additional tape of hard drive.

As the size of your data increases, you will find that external hard drives are a much cheaper and more practical solution than going to large capacity tape drives. Hard drive prices are now below $100 for 1TB capacity (December 2010). A tape drive capable of providing similar capacity will cost at least $2,500 plus tape media.

of 8 7.4 Use a cloud or on-line backup service Cloud computing is the big buzz in IT at the moment. Essentially cloud computing enables you to purchase computing services and resources, rather than purchasing the hardware and infrastructure. For example, if you want to use email, you purchase a service or facility that provides email, rather than purchasing and configuring your own physical server. Common characteristics of cloud based computing include high degree of accessibility, high levels of security and availability, and in most cases, you pay a regular amount of subscription to use the services.

There are many cloud backup solutions available, and some of them are very low cost. Keep in mind that while cloud backup tends to be very good for your documents, multi-media files and even your CLSIS backup files, it is not so good for backing up your server system files and large databases such as Microsoft Exchange. You will almost certainly need to maintain a local backup solution for that data so that you can restore your server.

7.5 Test your backup regularly On a regular basis (at least quarterly) you need to test your backup by performing a restore. The procedure should be as follows:

 Choose any file on your server, and move it to a temporary location, for example, your local desktop.

 If you know how to restore data using your backup your system, attempt to restore that file back to its original directory. If necessary, have your IT provider assist with this. If the file can be restored, your backup is probably working well.

 To test backup history maintenance, create a test file, and save it, making a note of the filename and location. Wait a couple of days. Delete the file, then wait three months. Ask your IT provider to restore that file. If they can get it back, then your backup history is probably working well. In theory, if you are using the backup rotation scheme described below, you should be able to get that file back after a year or more.

of 8 8 Backup Media Rotation Schedule The backup regime follows a daily/weekly/monthly/quarterly rotation cycle. Differential backups of all data and system files are performed daily, and full backups are performed weekly.

Following financial years, tapes are marked and utilised as follows:

Tape Label Usage Storage Monday Re-used each week Tuesday Re-used each week Keep on site Wednesday Re-used each week Thursday 1 Re-used on the first Thursday of each month Keep off site Thursday 2 Re-used on the second Thursday of each month until the week Thursday 3 Re-used on the third Thursday of each month they are due to Thursday 4 Re-used on the fourth Thursday of each month that has 5 Thursdays be used. Friday Re-used each week Keep on site M1/7 Used on the last Thursday in January and re-used on the last Thursday in July M2/8 Used on the last Thursday in February and re-used on the last Thursday in August Keep off site until the week M4/10 Used on the last Thursday of April and re-used on the last Thursday in October they are due to M5/11 Used on the last Thursday of May and re-used on the last Thursday in November be used. Y09/10 Used on the 30th of June (regardless of day – closest day if not running backup Consider using (for June 30th 7 days.) and kept for 5 years. Next year’s tape would be Y10/11 etc. If you are safe deposit box 2010) using differential backups on days other than Thursdays, backup software must for long term be manually configured to do a full backup on June 30. storage (Qx and Q1 Re-used on the last Thursday of September each year Yxx/xx tapes) Q2 Re-used on the last Thursday in December each year Q3 Re-used on the last Thursday of March each year (3rd quarter of financial year)

If you ran this scheme throughout the 2010/11 financial year, as of 30 June 2011 you will have the following tapes in your set (in order of age):

Q1, Q2, M1/7, M2/8, Q3, M4/10, M5/11, Thursday 1, Thursday 2, Thursday 3, Thursday 4, Friday, Monday, Tuesday, Wednesday, Y10/11

Note that to maximise tape life, daily and weekly tapes should be re-labelled and used as long term storage and replaced with new tapes. For example, when you need to create your Q1 tape, use one of your daily tapes, and replace that tape with a new one. Keep a log of when each tape was first put into use.

of 8 9 Media Change Record (sample for December 2010, weekday backup only)

Backup Log Errors Date Day Tape Label Support Contacted Signed Yes No 1/12/2010 Wednesday Wednesday 2/12/2010 Thursday Thursday 1 3/12/2010 Friday Friday 6/12/2010 Monday Monday 7/12/2010 Tuesday Tuesday 8/12/2010 Wednesday Wednesday 9/12/2010 Thursday Thursday 2 10/12/2010 Friday Friday 13/12/2010 Monday Monday 14/12/2010 Tuesday Tuesday 15/12/2010 Wednesday Wednesday 16/12/2010 Thursday Thursday 3 17/12/2010 Friday Friday 20/12/2010 Monday Monday 21/12/2010 Tuesday Tuesday 22/12/2010 Wednesday Wednesday 23/12/2010 Thursday Thursday 4 24/12/2010 Friday Friday 27/12/2010 Monday Monday 28/12/2010 Tuesday Tuesday 29/12/2010 Wednesday Wednesday 30/12/2010 Thursday M4/10 31/12/2010 Friday Friday