Optimal RAID Configurations for Video Surveillance

WHITE PAPER: Optimal RAID Configurations for Video Surveillance

Today’s video surveillance systems now include a higher number of cameras continuously capturing video content at higher resolutions. In most cases, this video content is captured for the what-if scenarios and the post-incident review and analysis that follows. Whether it is for this forensic-type analysis, liability protection, identification of those who have committed crimes, or simply a matter of regulatory compliance, video data is extremely valuable in many situations.

The nature of high-resolution 24x7 surveillance systems means that there is an enormous amount of data being generated on a continual basis. The constant flow of data being written to data storage systems introduces a higher level of risk of disk drive failure or other types of critical read/write failures. As a result of these factors, many leading storage hardware manufacturers are now providing guidance on best practices for storing video data – and limiting the types of RAID levels employed.

This paper looks at the different ways to protect video surveillance systems from data loss through the use of storage technologies, specifically RAID, and identifies those which are best for storing critical video data.

Storing Critical Video Data

The probability of disk failure increases disk drives being exposed to the same types of exponentially as systems age. Since most video conditions, operating characteristics and aging surveillance systems are installed with the intention within the same system. of using the system for as many years as possible, the likelihood of drive failures is higher than most For those tasked with designing and operating other IT systems. An appropriate data protection video surveillance systems as well as end users, configuration is required from day one for systems protecting video data is often as important as that record critical video data. protecting critical financial data. Appropriate systems must be designed to ensure data protection, Statistics show that the probability of a second while also maintaining required performance drive failing before the first has been recovered, levels. For mission critical systems and valuable is higher than the probability of random drive video data, operators will incorporate appliances failures. This is attributed to the longer rebuild with higher storage capacities and enough drives times due to the larger disk capacities typically to provide data protection through the use of RAID. being used today. It is also due in part to the

Razberi Technologies | White Paper | 1 What is RAID If a drive fails, the missing data is recalculated from the checksum, providing the data redundancy RAID (Redundant Array of Independent Disks) is aspect of RAID. This method benefits from the a technology used to increase the performance increased performance from striping the data. and/or reliability of data storage. A RAID system All the existing RAID types are based on striping, consists of two or more drives working in parallel mirroring, parity, or combination of these storage in an array. These disks can be hard discs, but techniques. there is a trend to also use the technology for SSD (solid state drives). Data is distributed across The software to perform the RAID-functionality the drives in one of several ways, referred to and control the drives can either be located on as RAID levels, depending on the required level a separate controller card (a hardware RAID of redundancy and performance. RAID levels controller) such as that used within a Razberi are optimized for specific situations. There were appliance, or it can simply be a driver. Some originally five RAID levels, but many variations versions of Windows or Mac OS X include software have evolved. The most common RAID levels RAID functionality. Hardware RAID controllers may (0/1/5/6/10) are described in this paper. cost a little more than a software option, but they provide better performance, especially with RAID 5 Each RAID level provides a different balance of and 6. specific operational goals: reliability, availability, performance, and capacity. RAID levels RAID Levels greater than RAID 0 provide protection against unrecoverable read errors, as well as against entire RAID 0 - STRIPING drive failures. In a RAID 0 system data is split up in blocks that get written across all the drives in the array. To understand the different RAID levels, it is By using multiple disks (at least 2) at the same important to first understand the different methods time, this offers superior I/O performance. This of storing data that are applied with RAID: striping, performance can be enhanced further by using mirroring, and parity. multiple controllers, ideally one controller per disk.

Striping – this method involves splitting the flow Advantages of data into blocks and then writing of these blocks • RAID 0 offers great performance, both in read across the RAID one by one onto different drives. and write operations. There is no overhead This method of data storage positively affects with this method meaning that the amount of performance, allowing for the concurrent reading available storage equals the sum of all drives’ of data across the different disks, increasing total raw storage capacity. data throughput. Striping effectively uses the total • All storage capacity is used, there is no storage capacity of the array. overhead. • The technology is easy to implement. Mirroring – this storage method stores identical Disadvantages copies of data on different disks simultaneously. • RAID 0 is not fault-tolerant. If one drive fails, Mirroring provides good performance on writing all data in the RAID 0 array are lost. It should data – similar to a single disk, and can be used not be used for mission-critical systems. for concurrent reads across the array resulting in better read performance. Appropriate Applications RAID 0 is ideal for non-critical storage of high Parity – this method utilizes striping and bit rate video data (high resolution cameras with checksum methods. Data is written via striping high frame rates & quality settings). RAID 0 is across the array with the added benefit of utilizing not recommended for critical video surveillance a parity disk to store the checksum information. recordings that must be kept for regulatory reasons.

Razberi Technologies | White Paper | 2 RAID Levels (Continued) losing data or access to data. Although RAID 5 can be achieved in software, a hardware controller RAID 1 - MIRRORING is recommended. Often extra cache memory is Data is stored twice by writing to both a data drive used on these controllers to improve the write (or set of data drives) and a mirror drive (or set performance. of drives). If a drive fails, the RAID controller uses Advantages either the data drive or the mirror drive for data • Read data transactions are very fast while write recovery and continues operation. You need at data transactions are somewhat slower (due least 2 drives for a RAID 1 array. to the parity that has to be calculated). This is Advantages important for video surveillance applications • RAID 1 offers excellent read speed and a write- which have read-intense workloads. However, speed that is comparable to that of a single most surveillance workloads have less than a drive. 10% read requirement (i.e. the storage array • In case a drive fails, data do not have to be is typically used more than 90% of the time for rebuilt, they just have to be copied to the writing data). replacement drive. • If a drive fails, you still have access to all data, • RAID 1 is a very simple technology. even while the failed drive is being replaced and the storage controller rebuilds the data on Disadvantages the new drive. • The main disadvantage is that the effective storage capacity is only half of the total drive Disadvantages capacity because all data get written twice. • Drive failures have an effect on throughput, • Actual read throughput of most RAID 1 although this is still acceptable. implementations is slower than the fastest drive. • If one of the disks in an array using large Write throughput is always slower because disks fails and is replaced, restoring the data every drive must be updated, and the slowest (the rebuild time) may take a day or longer, drive limits the write performance. depending on the load on the array and the speed of the controller. If another disk goes bad Appropriate Applications during that time, data is lost forever. RAID 1 is suitable for mission critical storage, such • If the system encounters an unrecoverable read as video storage for casinos, city surveillance or error (URE) during the rebuild of a failed drive, other applications which are required to store data the rebuild will fail and the data is ultimately due to regulatory reasons. It is also suitable for lost. UREs used to be an acceptable problem, small servers in which only two data drives will be but as drive capacity has increased, the odds used. of encountering a URE on a large video storage drive have increased to unacceptable levels. It RAID 5 - STRIPING WITH PARITY is now not only possible, but quite likely, that a RAID 5 is the most common secure RAID level RAID 5 array will fail to rebuild and therefore for data center applications, but is not a best offer no protection against data loss. practice for video storage. It requires at least 3 Appropriate Applications drives but can work with up to 16. Data blocks RAID 5 is not recommended for mission critical are striped across the drives and on one drive a data such as high-value video surveillance footage parity checksum of all the block data is written. The as it will not provide enough redundancy and parity data are not written to a fixed drive, they data protection. Additionally, this RAID level is not are spread across all drives. Using the parity data, suitable for “live” video storage as it may not be the computer can recalculate the data of one of able to handle the higher bit rate and frequency, the other data blocks, should that data no longer particularly with megapixel camera streams. As be available. That means a RAID 5 array can, in mentioned previously, major storage vendors such theory, withstand a single drive failure without as Dell have actually issued advisories against

Razberi Technologies | White Paper | 3 single drive failure without losing data or access Appropriate Applications using RAID 5 for any mission critical applications RAID 6 is a good all-round system that combines or high value data storage. Dell’s web site efficient storage with excellent security and decent provides TR1020 Member RAID position advisory performance. It is preferable over RAID 5 for information. critical video surveillance systems that use many large drives for data storage. However, this RAID RAID 5 might be an adequate solution for archiving level may not suitable for “live” video storage video data on archival storage arrays since the without a high performance hardware RAID value of archived video is often less and potentially controller as low performance devices will not be used in conjunction with primary storage hardware able to handle the higher bit rate and frequency, such as the Razberi ServerSwitchIQ appliance. particularly with megapixel camera streams.

RAID 6 - STRIPING WITH DOUBLE PARITY RAID 6 is similar to RAID 5, but at this level, the parity data is written to two drives. That means it requires at least 4 drives and can withstand 2 RAID 6 is a good all-round drives dying simultaneously. The chances that two system that combines efficient drives break down at exactly the same moment are, storage with excellent security of course, very small. However, if a drive in a RAID and decent performance. It 5 systems dies and is replaced by a new drive, it is preferable over RAID 5 takes hours to rebuild the swapped drive. If another for critical video surveillance drive dies during that time or the rebuild runs into systems that use many large unexpected errors or failures, you still lose all of your data. With RAID 6, the RAID array will drives for data storage. even survive that second failure and make rebuild process much more robust. Advantages • Like with RAID 5, read data transactions are RAID 10 – COMBINATION OF very fast. RAID 1 & RAID 0 • If two drives fail, you still have access to all It is possible to combine the advantages (and data, even while the failed drives are being disadvantages) of RAID 0 and RAID 1 in one single replaced. So RAID 6 is more secure than RAID system. This is called a nested or hybrid RAID 5. This also means that the URE problem noted configuration. It provides security by mirroring in the RAID 5 configuration is not a problem all data on secondary drives while using striping in RAID 6 since failure would require two URE across each set of drives to speed up data events, a very unlikely occurrence. transfers. This method creates a striped set from • RAID 6 makes larger RAID groups more a series of mirrored drives. The array can sustain practical, especially for high-availability multiple drive losses so long as no mirror loses all systems, as large-capacity drives take longer to its drives. restore. Advantages Disadvantages • If something goes wrong with one of the disks • Write data transactions are slowed down due in a RAID 10 configuration, the rebuild time to the parity that has to be calculated. is very fast since all that is needed is copying • As with RAID 5, drive failures have a negative all the data from the surviving mirror to a new effect on throughput. drive. As an example, the rebuild time could • Rebuilding an array in which one drive failed take as little as 30 minutes for drives of 1 TB. can take a long time.

Razberi Technologies | White Paper | 4 Disadvantages Summary & Recommendations • Only 50% of the total raw capacity of the drives is usable due to mirroring, so compared • Critical video surveillance applications – use to large RAID 5 or RAID 6 arrays, this is an RAID 6 or RAID 10 expensive way to have redundancy. • Non-critical video surveillance applications – • As long as one disk in each mirrored pair is use RAID 0 or RAID 1 functional, data can be retrieved. However, if • RAID 5 should not be used for video two disks in the same mirrored pair fail, all surveillance data storage data will be lost because there is no parity in the striped sets. Choosing the appropriate level of RAID depends Appropriate Applications on how critical the data is that you are storing. If Given that RAID 10 is superior to other RAID you are required by law to keep video data for a levels in terms of write performance and reliability certain period of time, then there is good reason (particularly with 4 or more disks) it is ideal for to employ RAID level 6 or 10. If you are capturing applications which require a high level of write video data that could potentially be used in legal performance such as video surveillance systems liability cases such as slip and falls, then there running today’s megapixel cameras. Additionally, could also be good reason to employ RAID levels the higher level of redundancy associated with that provide greater protection. In addition to the RAID 10 makes it an ideal option for the cases level of protection, you must consider the effect of where high data redundancy is required. Since RAID on your system performance and utilization the cost of storage has dropped considerably of storage capacity. The table on the next page and, instead of being at a premium, is typically helps summarize and compare the various levels of in abundance today and because of the higher RAID to determine which is best for your video data performance characteristics, the move to RAID 10 storage purposes. has become a more standard approach for video surveillance.

Beyond RAID

All RAID levels except RAID 0 offer protection from a single drive failure. As mentioned, a RAID 6 Choosing the appropriate level system even survives 2 disks dying simultaneously. of RAID depends on how critical For complete security you do still need to back-up the data is that you are storing. the data from a RAID system. • That back-up will come in handy if all drives fail simultaneously because of a power spike. • It is a safeguard when the storage system gets stolen or damaged. Back-ups can be kept off- Razberi recommends RAID 6 or 10 for all critical site at a different location. This can come in video surveillance applications and RAID 0 or 1 for handy if a natural disaster or fire destroys your non-critical applications. The limitations and the workplace. higher level of risk associated with RAID 5 should • The most important reason to back-up multiple disqualify that RAID level from most if not all video generations of data is user error. If someone surveillance systems. accidentally deletes some important video evidence and this goes unnoticed for some period of time, a good set of back-ups ensure you can still retrieve those files.

Razberi Technologies | White Paper | 5 Summary and Comparison of Various RAID Levels

Features RAID 0 RAID 1 RAID 5 RAID 6 RAID 10

Storage Striping with Mirrored Striping Mirror Striping with Parity Method Double Parity Striping Minimum # 2 2 3 4 4 Drives Up to one disk Data Single-drive None Single-drive failure Two-drive failure failure in each Protection failure sub-array Read High High High High High Performance Write High Medium Low Low High Perfomance Storage Capacity 100% 50% 67-94% 50-88% 50% Utilization (% of Raw) Not recommended Used for critical Not recommended Recommended for Recommended for critical video video storage for critical video critical video for critical video storage High R/W for Low write speed Low write speed Provides the Highest R/W large bit rate performance performance best combina- for large bit rate recording & tion of speed & recording & play- playback Long rebuild times Long rebuild times redundancy back Some redun- Provides some Provides Low utilization Maximum dancy but at a redundancy redundancy of available storage utilization cost of half the storage for but provides no total available Offers more stor- Offers more recording Summary and redundancy storage age for video storage for video Applications Best used in Best used in ar- archiving servers chiving servers

Not recommend- Not recommended for use w/live ed for use w/live data base due to data base due to URE rebuild issue, size of megapixel size of megapixel bit rates, 24x7 bit rates, 24x7 requirement of sur- requirement of surveillance & rebuild veillance & rebuild times times

Razberi Technologies | White Paper | 6 Hardware Recommendations

Appropriate hardware for the capture and storage of video surveillance footage include the Razberi ServerSwitchIQ Pro & ServerSwitchIQ Enterprise appliances. These platforms provide superior performance to cost of storage, based on the number of cameras that can be connected.

Razberi ServerSwitchIQ 16/24 Port 1U Pro Models – These models use the Intel® Rapid Storage Technology driver application found on the Start Menu for configuring RAID. Razberi Technologies recommends reviewing the latest documentation covering this tool from the http:// www.intel.com web site.

Razberi ServerSwitchIQ 24 Port 2U Enterprise Models – These models use the High Point 45XX series hardware for controlling up to eight hard disk drives. Razberi Technologies recommends reviewing the latest documentation covering the RocketRAID 45XX series from the http://www.highpoint-tech.com web site.

Contact Razberi Technologies today to request a demonstration and discover what the intelligent surveillance appliance can do for you at: www.razberi.net

Razberi is the leading provider of intelligent surveillance appliances. Razberi ServerSwitchIQ™ is a complete line of all-in-one, scalable appliances that combine a managed PoE switch, server, storage and intelligence. All Razberi appliances are open-architected to work with world class video management systems (VMS) out of the box. Intelligent features include VyneWatch™ health monitoring and LocBeri™ cyber security features to lock down vulnerable ports. By deploying Razberi appliances in a distributed architecture near the edge of the network, megapixel video impact on the network is reduced by up to 95%. Learn more at: www.razberi.net

Americas: 469-828-3380 UK/EMEA: 0203 773 3689 [email protected] wpre20161005