Raidix & Xsan. a Data Storage Solution Best Practices For
Total Page:16
File Type:pdf, Size:1020Kb
RAIDIX & XSAN. A DATA STORAGE SOLUTION BEST PRACTICES FOR MEDIA POSTPRODUCTION ABSTRACT CONTENTS The RAIDIX–Xsan whitepaper addresses media Introduction 1 postproduction professionals looking to build a high performance and high availability data storage system Video processing solutions for postproduction 3 in the Apple environment. RAIDIX engineers offer a solution to ensure shared data access from multiple Components 3 video editing workstations and maintain sustainable high speed of multi-thread data processing. Solution architecture 5 Recommendations 6 Bundled with Apple's Xsan FS, RAIDIX Media delivers a complex data storage platform for the media Configuring ATTO FC Thunderbolt 6 industry. Check out the hardware recommendations, Disk recommendations 6 fine-tuning tips and architecture models in this article. Dedicated Ethernet subnetwork for MDC 6 One LUN for one RAID 6 RAID configuration 7 About RAIDIX 8 INTRODUCTION RAIDIX Media is intended for creating high-performance storage systems in Media & Entertainment, based on standard hardware components. RAIDIX perfectly matches the strict requirements of high performance, reliability and operational continuity owing to proprietary mathematical algorithms and use of parallel calculations. RAIDIX supports one-controller (single node) and two-controller (Active-Active) system operation modes. In the two-controller mode both nodes are active, operate concurrently and have access to a single array of disks. By nodes we mean hardware-independent components of data storage systems, having their own processors, cache memory, motherboards — and poolable into a cluster. All rights reserved. © RAIDIX, 2016 RAIDIX & XSAN. A DATA STORAGE SOLUTION CONFIGURATION GUIDE BEST PRACTICES FOR MEDIA POSTPRODUCTION One of the most promising market segments using multi-thread data processing is video postproduction (editing, video processing). The majority of media studios use Apple equipment and a standard professional software kit: Apple Final Cut Pro, Adobe Premiere Pro, Autodesk Discreet Smoke, DaVinci Resolve and other. An ever-growing volume of information and use of resource-intensive software in postproduction imply growing demand for high performance data storage systems with record data availability. At the same time, modern video processing requires the same data being available to several editing workstations with various functions. Other media production needs include: 1. High-speed access, which implies a local 2. Clustered shared access, which implies the use storage and high data availability of a file system with shared file access from multiple workstations. The solution for these seemingly incompatible tasks is offered by Apple’s Xsan file system. Xsan is Apple's 64-bit cluster FS for Mac OS X or Mac OS X Server. The system enables quick access to large data volumes. Xsan allows shared use of files and volumes of up to 16 TB each, in a high-speed optical network. Xsan provides high availability along with a near-zero failure rate, and allows for the use of SAN network for data consolidation — as a viable alternative to NAS- devices. Apple-powered media studios will find in this document the essential infrastructure tips, equipment recommendations, guidelines for fine-tuning and configuration of high performance data storage systems. All rights reserved. © RAIDIX, 2016 2 RAIDIX & XSAN. A DATA STORAGE SOLUTION CONFIGURATION GUIDE BEST PRACTICES FOR MEDIA POSTPRODUCTION VIDEO PROCESSING SOLUTIONS FOR POSTPRODUCTION The critical metrics of data storage in the media system is intended for sharing resources industry include: between multiple client machines. Volume of stored data The bundle of a RAIDIX-powered data storage system and the clustered Apple Xsan FS meets Budget the media studio requirements for high speed and data availability. The throughput delivered Performance by this solution amounts to four streams of un- compressed 4K video. Availability Key to building such systems is the right choice The clustered file system should allow a picture of components, as well as thorough network editing specialist to access any file from any client and data storage configuration. workstation. Xsan addresses the postproduction challenges by providing quick access to sequen- Read on for hardware, architecture and fine- tial data. By the same token, the Apple Xsan file tuning recommendations. COMPONENTS Data storage system. Builds on a one-controller server SM 6048R- E1CR36N with RAIDIX software installed. Configuration: FC 16Gb + JBOD 847E1C-R1K28JBOD. The system supports SATA, SAS and NL-SAS disks. Metadata controller. To ensure high availability, it's recommended to use one or preferably two Apple Mac Mini devices as metadata controllers (MDC). Client workstation. The Apple Mac Pro server is an optimal choice for a workstation. All rights reserved. © RAIDIX, 2016 3 RAIDIX & XSAN. A DATA STORAGE SOLUTION CONFIGURATION GUIDE BEST PRACTICES FOR MEDIA POSTPRODUCTION Fibre Channel/Thunderbolt. Since Mac mini and Mac Pro do not support FC adapters. In order to integrate them with a SAN network, it's recommended to use ATTO ThunderLink FC 2082 for Mac Mini, and ATTO ThunderLink FC 2162 for Mac Pro. Xsan clustered system enables shared access to video material from any workstation. Install and register on Apple Mac Mini prior to configuration. Fibre Channel (FC) switch. Given a limited number of client machines (up to 20), the connection between FC adapters can be performed directly: Data Storage System to MDC and the clients. In this case, an FC switch is not required. For deploying SAN networks with a bigger number of clients, it's necessary to use at least one FC switch. Commutation. The majority of components are connected through an optical fiber cable with LC-LC connectors, and plugged directly into ATTO FC adapters, FC Thunderbolt, FC switches, client workstations and the data storage system. Note! The optic fiber cables are connected to FC adapters in each port through SFP transceivers. Most FC adapters include SFP transceivers as part of the package. However, some commutators are shipped without transceivers, so it's highly recommended to clarify the list of equipment prior to purchase. Exception from the common commutation model, the connection between Apple Mac Mini and Fiber Channel Thunderbolt is performed through a native Thunderbolt cable. MDC servers and all client workstations should be connected with an ETH channel (network) for metadata. All rights reserved. © RAIDIX, 2016 4 RAIDIX & XSAN. A DATA STORAGE SOLUTION CONFIGURATION GUIDE BEST PRACTICES FOR MEDIA POSTPRODUCTION SOLUTION ARCHITECTURE THE SCHEME OF COMMUTATION FOR BUILDING HIGH AVAILABILITY DATA STORAGE SYSTEMS IN MEDIA PRODUCTION Apple Mac Pro Apple Mac Pro Apple Mac Pro Thunderbolt Thunderbolt Thunderbolt Atto Thunderlink 2162 Atto Thunderlink 2162 Atto Thunderlink 2082 Atto Thunderlink 2082 Fiber Channel Switch Thunderbolt Thunderbolt Apple Mac Mini (MDC) RAIDIX Storage Apple Mac Mini (MDC) All rights reserved. © RAIDIX, 2016 5 RAIDIX & XSAN. A DATA STORAGE SOLUTION CONFIGURATION GUIDE BEST PRACTICES FOR MEDIA POSTPRODUCTION RECOMMENDATIONS Below you will find recommendations for configuring and fine-tuning data storage systems for ensuring high performance and reliability. CONFIGURING ATTO FC THUNDERBOLT On the client side, it's recommended to use the latest versions of firmware and drivers. Default settings for FC Thunderbolt are advisable for most cases. All settings (including reset to default settings) are made with an ATTO ConfigTool utility. DISK RECOMMENDATIONS RAIDIX recommends using NL SAS 7K disks. These disks have a better MB/$ parameters and ensure high density of storage and reliability. DEDICATED ETHERNET SUBNETWORK FOR MDC Every Apple Mac Pro workstation is shipped with at least two Ethernet 1G ports, embedded into the motherboard. One of these ports should be connected to a dedicated Xsan subnetwork, and used for primary trafficking of Xsan metadata. This connection method will ensure high throughput for Xsan metadata and minimize latency on data delivery. Apple Mac Mini, deployed as a metadata controller, has one Ethernet port. It's recommended that this port be used in a dedicated network with other clients. This configuration restricts access to Mac Mini from outside the dedicated subnetwork. ONE LUN FOR ONE RAID Since sequential bandwidth is a critical metric in the video processing environment, it's advisable to minimize the number of search operations on the disks. In this regard, RAIDIX recommends to create one LUN for the entire RAID array. Besides, the Xsan volume should be created from block devices of a single data storage system. All rights reserved. © RAIDIX, 2016 6 RAIDIX & XSAN. A DATA STORAGE SOLUTION CONFIGURATION GUIDE BEST PRACTICES FOR MEDIA POSTPRODUCTION RECOMMENDATIONS RAID CONFIGURATION RAIDIX achieves high-speed sequential access and disk space optimization by using parallel calculations and unique mathematical algorithms. Besides, the solution builds on a proprietary high reliability RAID 7.3 with triple parity. In 7.3, three checksums are calculated using various algorithms, and three disks are allocated for checksums. To ensure higher throughput, it's necessary to create four initialized RAID 7.3 arrays of 18 disks each (2 arrays are the node-placed disks, 2 others are JBOD disks). RAIDIX recommends to allocate a RAM cache of 12 GB and higher — for each RAID array. The remaining 8 disks should be evenly distributed across the spare pool: two disks for each RAID array. For the volume containing