Using Disks in Backup Environment and Virtual Tape Library (VTL) Implementation Models Emin ÇALIKLI

Total Page:16

File Type:pdf, Size:1020Kb

Using Disks in Backup Environment and Virtual Tape Library (VTL) Implementation Models Emin ÇALIKLI Using Disks in Backup Environment and Virtual Tape Library (VTL) Implementation Models Emin ÇALIKLI EMC Proven Profesional Knowledge Sharing 2009 Emin ÇALIKLI Senior Technical Consultant Gantek Technologies – Turkey [email protected] Table of Contents Introduction .......................................................................................................................... 3 Do we really know the exact meaning of RTO & RPO?....................................................... 3 What do you expect from disks?.......................................................................................... 4 I/O pattern difference between OLTP Systems and Backup Systems................................. 6 Disk Space Utilization Problems on Every Tier.................................................................... 8 VTL implementation types.................................................................................................... 9 Summary............................................................................................................................ 13 References;........................................................................................................................ 13 Biography........................................................................................................................... 14 Disclaimer: The views, processes or methodologies published in this compilation are those of the authors. They do not necessarily reflect EMC Corporation’s views, processes, or methodologies Introduction IT professionals have compared storing data on tape and disk for many years; the demands remain the same. Most believe that disk storage will overcome tape technology, and we will use disks to meet all of our storage requirements. It is difficult to meet long term data retention needs with disk solutions due to cost. Still, a disk is less expensive than tape. Of course, it depends on your point of view and your application characteristics. Many learn about backup only when they need a consistent restore. We live in a digital world, and we have to protect critical information in every way possible. Do we really know the exact meaning of RTO & RPO? Unless we discover a time machine, it is impossible to predict disasters. If a disaster is possible, it is only a matter of time before it happens. Data Protection is a crucial service requiring high level commitment between the business and IT. There are two important terms to identify the “money loss rate” caused by a disaster. These are Recovery Point Objectives (RPO) and Recovery Time Objectives (RTO). RPO represents “data loss tolerance” and RTO represents “how can I return my last consistent state in a given time?” The following picture represents a typical disaster flow: Usually, people focus on the RPO side and calculate their loss using RPO values. This is not sufficient. For me, RTO is the most important process for calculating monetary loss. I propose this basic formula: Amount of Loss = VLD + LTC + OC VLD (Value of Lost Data): The cost of data which is already processed until the disaster strikes. The loss rate depends on time rather than the amount of data. LTC (Loss Transaction Cost): The cost of missed data that could not be handled during the recovery process. (Ex: Customer transactions, deal processes, etc.) OC (Other Costs): Related Systems Downtime Costs, Operational Costs, Hidden Costs The ‘realize and determine’ processes make it impossible to recover the systems(s) in “0” seconds. If the subject is “RPO” we can approach “0” seconds using synchronous replication, Continuous Data Protection, Mirroring etc. Let’s use the following picture to review data protection levels. There are many data protection methods; usage behavior depends on your application availability requirements. What do you expect from disks? The largest effort is to obtain maximum throughput from the system on every tier. Fast response means fast data delivery; fast data delivery means satisfied customers. No one likes to wait in any type of queue; this is the key consideration for performance management. Disks are the only the single storage solution that we can use effectively for online applications. Memory and Solid State Disk (SSD) devices are expensive and have limited capacity. Expectations from disks include: 1- Throughput (IOPS) 2- Capacity (GB) 3- Bandwidth (MB/s) This order represents OLTP systems requirements from disk systems. Also, disks are designed for random access I/O types and are capable, if designed correctly, of obtaining sequential access requirements. We are usually combining disks to achieve improved performance and a reliable storage environment. Disks are not appropriate devices for long-term data storage purposes as they are more expensive than tapes. Furthermore, disks are very sensitive to physical shocks. If we are talking about tape, our priorities should be: 1- Bandwidth (MB/s) 2- Capacity (GB) 3- Throughput (IOPS) These requirements are the opposite of disk requirements. Tapes are not designed for random access operations; they are designed for sequential access. Backup operations require high bandwidth and sequential access. Also, tapes are the correct devices for long-term data retention needs, because no one expects rapid file delivery so they do not need to spin like disks. Tapes are very useful for backup and archive operations due to cost and performance. I/O pattern difference between OLTP Systems and Backup Systems Let’s see how the “Throughput (IOPS)” and “Bandwidth (MB/s)” performance are changing against I/O block size changes. This is a conceptual comparison of Throughput and Bandwidth; the graph lines are not as smooth in practice. This illustration tells us a lot about disk usage behavior in every tier of IT. Small Block Size means less bandwidth usage and high throughput; larger block size means high bandwidth and low throughput. This is why we have to consider I/O requirements when designing disk storage for an OLTP environment. Architects or designers should integrate the applications I/O characteristics and I/O distribution (amount of read and write requests) types for successful storage design. Disks could act as I/O-centric devices or Capacity-centric devices for the systems. Some applications require high throughput and some require high disk capacity. If we are talking about backup, disks are a good solution to cover near term data storage purposes for the backups. Cache is an indispensable device for storage systems since it improves read and write performance dramatically. However, if the subject is heavy “sequential access” via “big block sizes,” cache becomes a big hindrance to the system. Also, it depends on systems data characteristics and I/O handling performance. If we want use disks in a backup environment, we have to clarify following questions: ‐ How are the disks using by backup software? ‐ Do I have to use (or configure) storage cache? ‐ What is the backup applications’ block size? ‐ What are the estimated bandwidth requirements? We have to remember that backup is not a business requirement; availability and continuity are the business requirements. Disk Space Utilization Problems on Every Tier Storage utilization is another problem. It is difficult, and sometimes impossible, to move unused disk spaces between systems. This situation is called “White Space Utilization” or “Poor Utilization.” Storage administrators and server administrators often use different terms when talking about disk space utilization. Storage Administrators usually talk about “LUNs” and are not interested in the contents. System Administrators care about “Disk Sizes” or “Disk Performance” and are not interested in where it is located or how it is protected. Tapes are still less expensive than disks. Tape drives (writing heads), however, are very expensive so we need to use tape drives efficiently during backup operations. The problem is that typical backup clients cannot saturate tape drives efficiently. We need a staging area for low speed backup clients. At this point, a disk looks like a good choice. So, I need an intelligent backup device that has to solve these problems: ‐ Poorly utilized tape drive resources, ‐ Long backup windows, ‐ Higher tape costs, ‐ Tape management and vaulting problems, (Tape Security, Theft or lost tapes, damage risk etc.) A traditional “backup-to-disk-to-tape” (B2D2T) approach will not solve all of these problems. I don‘t want to be busy with Volume Management, RAID Configuration, Space Management, Disk-to-Tape Operation etc., I want to copy the (consistent) data to a secondary (or maybe third) storage area as soon as possible. Approximately 12 years ago, IBM© developed the first Virtual Tape Library (VTL) [1] for its mainframe systems. VTL is a (hypervisor like) “Virtualization Layer” that represents disks as a tape library. This new approach helped to improve backup performance and reduce the number of tape drives. Several years later, ISVs (Independent Software Vendors) developed similar VTL’s for Open Systems. In addition, VTL (in EMC’s naming terminology, EDL) is a good solution for reducing RPO / RTO values in backup environments. If you are using VTL you can eliminate these problems: ‐ Tape Drive hardware problems ‐ Tape Library partitioning problems ‐ Decreasing tape drive costs ‐ Highly utilized tape drive resources (VTL staging or post processing) ‐ Reducing backup windows ‐ Security problems, (backup encryption) Without de-duplication, replicating backup to another
Recommended publications
  • Using EMC VNX Storage with Vmware Vsphere Techbook CONTENTS
    Using EMC® VNX® Storage with VMware vSphere Version 4.0 TechBook P/N H8229 REV 05 Copyright © 2015 EMC Corporation. All rights reserved. Published in the USA. Published January 2015 EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. For the most up-to-date regulatory document for your product line, go to EMC Online Support (https://support.emc.com). 2 Using EMC VNX Storage with VMware vSphere TechBook CONTENTS Preface Chapter 1 Configuring VMware vSphere on VNX Storage Technology overview................................................................................... 18 EMC VNX family..................................................................................... 18 FLASH 1st.............................................................................................. 18 MCx multicore optimization..................................................................
    [Show full text]
  • DMFS - a Data Migration File System for Netbsd
    DMFS - A Data Migration File System for NetBSD William Studenmund Veridian MRJ Technology Solutions NASAAmes Research Center" Abstract It was designed to support the mass storage systems de- ployed here at NAS under the NAStore 2 system. That system supported a total of twenty StorageTek NearLine ! have recently developed DMFS, a Data Migration File tape silos at two locations, each with up to four tape System, for NetBSD[I]. This file system provides ker- drives each. Each silo contained upwards of 5000 tapes, nel support for the data migration system being devel- and had robotic pass-throughs to adjoining silos. oped by my research group at NASA/Ames. The file system utilizes an underlying file store to provide the file The volman system is designed using a client-server backing, and coordinates user and system access to the model, and consists of three main components: the vol- files. It stores its internal metadata in a flat file, which man master, possibly multiple volman servers, and vol- resides on a separate file system. This paper will first man clients. The volman servers connect to each tape describe our data migration system to provide a context silo, mount and unmount tapes at the direction of the for DMFS, then it will describe DMFS. It also will de- volman master, and provide tape services to clients. The scribe the changes to NetBSD needed to make DMFS volman master maintains a database of known tapes and work. Then it will give an overview of the file archival locations, and directs the tape servers to move and mount and restoration procedures, and describe how some typi- tapes to service client requests.
    [Show full text]
  • Data Protection and Recovery in the Small and Mid-Sized Business (SMB)
    Data Protection and Recovery in the Small and Mid-sized Business (SMB) An Outlook Report from Storage Strategies NOW By Deni Connor, Patrick H. Corrigan and James E. Bagley Intern: Emily Hernandez October 11, 2010 Storage Strategies NOW 8815 Mountain Path Circle Austin, Texas 78759 Note: The information and recommendations made by Storage Strategies NOW, Inc. are based upon public information and sources and may also include personal opinions both of Storage Strategies NOW and others, all of which we believe are accurate and reliable. As market conditions change however and not within our control, the information and recommendations are made without warranty of any kind. All product names used and mentioned herein are the trademarks of their respective owners. Storage Strategies NOW, Inc. assumes no responsibility or liability for any damages whatsoever (including incidental, consequential or otherwise), caused by your use of, or reliance upon, the information and recommendations presented herein, nor for any inadvertent errors which may appear in this document. This report is purchased by Geminare, who understands and agrees that the report is furnished solely for its internal use only and may not be furnished in whole or in part to any other person other than its directors, officers and employees, without the prior written consent of Storage Strategies NOW. Copyright 2010. All rights reserved. Storage Strategies NOW, Inc. 1 Sponsor 2 Table of Contents Sponsor ..................................................................................................................................................................
    [Show full text]
  • [1 ] Storagetek Automated Cartridge System
    StorageTek[1] Automated Cartridge System Library Software Product Information Release 8.4 E62371-05 March 2018 StorageTek Automated Cartridge System Library Software Product Information, Release 8.4 E62371-05 Copyright © 2015, 2018, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs.
    [Show full text]
  • Compass™ Virtual Tape Library a High-Performance, Software-Defined Backup Solution to Manage Your Existing Infrastructure’S Tape Libraries
    COBALT IRON DATASHEET Compass™ Virtual Tape Library A high-performance, software-defined backup solution to manage your existing infrastructure’s tape libraries FEATURES What is Compass VTL Fully integrated VTL functionality within The VTL market is at the precipice of disruption where organizations are now directed to Compass data protection modernize and unify the enterprise data protection environment. Customers still have platform requirements for tape operations but they require operational simplicity and cost Eliminate physical tape effectiveness. Cobalt Iron introduces a software-defined VTL that is fully integrated media, transport, and into its Compass SaaS platform. Unlike any other VTL solution, Compass VTL is a storage costs software-enabled feature that is directly integrated within an end-to-end data protection Fully automated backup, restore, and disaster solution. With Compass VTL, organizations can deploy a single, unified solution that recovery of virtual tape unleashes data from traditional physical and virtual tape data silos all while modernizing libraries, drives, and tape environments which require tape resources for backup and recovery. Emulates multi-platform, heterogeneous tape environments Compass VTL Benefits Leverage multi-tier storage both on-premises and in the cloud Intelligently scales locally SIMPLICITY FEATURES SAVINGS and in the cloud Direct VTL integration Enterprise data protection With Compass VTL, organizations Incorporates AES 256 bit encryption and with industry-leading solution delivered in
    [Show full text]
  • IBM Storage: Product Guide
    IBM Storage Product guide Contents 3 Introduction 5 Portfolio 7 Storage for Data and AI 11 Storage for hybrid multicloud 71 Modern data protection 12 Storage for IBM Z 52 Storage for SAP HANA 92 Storage for containers 33 Storage networking 73 Converged infrastructure 40 IBM Services for storage 41 Conclusion Storage made simple Helping is the right foundation for your data-driven make hybrid multicloud. storage simple. Introduction Throughout the world, IBM Storage has the broadest portfolio in What’s possible Hybrid multicloud the industry that can enable organizations to Reduce complexity with container-enabled organizations in different effectively deliver data-driven IT services across with IBM Storage enterprise storage, deployed seamlessly their hybrid multicloud environments. Using across on-premises and hybrid cloud industries are using IBM® IBM solutions, enterprises can more efficiently storage environments. Storage and software-defined use their compute and storage resources to help achieve their goals. Whether it’s human storage (SDS) solutions for big genome research for cancer treatments and AI and data personalized medicine, aerodynamics testing Simplify your infrastructure with optimized data, analytics, AI and deep for race cars, or stock portfolio analysis for risk efficiency to drive faster results that are learning, high-performance assessment, IBM Storage can help enterprises massively scalable and globally available become more agile, modern and security-rich. from edge to inference. computing and the cloud to improve business results in Data protection and cyber resiliency their digital transformation. Maximize backup storage efficiency, data security and performance with maximum uptime and resiliency while lowering costs. Cloud paks and containers Surpass your business goals with speed, productivity and agility, all enabled by a security-rich, persistent storage infrastructure with enterprise-class services for private cloud and Red Hat® OpenShift® container storage.
    [Show full text]
  • Optimizing Vmware Protection with HP and Veeam
    Optimizing VMware protection with HP and Veeam Technical white paper Table of contents Executive summary ............................................................................................................................... 2 Key points ....................................................................................................................................... 2 Introduction ......................................................................................................................................... 2 Technology overview ........................................................................................................................... 2 VMware vStorage APIs and Change Block Tracking ............................................................................. 3 Veeam Backup source-side deduplication ............................................................................................ 3 HP StoreOnce D2D appliance target-side deduplication ....................................................................... 3 Combining Veeam Backup inline and HP D2D target-based deduplication ................................................ 3 HP StoreOnce D2D appliances and Veeam Backup ................................................................................. 4 An appliance vs. general purpose disk for data protection .................................................................... 4 Important considerations when sizing your HP StoreOnce D2D appliance .................................................
    [Show full text]
  • Downloaded for Free From
    The Design of the NetBSD I/O Subsystems SungWon Chung Pusan National University 2 This book is dedicated to the open-source code developers in the NetBSD community. The original copy of this publication is available in an electronic form and it can be downloaded for free from http://arXiv.org. Copyright (c) 2002 by SungWon Chung. For non-commercial and personal use, this publication may be reproduced, stored in a retrieval system, or transmitted in any form by any means, electronic, mechanical, photocopying, recording or otherwise. For commercial use, no part of this publication can be reproduced by any means without the prior written permission of the author. NetBSD is the registered trademark of The NetBSD Foundation, Inc. Contents Preface 14 I Basics to Learn Filesystem 15 1 Welcome to the World of Kernel ! 17 1.1 How Does a System Call Dive into Kernel from User Program ? . 17 1.1.1 Example: write system call . 17 1.1.2 Ultra SPARC 0x7c CPU Trap . 18 1.1.3 Jump to the File Descriptor Layer . 24 1.1.4 Arriving at Virtual Filesystem Operations . 28 1.2 General Data Structures in Kernel such as List, Hash, Queue, ... 30 1.2.1 Linked-Lists . 30 1.2.2 Tail Queues . 34 1.2.3 Hash . 38 1.3 Waiting and Sleeping in Kernel . 39 1.4 Kernel Lock Manager . 39 1.4.1 simplelock and lock . 39 1.4.2 Simplelock Interfaces . 40 1.4.3 Lock Interfaces . 40 1.5 Kernel Memory Allocation . 43 1.6 Resource Pool Manager .
    [Show full text]
  • Virtual Tape Library (VTL) with Deduplication 10.0
    PRODUCT SPECS VERSION FalconStor Virtual Tape Library (VTL) with Deduplication 10.0 Virtual Tape Library (VTL) Cluster / Single Instance Repository Cluster Deduplication Gateway Scalable HA clustered gateways for existing san storage. The FalconStor® Optimized Backup and Deduplication solution and Single Instance Repository (SIR) cluster gateways enable design flexibility, scalability, high availability (HA), and integrated global deduplication for large enterprise or integrator environments with existing SAN storage. Unlike single-function grid architectures, it is the only solution that scales HA backup nodes and HA cluster deduplication nodes independently, offering the flexibility to add gateways to handle larger data sets and tighter backup windows, or additional cluster deduplication nodes for enhanced performance, scalability, and data retention while offering customizable levels of security. Scaling up to 12PB of shared usable storage with seamless global deduplication across all nodes, Optimized Backup Cluster gateways eliminate individual deduplication silos. This helps IT departments to meet stringent data requirements and ensure continuous operations while containing the cost associated with explosive data growth. Highlights GLOBAL DEDUPLICATION ● Integrated NAS & virtual tape deduplication & compression ● Up to 95% data reduction ● Minimizes costs: power, cooling, floor space, IT resources ● Extends local disk retention for fast data recovery ● Flexible policies: Inline, post-process, Turbo, or no deduplication 16GB FC & 1/10GbE
    [Show full text]
  • Compass Virtual Tape Library Feature
    Compass Virtual Tape Library Feature Compass VTL Feature Architecture • Software agnostic data path • Client-less, standard data path for platforms • Virtual tape library interface to the Accelerator • Emulates tape libraries, tape drives, and tape cartridges • Multiple instances of libraries and drives • Fiber channel connectivity, high-performance 2 Compass VTL Feature Use Cases • Data Domain • ProtecTIER replacement (EOS 11/2021) • AS/400, iSeries, IBM i • HPE OpenVMS • Large systems/databases on AIX, Linux, and Windows • Aging tape landscapes for faster conversion off 3 Compass VTL Feature – Platforms & Emulations Supported Emulations Platforms • IBM 3584 • IBM i/iSeries • STK L700 • IBM Power AIX • ACSLS • Linux on Intel • Proprietary LTO-1, LTO-3, • Windows on LTO-4, LTO-6 Intel • 3592 • HPE OpenVMS 4 Compass VTL Feature Differentiation • Single, enterprise SaaS data protection • Analytics-driven monitoring, solution management, and optimizations for • Software-defined cloud connectivity handsfree operations and while supporting on-premises maintenance of entire backup workloads infrastructure • Automates IBM i/iSeries DR • Software-enaBled liBrary, drive, and operations, transparent cloud cartridge operations with automated connectivity, and direct integrations of management via Compass IBM i/iSeries • Unified VTL and data protection 5 Compass VTL Feature Sizing and Tiers Sizing Options Deployment Tiers • Sized by daily ingest volumes 0 to 2 TB • De-staging of ingested data 3 to 8 TB to Compass 9 to 16 TB • Long term retention is
    [Show full text]
  • Using Virtual Tape Libraries with Veritas Netbackup™ Software
    WHITE PAPER: TECHNICAL Using Virtual Tape Libraries with Veritas NetBackup™ Software Don Peterson, Principal Product Manager April 2009 1 Using Virtual Tape Libraries with Veritas NetBackup Software Table of Contents What is a Virtual Tape Library? 3 How Is a VTL Qualified with NetBackup Software? 3 How are VTLs licensed with NetBackup? 4 Comparing the VTL and a Standard Tape Library 4 Comparing the VTL and Disk Storage Units 5 Can I use my VTL with NetBackup OpenStorage? 6 Does it Matter How Many Virtual Tape Drives are Configured? 7 Should Multiplexing Be Used with a VTL? 9 How Does Deduplication Affect Backups to VTLs? 9 Will Different Virtual and Physical Drive Types Cause Problems or Affect Performance? 10 Using NetBackup to Duplicate Backup Images from a VTL to Physical Tape 11 Can I Use the VTL Software to Copy Virtual Media to Physical Tape? 12 Is There a Reason to Use the Shared Storage Option When Using a VTL? 14 Why Does the VTL Indicate the Wrong Amount of Free Space? 14 Summary 16 2 Using Virtual Tape Libraries with Veritas NetBackup Software What is a Virtual Tape Library? A Virtual Tape Library (VTL) is typically a dedicated appliance, which emulates a physical tape library and tape drives and stores backup images on disk. The VTL allows existing tape backup software to be used and the paradigm for managing backups remains fairly similar to that of using physical tape. However, there are some differences and we’ll discuss the implication of thinking the paradigm is identical later in the paper.
    [Show full text]
  • Virtual Tape Library Appliance
    Datasheet Virtual Tape Library Appliance StarWind Virtual Tape Library Appliance (VTLA) is ransomware-resilient archival storage for your business-critical data that complements your on-premises environment with public and object storage tiers. Step away from costly tape-centric backup infrastructures in favor of “air-gapped” virtual tapes to ensure that your data archival and retention requirements are always met. Automatically replicate and tier your data offsite to AWS, Microsoft Azure, Backblaze B2, or any other public cloud storage for maximum storage infrastructure cost-efficiency and data security. Simplify on-premises storage scaling and management with a range of certified industry-standard object storage solutions. Double Fine Productions achieved higher applications performance without compromising infrastructure management simplicity “Price + Simplicity. StarWind VSAN offers great pricing and allows us to get work done without expensive custom hardware. It also makes getting back online quick and easy after an outage or hardware failure” Aaron Hayes, IT Manager Simplicity > Storage is presented as LTO for easy integration into existing tape-centric backup environments > Meet regulatory requirements easily with secure automated data management policies > Unmanned DR shipping: automatic tiering and replication to the cloud per user-set policies Features > Shrink backup time and improve RPO using fast disk storage combined with 10/25 GbE connectivity > Get RTO close to zero with instant restores from disk-backed virtual tapes > Protect your data from ransomware and disasters by air-gapping it with VTLA and an offsite copy Savings > Save on space and resources by replacing physical tape infrastructure with just one appliance > Save money on media storage and shipping by leveraging secure and scalable cloud storage > Save time on implementation: little to no changes required to the existing infrastructure DATASHEET 1 HYPERCONVERGENCE FOR SMB & ROBO Models StarWind VTL Appliance comes in 3 models to fulfill various capacity requirements.
    [Show full text]