DOCSLIB.ORG

Implementing IBM Storage Data Deduplication Solutions

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Implementing IBM Storage Data Deduplication Solutions Front cover Implementing IBM Storage Data Deduplication Solutions Fitting deduplication into your enterprise environment Adding deduplication to your storage hierarchy Gaining the space benefits of IBM deduplication Alex Osuna Eva Balogh Alexandre Ramos Galante de Carvalho Rucel F. Javier Zohar Mann ibm.com/redbooks International Technical Support Organization Implementing IBM Storage Data Deduplication Solutions March 2011 SG24-7888-00 Note: Before using this information and the product it supports, read the information in “Notices” on page ix. First Edition (March 2011) This edition applies to Version 2.5 of ProtecTIER, versions 6.1and 6.2 of IBM Tivoli Storage Manager and Data ONTAP 7.3.4 for N series. © Copyright International Business Machines Corporation 2011. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . ix Trademarks . .x Preface . xi The team who wrote this book . xi Now you can become a published author, too! . xiii Comments welcome. xiii Stay connected to IBM Redbooks . xiv Part 1. Introduction to Deduplication . 1 Chapter 1. Introduction to Deduplication . 3 1.1 IBM data reduction and deduplication. 4 1.2 Deduplication overview . 4 1.2.1 Chunking. 5 1.2.2 Processing . 6 1.2.3 Consolidation . 6 1.3 Architecture . 6 1.3.1 Where deduplication processing occurs . 6 1.3.2 When deduplication occurs. 7 1.4 Benefits of data deduplication . 8 Chapter 2. Introduction to N series Deduplication. 9 2.1 How deduplication for IBM System Storage N series storage system works . 10 2.2 Deduplicated Volumes . 13 2.3 Deduplication metadata . 14 2.4 Sizing for performance and space efficiency . 15 2.4.1 Deduplication general best practices . 15 2.5 Compressing and Deduplicating . 17 Chapter 3. Introduction to ProtecTIER deduplication . 19 3.1 Terms and definitions . 20 3.2 Overview of HyperFactor technology and deduplication. 22 3.3 IBM System Storage TS7600 with ProtecTIER. 24 3.3.1 TS7610 - Entry Edition . 24 3.3.2 TS7650 - Appliance Edition . 24 3.3.3 TS7650G - Enterprise Edition . 24 3.3.4 TS7680 - Gateway Edition for System z . 24 3.4 Deduplication with HyperFactor technology . 25 3.4.1 Impact of HyperFactor . 25 3.4.2 ProtecTIER data ingest flow . 26 3.4.3 ProtecTIER VTL concepts . 28 3.4.4 ProtecTIER OST concepts . 28 3.4.5 Steady state . 29 Chapter 4. Introduction to IBM Tivoli Storage Manager deduplication . 31 4.1 Deduplication overview . 32 4.2 How ITSM data deduplication works. 33 4.3 IBM Tivoli Storage Manager deduplication overview . 34 © Copyright IBM Corp. 2011. All rights reserved. iii 4.3.1 ITSM server-side deduplication overview . 34 4.3.2 ITSM client-side deduplication overview. 35 Part 2. Planning for deduplication. 39 Chapter 5. N series deduplication planning . 41 5.1 Supported hardware models and ONTAP versions . 42 5.2 Deduplication and Data ONTAP version considerations. 42 5.3 Deduplication licensing . 43 5.4 Compatibility with native N series functions . 43 5.4.1 Deduplication and high availability pairs . 43 5.4.2 Deduplication and non-disruptive upgrades . 44 5.4.3 Deduplication and Performance Acceleration Modules . 44 5.4.4 Deduplication and Snapshot copies . 44 5.4.5 Deduplication and SnapRestore . 45 5.4.6 Deduplication and SnapMirror replication . 45 5.4.7 Deduplication and SnapVault . 49 5.4.8 Deduplication and SnapLock . 50 5.4.9 Deduplication and MultiStore (vFiler) . 51 5.4.10 Deduplication and LUNs . 51 5.4.11 Deduplication and the volume copy command . 55 5.4.12 Deduplication and FlexClone volumes . 55 5.4.13 Deduplication and read reallocation . 56 5.4.14 Deduplication with Quotas . 56 5.4.15 Deduplication with NDMP . 56 5.4.16 Deduplication with DataFabric Manager/Protection Manager. 57 5.5 Compatibility with non-native functions . 57 5.5.1 Deduplication with VMWare . 57 5.5.2 Deduplication with Tivoli Storage Manager. 58 5.5.3 Deduplication with Backup Exec. 58 5.5.4 Deduplication with Lotus Domino . 58 5.5.5 Deduplication with Microsoft Exchange . 58 5.6 Data Characteristics . 58 5.7 Deduplication and storage capacity . 60 5.7.1 Metadata . 60 5.8 Deduplication and performance . ..
Load more

Recommended publications
  • Tape Drive Technology Comparison Sony AIT · Exabyte Mammoth · Quantum DLT by Roger Pozak Boulder, CO (303) 786-7970 Roger [email protected] Updated April 1999
    Tape Drive Technology Comparison Sony AIT · Exabyte Mammoth · Quantum DLT By Roger Pozak Boulder, CO (303) 786-7970 [email protected] Updated April 1999 1 Introduction Tape drives have become the preferred device for backing up hard disk data files, storing data and protecting against data loss. This white paper examines three leading mid-range tape drives technologies available today: Sony AIT, Exabyte Mammoth and Quantum DLT. These three technologies employ distinctly different recording formats and exhibit different performance characteristics. Therefore, choosing among and investing in one of these technologies calls for a complete understanding of their respective strengths and weaknesses. Evolution of Three Midrange Tape Drive Technologies Exabyte introduced the 8mm helical scan tape drive in 1985. The 8mm drive mechanical sub-assembly was designed and manufactured by Sony while Exabyte supplied the electronics, firmware, cosmetics and marketing expertise. Today, Exabyte’s Mammoth drive is designed and manufactured entirely by Exabyte. Sony, long a leading innovator in tape technology, produces the AIT (Advance Intelligent Tape) drives. The AIT drive is designed and manufactured entirely by Sony. Although the 8mm helical scan recording method is used, the AIT recording format is new and incompatible with 8mm drives from Exabyte. Quantum Corporation is the manufacturer of DLT (Digital Linear Tape) drives. Quantum purchased the DLT technology from Digital Equipment Corporation in 1994 and has successfully developed and marketed several generations of DLT drive technology including the current DLT- 7000 product. Helical Scan vs Linear Serpentine Recording Sony AIT and Exabyte Mammoth employ a helical scan recording style in which data tracks are written at an angle with respect to the edge of the tape.
    [Show full text]
  • Secure Data Storage – White Paper Storage Technologies 2008
    1 Secure Data Storage – White Paper Storage Technologies 2008 Secure Data Storage - An overview of storage technology - Long time archiving from extensive data supplies requires more then only big storage capacity to be economical. Different requirements need different solutions! A technology comparison repays. Author: Dr. Klaus Engelhardt Dr. K. Engelhardt 2 Secure Data Storage – White Paper Storage Technologies 2008 Secure Data Storage - An overview of storage technology - Author: Dr. Klaus Engelhardt Audit-compliant storage of large amounts of data is a key task in the modern business world. It is a mistake to see this task merely as a matter of storage technology. Instead, companies must take account of essential strategic and economic parameters as well as legal regulations. Often one single technology alone is not sufficient to cover all needs. Thus storage management is seldom a question of one solution verses another, but a combination of solutions to achieve the best possible result. This can frequently be seen in the overly narrow emphasis in many projects on hard disk-based solutions, an approach that is heavily promoted in advertising, and one that imprudently neglects the considerable application benefits of optical storage media (as well as those of tape-based solutions). This overly simplistic perspective has caused many professional users, particularly in the field of long-term archiving, to encounter unnecessary technical difficulties and economic consequences. Even a simple energy efficiency analysis would provide many users with helpful insights. Within the ongoing energy debate there is a simple truth: it is one thing to talk about ‘green IT’, but finding and implementing a solution is a completely different matter.
    [Show full text]
  • Environmental Stability Study and Life Expectancies of Magnetic Media for Use with Ibm 3590 and Quantum Digital Linear Tape Systems
    ARKIVAL TECHNOLOGY CORPORATION ENVIRONMENTAL STABILITY STUDY AND LIFE EXPECTANCIES OF MAGNETIC MEDIA FOR USE WITH IBM 3590 AND QUANTUM DIGITAL LINEAR TAPE SYSTEMS Ronald D. Weiss Arkival Technology Corporation 427 Amherst St. Nashua, NH 03063 Phone: (603) 881-3322 www.ARKIVAL.com 1 ARKIVAL TECHNOLOGY CORPORATION ENVIRONMENTAL STABILITY STUDY AND LIFE EXPECTANCIES OF MAGNETIC MEDIA FOR USE WITH IBM 3590 AND QUANTUM DIGITAL LINEAR TAPE SYSTEMS National Archives and Records Administration #NAMA-01-F-0061 FINAL REPORT PERIOD of WORK: October 2001- June 2002 Federal Supply Service Contract Number: GS 35F 0611L Arkival Technology Corporation 427 Amherst St. Nashua, NH 03063 Phone: (603) 881-3322 www.ARKIVAL.com 2 TABLE of CONTENTS Executive Summary………………………………………………………… 5 1.0 STUDY OVERVIEW…………………………………………………… 7 1.1 Technical Areas of Study 1.2 Study Duration 1.3 Tape Drive Recording Features 1.4 Test Environments 1.5 Data Collection and Analysis 2.0 TECHNICAL STUDIES 2.1 Magnetic & Microstructure Analyses of the Recording Media….. 9 2.1.1 Transmission Electron Microscopy (TEM) Analysis 2.1.2 Vibrating Sample Magnetometer (VSM) Analysis 2.2 VSM/Aging Study…………………………………………………. 14 2.3 Binder Hydrolysis ………………………………………………... 17 2.4 Physical Properties ……………………………………………….. 24 2.4.1 Resistivity 2.4.2 Friction 2.5 RF Amplitude response …………………………………………….28 2.6 Error Study ……………………………………………………….. 31 2.6.1 Catastrophic Tape and Drive Failures 2.7 Life Expectancies (LE) …………………………………………… 38 2.7.1 Magnetic Saturation LE 2.7.2 Error Rate (B*ER) LE Conclusions
    [Show full text]
  • Tape Drive Mounting Hole Dimensions
    3URGXFW6SHFLILFDWLRQ 3URGXFW6SHFLILFDWLRQ 3URGXFW6SHFLILFDWLRQ 7DSH'ULYH7DSH'ULYH '/79 $ DLT-V4 Product Specification, 81-81349-02 A01, November 2005, Made in USA. Quantum Corporation provides this publication “as is” without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability or fitness for a particular purpose. Quantum Corporation may revise this publication from time to time without notice. COPYRIGHT STATEMENT Copyright 2005 by Quantum Corporation. All rights reserved. Your right to copy this document is limited by copyright law. Making copies or adaptations without prior written authorization of Quantum Corporation is prohibited by law and constitutes a punishable violation of the law. TRADEMARK STATEMENT Quantum, the Quantum logo, DLT, DLTtape, the DLTtape logo, and Super DLTtape logo are all registered trademarks of Quantum Corporation. DLTIce, DLTSage, and Super DLTtape are all trademarks of Quantum Corporation. Other company and product names used in this document are trademarks, registered trademarks, or service marks of their respective owners. Contents Preface ix Chapter 1 Physical Specifications 1 Physical Description ........................................................................................................ 2 Physical Dimensions and Weights ......................................................................... 2 Internal Tape Drive Mounting Hole Dimensions........................................................ 4 Chapter 2 Functional Specifications
    [Show full text]
  • Digital Linear Tape (DLT) Technology and Product Family Overview
    Digital Linear Tape (DLT) Technology and Product Family Overview Demetrios Lignos Quantum Corporation 333 South Street Shrewsbury, MA 01545-4112 +1-508-770-3495 [email protected] Introduction The demand that began a couple of years ago for increased data storage capacity continues [1]. Peripheral Strategies (a Santa Barbara, California, Storage Market Re- search Firm) projects the amount of data stored on the average enterprise network will grow by 50 percent to 100 percent per year. Furthermore, Peripheral Strategies says that a typical mid-range workstation system containing 30GB to 50GB of stor- age today will grow at the rate of 50% per year. Dan Friedlander, a Boulder, Colo- rado-based consultant specializing in PC-LAN backup, says “The average NetWare LAN is about 8GB, but there are many that have 30GB to 300GB.....” The substantial growth of storage requirements has created various tape tech- nologies that seek to satisfy the needs of today’s and, especially, the next generation’s systems and applications. There are five leading tape technologies in the market to- day: QIC (Quarter Inch Cartridge), IBM 3480/90, 8mm, DAT (Digital Audio Tape) and DLT (Digital Linear Tape). Product performance specifications and user needs have combined to classify these technologies into low-end, mid-range, and high-end systems applications. Although the manufacturers may try to position their products differently, product specifications and market requirements have determined that QIC and DAT are primarily low-end systems products while 8mm and DLT are compet- ing for mid-range systems applications and the high-end systems space, where IBM compatibility is not required.
    [Show full text]
  • DLT-S4 Product Manual, 81-81278-01 A01, July 2006, Made in USA
    3URGXFW0DQXDO3URGXFW0DQXDO 3URGXFW0DQXDO 3URGXFW0DQXDO '/767DSH'ULYH '/76 $ DLT-S4 Product Manual, 81-81278-01 A01, July 2006, Made in USA. Quantum Corporation provides this publication “as is” without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability or fitness for a particular purpose. Quantum Corporation may revise this publication from time to time without notice. COPYRIGHT STATEMENT Copyright 2006 by Quantum Corporation. All rights reserved. Your right to copy this document is limited by copyright law. Making copies or adaptations without prior written authorization of Quantum Corporation is prohibited by law and constitutes a punishable violation of the law. TRADEMARK STATEMENT Quantum, the Quantum logo, DLT, DLTtape, and the DLTtape logo are registered trademarks of Quantum Corporation in the U.S. and other countries. The DLT logo, GoVault, DLTSage, and SuperLoader are trademarks of Quantum Corporation. All other trademarks are the property of their respective owners. Contents Preface xiii Chapter 1 Product Overview 1 Storage Capacity and Transfer Rates............................................................................. 2 Tape Drive Models........................................................................................................... 2 Tape Drive Features......................................................................................................... 4 Maximum Data Transfer Rate .......................................................................................
    [Show full text]
  • IBM 7206 Model VX2 External Tape Drive with VXA-2 Support Provides an Excellent Migration Path from 4-Mm Tape Drives
    Hardware Announcement July 9, 2002 IBM 7206 Model VX2 External Tape Drive with VXA-2 Support Provides an Excellent Migration Path from 4-mm Tape Drives Overview Key Prerequisites At a Glance The IBM 7206 Model VX2 External AIX V4.1.5, 4.2.0, 4.3.0, 5.1.0 or Tape Drive is a stand-alone, SCSI, later The IBM 7206 Model VX2 External VXA-2 streaming drive that attaches Tape Drive is a large-capacity tape externally to the IBM Planned Availability Date drive that writes data to tape using pSeries and RS/6000 families of a Discrete Packet Format. workstations and servers. August 23, 2002 Features include: Key Features • Increased tape capacity of up to four times that of the 7206 The 7206 Model VX2 External Tape Model 220 — up to 80 GB native Drive writes data to tape using a per cartridge (160 GB with 2:1 Discrete Packet Format. The tape compression) drive provides a media capacity of • up to 80 GB (160 GB with 2:1 A higher data transfer rate — compression) data storage per up to 6.0 MB per second cartridge. It has a sustained data • Data formatted in packets with transfer rate of up to 6.0 MBps. variable speed scan Benefits • Multiple data packet scan in one single tape pass Providing an excellent migration path from 4-mm tape drives, the 7206 Model VX2 is a cost-effective solution for your save, restore and archiving functions. • The increased storage capacity up to 80 GB (160 GB with compression) is nearly four times the capacity of the IBM 7206 Model 220, reducing the number of tape cartridges needed.
    [Show full text]
  • IBM 7205 Model 440 40 GB External Digital Linear Tape Drive Enhances Data Storage
    Hardware Announcement March 13, 2001 IBM 7205 Model 440 40 GB External Digital Linear Tape Drive Enhances Data Storage Overview Key Prerequisites At a Glance The IBM 7205 Model 440 is a Supported by AIX Version 4.3.3, or cost-effective solution for later, with maintenance package The 7205 Model 440 is a save-and-restore and archiving 4330-08 larger-capacity, functions. It is an external, higher-performance digital linear stand-alone, LVD Ultra2 tape drive that features: SCSI-attached, DLT8000 tape drive Planned Availability Date • that attaches to the IBM Increased tape capacity of pSeries and RS/6000 families of March 30, 2001 40 GB native per cartridge (80 workstations and servers. Building GB with 2:1 compression) using on the traditional strengths of DLTtape IV DLTtape technology, this DLT8000 • Improved data transfer rate up tape drive provides significant to 6 MB per second advantages in performance and (uncompressed), up to 12 MB capacity over any others in its class. per second with 2:1 The 7205 Model 440 half-inch compression cartridge tape drive provides leadership in data storage and • Ultra2/Wide Low Voltage retrieval for the most demanding Differential (LVD) SCSI interface data backup, archive, and near • External, self-powered online storage applications. packaging It has: For ordering, contact: • Your IBM representative, an IBM Business A media capacity of up to 40 GB Partner, or IBM Americas Call Centers at (80 GB with 2:1 compression) data 800-IBM-CALL Reference: RE001 storage per cartridge • A sustained data rate of up to 6 MB per second uncompressed, 12 MB per second compressed (using 2:1 compression) The IBM 7205 Model 440: • Is read and write compatible with DLT8000 and previous DLT formats • Provides an excellent migration path from ¼-inch, 4mm, or 8mm tape drives This announcement is provided for your information only.
    [Show full text]
  • Delivery Recommendations 070711
    Recommendation for Delivery of Recorded Music Projects 080107 rev 48 This document has been created as a Recommendation for Delivery of Recorded Music Projects. This document specifies the physical deliverables that are the foundation of the creative process, with the understanding that it is in the interest of all parties involved to make them accessible for both the short term and the long term. Thus, this document recommends reliable backup, delivery and archiving methodologies for current audio technologies, which should ensure that music will be completely and reliably recoverable and protected from damage, obsolescence and loss. The Delivery Specifications Committee, comprised of producers, engineers, record company executives and others working primarily in Nashville, New York and Los Angeles (and in conjunction with the AES Technical Committee on Studio Practices and Production and the AES Nashville Section), developed the Delivery Recommendations over the course of two years. During its development, the committee met regularly at the Recording Academy® Nashville Chapter offices to debate the issues surrounding the short term and long term viability of the creative tools used in the recording process, and to design a specification in the interest of all parties involved in the recording process. The committee reached consensus in July, 2002 and the committee’s recommendations were finalized and presented to The Recording Academy Producers & Engineers Wing membership, the overall recording community, and to press in Nashville on July 19, 2002. The document was also presented to the AES in the Studio Practices and Production Tech Committee meeting on October 7th, 2002 in Los Angeles, and on March 24th, 2003 in Amsterdam.
    [Show full text]
  • Magneto-Optical Drives, Flash Memory Devices, and Tape Drives Are Useful Supplements to Primary Storage
    13 0789729741 ch12 7/15/03 4:11 PM Page 663 CHAPTER 12 High-Capacity Removable Storage 13 0789729741 ch12 7/15/03 4:11 PM Page 664 664 Chapter 12 High-Capacity Removable Storage The Role of Removable-Media Drives Since the mid-1980s, the primary storage device used by computers has been the hard disk drive. However, for data backup, data transport between computers, and temporary storage, secondary storage devices such as high-capacity removable media drives, floptical drives, magneto-optical drives, flash memory devices, and tape drives are useful supplements to primary storage. Pure optical storage—such as CD-R, CD-RW, DVD-RAM, DVD+RW, DVD-RW, and others—is covered in Chapter 13, “Optical Storage.” These types of drives can also be used as a supplement to hard disk storage as well as for pri- mary storage. The options for purchasing removable devices vary. Some removable-media drives use media as small as a quarter or your index finger, whereas others use larger media up to 5 1/4''. Most popular removable- storage drives today have capacities that range from as little as 16MB to as much as 100GB or more. These drives offer fairly speedy performance and the capability to store anything from a few data files or less frequently used programs to complete hard disk images on a removable disk or tape. The next two sections examine the primary roles of these devices. Extra Storage As operating systems and applications continue to grow in size and features, more and more storage space is needed for these programs as well as for the data they create.
    [Show full text]
  • Timeline of Computer History
    Timeline of Computer History By Year By Category Search AI & Robotics (55) Computers (145) Graphics & Games (48) Memory & Storage (61)(61) Networking & The Popular Culture (50) Software & Languages (60) Manchester Mark I Williams- 1947 Kilburn tube EDSAC 1949 Manchester Mark I Williams-Kilburn tube At Manchester University, Freddie Williams and Tom Kilburn develop the Williams-Kilburn tube. The tube, tested in 1947, was the first high-speed, entirely electronic memory. It used a cathode ray tube (similar to an analog TV picture tube) to store bits as dots on the screen’s surface. Each dot lasted a fraction of a second before fading so the information was constantly refreshed. Information was read by a metal pickup plate that would detect a change in electrical charge. Maurice Wilkes with EDSAC Maurice Wilkes and his team at the University of Cambrid construct the Electronic Delay Storage Automatic Calcula (EDSAC). EDSAC, a stored program computer, used me delay line memory. Wilkes had attended the University of Pennsylvania's Moore School of Engineering summer sessions about the ENIAC in 1946 and shortly thereafter Magnetic drum memory began work on the EDSAC. 1950 MIT - Magnetic core memory ERA founders with various magnetic drum memories Eager to enhance America’s codebreaking capabilities, the US Navy contracts with Engineering Research Associates (ERA) for a stored program computer. The result was Atlas, completed in 1950. Atlas used magnetic drum memory, which stored information on the outside of a rotating cylinder coated with ferromagnetic material and circled by read/write heads in Jay Forrester holding early core memory plane fixed positions.
    [Show full text]
  • Federal Register/Vol. 67, No. 250/Monday, December 30, 2002
    Federal Register / Vol. 67, No. 250 / Monday, December 30, 2002 / Rules and Regulations 79517 § 51.317 [Corrected] NARA received seven responses to files using FTP can be accomplished in 2. On page 69666, third column, the proposed rule, six from Federal a variety of ways. The most common paragraph (g)(3), the words ‘‘paragraphs agencies and one from a private sector methods are dial-up modems and high- (1)’’ are corrected to read ‘‘paragraphs commenter. speed or broadband Internet connections. NARA works closely with (g)(1)’’. File Transfer Protocol each individual agency in arranging its § 51.318 [Corrected] FTP is a media-less transfer method specific FTP transfers to ensure that the 3. On page 69667, second column, that can be used to transfer electronic agency has an appropriate secure means paragraph (i)(e), the words ‘‘paragraphs records. FTP operates by using special of transferring the records by FTP. (1)’’ are corrected to read ‘‘paragraphs software located at the sending and DLTtape IV (i)(1)’’. receiving sites. This software, in combination with a telecommunications DLTtape IV cartridge tape is a high- Dated: December 20, 2002. network, provides the means for density magnetic cartridge tape that can A.J. Yates, transferring electronic records. The store up to 40 gigabytes of information Administrator, Agricultural Marketing agency may send any documentation in on each cartridge. DLTtape IV tapes are Service. electronic format to NARA via FTP as used by selected tape drive units [FR Doc. 02–32805 Filed 12–27–02; 8:45 am] part of the transfer of the electronic produced by several companies.
    [Show full text]