DOCSLIB.ORG

IBM Real-Time Compression in IBM SAN Volume Controller and IBM Storwize V7000

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
IBM Real-Time Compression in IBM SAN Volume Controller and IBM Storwize V7000 Front cover IBM Real-time Compression in IBM SAN Volume Controller and IBM Storwize V7000 Discover how embedded compression software aids data reduction Learn about IBM Random Access Compression Engine technology See the compression savings that can be achieved Christian Burns Bosmat Tuv-El Jorge Quintal Jon Tate ibm.com/redbooks Redpaper International Technical Support Organization IBM Real-time Compression in IBM SAN Volume Controller and IBM Storwize V7000 March 2015 REDP-4859-01 Note: Before using this information and the product it supports, read the information in “Notices” on page vii. Second Edition (March 2015) This edition applies to those hardware and software products as detailed in the paper only. © Copyright International Business Machines Corporation 2012, 2015. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . vii Trademarks . viii Preface . ix Authors. ix Now you can become a published author, too! . xi Comments welcome. xi Stay connected to IBM Redbooks . xi Summary of changes. xiii March 2015, Second Edition . xiii Chapter 1. Overview . 1 1.1 Current IT challenges . 2 1.2 The solution: IBM Real-time Compression . 2 1.3 Common use cases . 3 1.3.1 General-purpose volumes. 3 1.3.2 Databases. 4 1.3.3 Virtualized infrastructures . 4 1.4 IBM Real-time Compression . 4 Chapter 2. Compression technology. 5 2.1 Compression technology history . 6 2.2 Data efficiency technologies . 7 2.2.1 Thin provisioning technology . 7 2.2.2 FlashCopy (thin-provisioned snapshots). 8 2.2.3 IBM Easy Tier . 8 2.2.4 Archiving and space management . 9 2.2.5 Data deduplication . 9 2.3 Data compression technologies . 10 Chapter 3. IBM Real-time Compression . 11 3.1 Overview . 12 3.2 IBM Real-time Compression . 12 3.2.1 Random Access Compression Engine . 13 3.2.2 Location-based compression . 15 3.2.3 Temporal compression . 17 3.2.4 RACE in the SAN Volume Controller software stack . 18 3.2.5 Data write flow . 20 3.2.6 Data read flow. 20 3.2.7 Compression of existing data . 20 3.3 Software and hardware updates that enhance IBM Real-time Compression . 21 3.3.1 Software enhancements . 21 3.3.2 Hardware enhancements . 22 Chapter 4. Planning . 25 4.1 Candidate data sets for compression . 26 4.1.1 Data types. 26 4.2 Requirements . 26 © Copyright IBM Corp. 2012, 2015. All rights reserved. iii 4.2.1 Supported hardware configurations . 27 4.2.2 Hardware requirements. 27 4.2.3 Compressible data . 28 4.3 Comprestimator. 29 4.3.1 Downloading Comprestimator. 29 4.3.2 Installing Comprestimator . 29 4.3.3 Installing Comprestimator on Linux or AIX . 30 4.3.4 Installing Comprestimator on IBM i through VIOS . 30 4.3.5 Installing Comprestimator on ESXi . 30 4.3.6 Installing Comprestimator on Windows server . 30 4.3.7 Using Comprestimator . 30 4.3.8 Available space. 33 4.3.9 Licensing. 34 4.4 General guidelines . 34 4.4.1 Compression ratio recommendations . 35 4.4.2 Guidelines for getting a high compression ratio . 35 4.4.3 Easy Tier. 35 4.4.4 I/O Groups and compressed volumes. 36 4.5 Generic volumes that are created from reused extents . 36 4.5.1 How reused extents consume system resources . 36 4.5.2 How reused extents affect Comprestimator results . 36 4.5.3 Avoiding this behavior. 37 Chapter 5. Configuring compressed volumes . 39 5.1 Compression . 40 5.2 Software licensing for the SAN Volume Controller and Storwize V7000. 40 5.2.1 Licensing for the SAN Volume Controller using the GUI. 41 5.2.2 Licensing for the SAN Volume Controller using the CLI . 42 5.2.3 Licensing for the Storwize V7000 using the GUI. 42 5.2.4 Licensing for the Storwize V7000 using the CLI . 43 5.3 Configuring compressed volumes using the SAN Volume Controller and Storwize V7000 GUI . ..
Load more

Recommended publications
  • Data Preparation & Descriptive Statistics
    Data Preparation & Descriptive Statistics (ver. 2.4) Oscar Torres-Reyna Data Consultant [email protected] PU/DSS/OTR http://dss.princeton.edu/training/ Basic definitions… For statistical analysis we think of data as a collection of different pieces of information or facts. These pieces of information are called variables. A variable is an identifiable piece of data containing one or more values. Those values can take the form of a number or text (which could be converted into number) In the table below variables var1 thru var5 are a collection of seven values, ‘id’ is the identifier for each observation. This dataset has information for seven cases (in this case people, but could also be states, countries, etc) grouped into five variables. id var1 var2 var3 var4 var5 1 7.3 32.27 0.1 Yes Male 2 8.28 40.68 0.56 No Female 3 3.35 5.62 0.55 Yes Female 4 4.08 62.8 0.83 Yes Male 5 9.09 22.76 0.26 No Female 6 8.15 90.85 0.23 Yes Female 7 7.59 54.94 0.42 Yes Male PU/DSS/OTR Data structure… For data analysis your data should have variables as columns and observations as rows. The first row should have the column headings. Make sure your dataset has at least one identifier (for example, individual id, family id, etc.) id var1 var2 var3 var4 var5 First row should have the variable names 1 7.3 32.27 0.1 Yes Male 2 8.28 40.68 0.56 No Female Cross-sectional data 3 3.35 5.62 0.55 Yes Female 4 4.08 62.8 0.83 Yes Male 5 9.09 22.76 0.26 No Female 6 8.15 90.85 0.23 Yes Female 7 7.59 54.94 0.42 Yes Male id year var1 var2 var3 1 2000 7 74.03 0.55 Group 1 1 2001 2 4.6 0.44 At least one identifier 1 2002 2 25.56 0.77 2 2000 7 59.52 0.05 Cross-sectional time series data Group 2 2 2001 2 16.95 0.94 or panel data 2 2002 9 1.2 0.08 3 2000 9 85.85 0.5 Group 3 3 2001 3 98.85 0.32 3 2002 3 69.2 0.76 PU/DSS/OTR NOTE: See: http://www.statistics.com/resources/glossary/c/crossdat.php Data format (ASCII)… ASCII (American Standard Code for Information Interchange).
    [Show full text]
  • Full Document
    R&D Centre for Mobile Applications (RDC) FEE, Dept of Telecommunications Engineering Czech Technical University in Prague RDC Technical Report TR-13-4 Internship report Evaluation of Compressibility of the Output of the Information-Concealing Algorithm Julien Mamelli, [email protected] 2nd year student at the Ecole´ des Mines d'Al`es (N^ımes,France) Internship supervisor: Luk´aˇsKencl, [email protected] August 2013 Abstract Compression is a key element to exchange files over the Internet. By generating re- dundancies, the concealing algorithm proposed by Kencl and Loebl [?], appears at first glance to be particularly designed to be combined with a compression scheme [?]. Is the output of the concealing algorithm actually compressible? We have tried 16 compression techniques on 1 120 files, and the result is that we have not found a solution which could advantageously use repetitions of the concealing method. Acknowledgments I would like to express my gratitude to my supervisor, Dr Luk´aˇsKencl, for his guidance and expertise throughout the course of this work. I would like to thank Prof. Robert Beˇst´akand Mr Pierre Runtz, for giving me the opportunity to carry out my internship at the Czech Technical University in Prague. I would also like to thank all the members of the Research and Development Center for Mobile Applications as well as my colleagues for the assistance they have given me during this period. 1 Contents 1 Introduction 3 2 Related Work 4 2.1 Information concealing method . 4 2.2 Archive formats . 5 2.3 Compression algorithms . 5 2.3.1 Lempel-Ziv algorithm .
    [Show full text]
  • PKZIP MVS User's Guide
    PKZIP for MVS MVS/ESA, OS/390, & z/OS User’s Guide PKMU-V5R5000 PKWARE, Inc. PKWARE, Inc. 9009 Springboro Pike Miamisburg, Ohio 45342 Sales: 937-847-2374 Support: 937-847-2687 Fax: 937-847-2375 Web Site: http://www.pkzip.com Sales - E-Mail: [email protected] Support - http://www.pkzip.com/support 5.5 Edition (2003) PKZIP for MVS™, PKZIP for OS/400™, PKZIP for VSE™, PKZIP for UNIX™, and PKZIP for Windows™ are just a few of the many members in the PKZIP® family. PKWARE, Inc. would like to thank all the individuals and companies -- including our customers, resellers, distributors, and technology partners -- who have helped make PKZIP® the industry standard for Trusted ZIP solutions. PKZIP® enables our customers to efficiently and securely transmit and store information across systems of all sizes, ranging from desktops to mainframes. This edition applies to the following PKWARE of Ohio, Inc. licensed program: PKZIP for MVS™ (Version 5, Release 5, 2003) PKZIP(R) is a registered trademark of PKWARE(R) Inc. Other product names mentioned in this manual may be a trademark or registered trademarks of their respective companies and are hereby acknowledged. Any reference to licensed programs or other material, belonging to any company, is not intended to state or imply that such programs or material are available or may be used. The copyright in this work is owned by PKWARE of Ohio, Inc., and the document is issued in confidence for the purpose only for which it is supplied. It must not be reproduced in whole or in part or used for tendering purposes except under an agreement or with the consent in writing of PKWARE of Ohio, Inc., and then only on condition that this notice is included in any such reproduction.
    [Show full text]
  • Implementing Compression on Distributed Time Series Database
    Implementing compression on distributed time series database Michael Burman School of Science Thesis submitted for examination for the degree of Master of Science in Technology. Espoo 05.11.2017 Supervisor Prof. Kari Smolander Advisor Mgr. Jiri Kremser Aalto University, P.O. BOX 11000, 00076 AALTO www.aalto.fi Abstract of the master’s thesis Author Michael Burman Title Implementing compression on distributed time series database Degree programme Major Computer Science Code of major SCI3042 Supervisor Prof. Kari Smolander Advisor Mgr. Jiri Kremser Date 05.11.2017 Number of pages 70+4 Language English Abstract Rise of microservices and distributed applications in containerized deployments are putting increasing amount of burden to the monitoring systems. They push the storage requirements to provide suitable performance for large queries. In this paper we present the changes we made to our distributed time series database, Hawkular-Metrics, and how it stores data more effectively in the Cassandra. We show that using our methods provides significant space savings ranging from 50 to 95% reduction in storage usage, while reducing the query times by over 90% compared to the nominal approach when using Cassandra. We also provide our unique algorithm modified from Gorilla compression algorithm that we use in our solution, which provides almost three times the throughput in compression with equal compression ratio. Keywords timeseries compression performance storage Aalto-yliopisto, PL 11000, 00076 AALTO www.aalto.fi Diplomityön tiivistelmä Tekijä Michael Burman Työn nimi Pakkausmenetelmät hajautetussa aikasarjatietokannassa Koulutusohjelma Pääaine Computer Science Pääaineen koodi SCI3042 Työn valvoja ja ohjaaja Prof. Kari Smolander Päivämäärä 05.11.2017 Sivumäärä 70+4 Kieli Englanti Tiivistelmä Hajautettujen järjestelmien yleistyminen on aiheuttanut valvontajärjestelmissä tiedon määrän kasvua, sillä aikasarjojen määrä on kasvanut ja niihin talletetaan useammin tietoa.
    [Show full text]
  • Pack, Encrypt, Authenticate Document Revision: 2021 05 02
    PEA Pack, Encrypt, Authenticate Document revision: 2021 05 02 Author: Giorgio Tani Translation: Giorgio Tani This document refers to: PEA file format specification version 1 revision 3 (1.3); PEA file format specification version 2.0; PEA 1.01 executable implementation; Present documentation is released under GNU GFDL License. PEA executable implementation is released under GNU LGPL License; please note that all units provided by the Author are released under LGPL, while Wolfgang Ehrhardt’s crypto library units used in PEA are released under zlib/libpng License. PEA file format and PCOMPRESS specifications are hereby released under PUBLIC DOMAIN: the Author neither has, nor is aware of, any patents or pending patents relevant to this technology and do not intend to apply for any patents covering it. As far as the Author knows, PEA file format in all of it’s parts is free and unencumbered for all uses. Pea is on PeaZip project official site: https://peazip.github.io , https://peazip.org , and https://peazip.sourceforge.io For more information about the licenses: GNU GFDL License, see http://www.gnu.org/licenses/fdl.txt GNU LGPL License, see http://www.gnu.org/licenses/lgpl.txt 1 Content: Section 1: PEA file format ..3 Description ..3 PEA 1.3 file format details ..5 Differences between 1.3 and older revisions ..5 PEA 2.0 file format details ..7 PEA file format’s and implementation’s limitations ..8 PCOMPRESS compression scheme ..9 Algorithms used in PEA format ..9 PEA security model .10 Cryptanalysis of PEA format .12 Data recovery from
    [Show full text]
  • Steganography and Vulnerabilities in Popular Archives Formats.| Nyxengine Nyx.Reversinglabs.Com
    Hiding in the Familiar: Steganography and Vulnerabilities in Popular Archives Formats.| NyxEngine nyx.reversinglabs.com Contents Introduction to NyxEngine ............................................................................................................................ 3 Introduction to ZIP file format ...................................................................................................................... 4 Introduction to steganography in ZIP archives ............................................................................................. 5 Steganography and file malformation security impacts ............................................................................... 8 References and tools .................................................................................................................................... 9 2 Introduction to NyxEngine Steganography1 is the art and science of writing hidden messages in such a way that no one, apart from the sender and intended recipient, suspects the existence of the message, a form of security through obscurity. When it comes to digital steganography no stone should be left unturned in the search for viable hidden data. Although digital steganography is commonly used to hide data inside multimedia files, a similar approach can be used to hide data in archives as well. Steganography imposes the following data hiding rule: Data must be hidden in such a fashion that the user has no clue about the hidden message or file's existence. This can be achieved by
    [Show full text]
  • Winzip 12 Reviewer's Guide
    Introducing WinZip® 12 WinZip® is the most trusted way to work with compressed files. No other compression utility is as easy to use or offers the comprehensive and productivity-enhancing approach that has made WinZip the gold standard for file-compression tools. With the new WinZip 12, you can quickly and securely zip and unzip files to conserve storage space, speed up e-mail transmission, and reduce download times. State-of-the-art file compression, strong AES encryption, compatibility with more compression formats, and new intuitive photo compression, make WinZip 12 the complete compression and archiving solution. Building on the favorite features of a worldwide base of several million users, WinZip 12 adds new features for image compression and management, support for new compression methods, improved compression performance, support for additional archive formats, and more. Users can work smarter, faster, and safer with WinZip 12. Who will benefit from WinZip® 12? The simple answer is anyone who uses a PC. Any PC user can benefit from the compression and encryption features in WinZip to protect data, save space, and reduce the time to transfer files on the Internet. There are, however, some PC users to whom WinZip is an even more valuable and essential tool. Digital photo enthusiasts: As the average file size of their digital photos increases, people are looking for ways to preserve storage space on their PCs. They have lots of photos, so they are always seeking better ways to manage them. Sharing their photos is also important, so they strive to simplify the process and reduce the time of e-mailing large numbers of images.
    [Show full text]
  • Zlib Home Site
    zlib Home Site http://zlib.net/ A Massively Spiffy Yet Delicately Unobtrusive Compression Library (Also Free, Not to Mention Unencumbered by Patents) (Not Related to the Linux zlibc Compressing File-I/O Library) Welcome to the zlib home page, web pages originally created by Greg Roelofs and maintained by Mark Adler . If this page seems suspiciously similar to the PNG Home Page , rest assured that the similarity is completely coincidental. No, really. zlib was written by Jean-loup Gailly (compression) and Mark Adler (decompression). Current release: zlib 1.2.6 January 29, 2012 Version 1.2.6 has many changes over 1.2.5, including these improvements: gzread() can now read a file that is being written concurrently gzgetc() is now a macro for increased speed Added a 'T' option to gzopen() for transparent writing (no compression) Added deflatePending() to return the amount of pending output Allow deflateSetDictionary() and inflateSetDictionary() at any time in raw mode deflatePrime() can now insert bits in the middle of the stream ./configure now creates a configure.log file with all of the results Added a ./configure --solo option to compile zlib with no dependency on any libraries Fixed a problem with large file support macros Fixed a bug in contrib/puff Many portability improvements You can also look at the complete Change Log . Version 1.2.5 fixes bugs in gzseek() and gzeof() that were present in version 1.2.4 (March 2010). All users are encouraged to upgrade immediately. Version 1.2.4 has many changes over 1.2.3, including these improvements:
    [Show full text]
  • PKZIP®/Securezip® for I5/OS® User's Guide
    PKZIP®/SecureZIP® ® for i5/OS User’s Guide SZIU- V10R05M02 PKWARE, Inc. PKWARE, Inc. 648 N Plankinton Avenue, Suite 220 Milwaukee, WI 53203 Main office: 888-4PKWARE (888-475-9273) Sales: 937-847-2374 (888-4PKWARE / 888-475-9273) Sales: Email: [email protected] Support: 937-847-2687 Support: http://www.pkware.com/support/system-i Fax: 414-289-9789 Web Site: http://www.pkware.com 10.0.5 Edition (2010) SecureZIP for z/OS, PKZIP for z/OS, SecureZIP for i5/OS®, PKZIP for i5/OS, SecureZIP for UNIX, and SecureZIP for Windows are just a few of the members of the PKZIP family. PKWARE Inc. would like to thank all the individuals and companies—including our customers, resellers, distributors, and technology partners—who have helped make PKZIP the industry standard for trusted ZIP solutions. PKZIP enables our customers to efficiently and securely transmit and store information across systems of all sizes, ranging from desktops to mainframes. This edition applies to the following PKWARE Inc. licensed programs: PKZIP for i5/OS (Version 10, Release 0.5, 2010) SecureZIP for i5/OS (Version 10, Release 0.5, 2010) SecureZIP Partner for i5/OS (Version 10, Release 0.5, 2010) PKWARE, PKZIP and SecureZIP are registered trademarks of PKWARE, Inc. z/OS, i5/OS, zSeries, and iSeries are registered trademarks of IBM Corporation. Other product names mentioned in this manual may be trademarks or registered trademarks of their respective companies and are hereby acknowledged. This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/) Any reference to licensed programs or other material, belonging to any company, is not intended to state or imply that such programs or material are available or may be used.
    [Show full text]
  • ORDERINFO 3 Copy
    ORDER info ORDER info Placing Your Order We will need to know the following information before your order can be quoted: PLEASE NOTE: Artwork with low resolution will project fuzzy and/or distorted and will not 1. Customer name and contact information. give the best appearance. Most web images are created at 72 dpi. Please do not try to resize an image taken from the Web from 72 dpi to a higher resolution. To receive the highest quality 2. The lighting instrument that will be projecting the gobo. image, we require artwork to be saved at a minimum resolution of 600 dpi with an image size 3. Quantity needed and gobo size (in millimeters). of at least 5 x 5 inches (127 x 127 mm). 4. Whether you intend to project the image on to a surface from the front or on to a screen ORDER from behind. Artwork received at lower resolutions will incur an additional (hourly) artwork fee. However, 5. Is the background of the image clear or black? the quality and detail of the gobo will still be determined by the original quality of the artwork. 6. When you need the order shipped. We also accept hard copies of artwork; such as photographs, slicks, or prints. Faxed images When placing orders, please e-mail the above information and attached artwork to are accepted, but usually do not reproduce well. Please submit a reasonably sized, clean copy info [email protected] . A quotation and proof will be generated and returned to you of your artwork. Additional (hourly) artwork fees may apply for poor quality artwork.
    [Show full text]
  • Forcepoint DLP Supported File Formats and Size Limits
    Forcepoint DLP Supported File Formats and Size Limits Supported File Formats and Size Limits | Forcepoint DLP | v8.8.1 This article provides a list of the file formats that can be analyzed by Forcepoint DLP, file formats from which content and meta data can be extracted, and the file size limits for network, endpoint, and discovery functions. See: ● Supported File Formats ● File Size Limits © 2021 Forcepoint LLC Supported File Formats Supported File Formats and Size Limits | Forcepoint DLP | v8.8.1 The following tables lists the file formats supported by Forcepoint DLP. File formats are in alphabetical order by format group. ● Archive For mats, page 3 ● Backup Formats, page 7 ● Business Intelligence (BI) and Analysis Formats, page 8 ● Computer-Aided Design Formats, page 9 ● Cryptography Formats, page 12 ● Database Formats, page 14 ● Desktop publishing formats, page 16 ● eBook/Audio book formats, page 17 ● Executable formats, page 18 ● Font formats, page 20 ● Graphics formats - general, page 21 ● Graphics formats - vector graphics, page 26 ● Library formats, page 29 ● Log formats, page 30 ● Mail formats, page 31 ● Multimedia formats, page 32 ● Object formats, page 37 ● Presentation formats, page 38 ● Project management formats, page 40 ● Spreadsheet formats, page 41 ● Text and markup formats, page 43 ● Word processing formats, page 45 ● Miscellaneous formats, page 53 Supported file formats are added and updated frequently. Key to support tables Symbol Description Y The format is supported N The format is not supported P Partial metadata
    [Show full text]
  • Data Compression and Archiving Software Implementation and Their Algorithm Comparison
    Calhoun: The NPS Institutional Archive Theses and Dissertations Thesis Collection 1992-03 Data compression and archiving software implementation and their algorithm comparison Jung, Young Je Monterey, California. Naval Postgraduate School http://hdl.handle.net/10945/26958 NAVAL POSTGRADUATE SCHOOL Monterey, California THESIS^** DATA COMPRESSION AND ARCHIVING SOFTWARE IMPLEMENTATION AND THEIR ALGORITHM COMPARISON by Young Je Jung March, 1992 Thesis Advisor: Chyan Yang Approved for public release; distribution is unlimited T25 46 4 I SECURITY CLASSIFICATION OF THIS PAGE REPORT DOCUMENTATION PAGE la REPORT SECURITY CLASSIFICATION 1b RESTRICTIVE MARKINGS UNCLASSIFIED 2a SECURITY CLASSIFICATION AUTHORITY 3 DISTRIBUTION/AVAILABILITY OF REPORT Approved for public release; distribution is unlimite 2b DECLASSIFICATION/DOWNGRADING SCHEDULE 4 PERFORMING ORGANIZATION REPORT NUMBER(S) 5 MONITORING ORGANIZATION REPORT NUMBER(S) 6a NAME OF PERFORMING ORGANIZATION 6b OFFICE SYMBOL 7a NAME OF MONITORING ORGANIZATION Naval Postgraduate School (If applicable) Naval Postgraduate School 32 6c ADDRESS {City, State, and ZIP Code) 7b ADDRESS (City, State, and ZIP Code) Monterey, CA 93943-5000 Monterey, CA 93943-5000 8a NAME OF FUNDING/SPONSORING 8b OFFICE SYMBOL 9 PROCUREMENT INSTRUMENT IDENTIFICATION NUMBER ORGANIZATION (If applicable) 8c ADDRESS (City, State, and ZIP Code) 10 SOURCE OF FUNDING NUMBERS Program Element No Project Nc Work Unit Accession Number 1 1 TITLE (Include Security Classification) DATA COMPRESSION AND ARCHIVING SOFTWARE IMPLEMENTATION AND THEIR ALGORITHM COMPARISON 12 PERSONAL AUTHOR(S) Young Je Jung 13a TYPE OF REPORT 13b TIME COVERED 1 4 DATE OF REPORT (year, month, day) 15 PAGE COUNT Master's Thesis From To March 1992 94 16 SUPPLEMENTARY NOTATION The views expressed in this thesis are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S.
    [Show full text]