riding the clouds - best practices in data center transformation, into the next century and beyond EMC Proven Professional Knowledge Sharing 2011 Paul Brant EMC Corporation [email protected] 2011 EMC Proven Professional Knowledge Sharing 2 Table of Contents Table of Figures................................................................................................. 10 Table of Equations ............................................................................................ 12 Abstract .............................................................................................................. 13 Introduction ....................................................................................................... 15 The need for Preservation in the Cloud/Next Generation Data Center ....................... 17 The ascent of the Computer in the Cloud/Next Generation Data Center .................... 18 The ascent of Data Longevity in the Cloud/Next Generation Data Center .................. 19 The ascent of Virtualization in the Cloud/Next Generation Data Center ..................... 21 The ascent of the Appliance in the Cloud/Next-Generation Data Center .................... 25 The ascent of the Intercloud – AKA Interconnected Global Connectivity .................... 26 The ascent of the Personal Data Store in the Cloud/Next-Generation Data Center .... 27 The Personal Information Management Ecosystem ................................................ 31 The Drivers for the Next-Generation Data Center ...................................................... 32 Attributes and Technologies ............................................................................ 35 Massively Parallel Processing (MPP) and other architectures .................................... 37 Best Practice – Understand advantages and trade-offs of scalable systems in the NGDC ............................................................................................................................... 37 SMP Architectures .................................................................................................. 38 Clustered Architecture Systems .............................................................................. 39 Best Practice – Understand Advantages in MPP (massively parallel processing) in addressing the NGDC ..................................................................................................... 41 Best Practice – Understand Amdahl’s and Gustafin’s Laws as it relates to MPP to achieve NGDC architectures ........................................................................................................ 42 Applianceization......................................................................................................... 47 Best Practice – Consider Applianceization in Big Data Architectures ...................... 48 Best Practice – Utilize Applianceization in the Financial Sector .............................. 49 Best Practice – For NGDC DBMS, consider utilizing appliances ............................. 51 2011 EMC Proven Professional Knowledge Sharing 3 Best Practice – Implement Applianceization within the infrastructure – VBLOCK Technology and Architectures .................................................................................................... 52 Best Practice – Implement a Unified Management solution for Appliance Architectures ............................................................................................................................... 53 Best Practice – Consider utilizing appliance structures in the cloud (IT in a box) .... 55 Data Transference ..................................................................................................... 57 Best Practice – Implement Directory-based Cache Coherency to create efficient local and remote distributed Data Transference ..................................................................... 59 Best Practice – Implement Directory-Based Cache Coherency for Data Federation63 Object Affinity ............................................................................................................ 65 Best Practice – Standardize Cloud Computing Interfaces that is object-based and vendor- neutral .................................................................................................................... 66 Best Practice – Implement an Object-Based Open Cloud Computing Interface into NGDC designs ................................................................................................................... 66 Security ..................................................................................................................... 70 Best Practice – Implement Identity assurance into the Personal Data Store Architecture ............................................................................................................................... 70 Best Practice – Implement PDS to enable the Cloud in an ever-changing social media world ............................................................................................................................... 71 Best Practice – Implement Federated login and password for Personal Data Stores72 Best Practice – Embrace Identity methodologies to Enhance Security ................... 73 Best Practice – Consider Containerization to determine who you should trust with your data ............................................................................................................................... 74 Virtualization .............................................................................................................. 75 Best Practice – Consider Virtualization considerations when implementing a Cloud or Intercloud................................................................................................................ 75 Best Practice – Implement the NGDC with VM metadata abstraction considerations using OVF ........................................................................................................................ 76 Best Practice – Consider Client and Server Virtualization into the NGDC ............... 77 Best Practice – Utilize Virtualization to address cost reduction and cost avoidance in the NGDC ..................................................................................................................... 78 Best Practice – Understand the benefits of Virtualized Service-Based Delivery ...... 79 Personal Data Store Attributes ................................................................................... 82 2011 EMC Proven Professional Knowledge Sharing 4 Best Practice – Implement Personal Data Stores into the NGDC architecture ........ 82 Best Practice – Implement additional Data Store flows to the existing lexicon ........ 87 Best Practice – Implement NGDC‘s to address the needs of Personal Data Stores as a service to the individual. ......................................................................................... 89 Archival Ecosystem ................................................................................................... 94 What is an Archive .......................................................................................................... 94 Best Practice – Implement OAIS in the Next Generation Data Center Archive Architectures ................................................................................................................... 97 Best Practice – Utilize Archival strategies for digital content preservation by leveraging the knowledge of the archival profession ............................................................................. 101 Best Practice – Implement the Self-contained Information Retention Format (SIRF) for long-term retention in the NGDC ................................................................................... 102 Best Practice – Integrate SIRF and OAIS into the NGDC .............................................. 106 Best Practice – Integrate SIRF and XAM into the NGDC ............................................... 109 Best Practice – Integrate SIRF and JHOVE into the NGDC ........................................... 110 Best Practice – Integrate SIRF and Bagit into the NGDC .............................................. 111 Best Practice – Next-Generation Data Center containerization should address specific Media and Platform requirements .................................................................................. 111 Storage Ecosystem .................................................................................................. 114 Storage Object Affinity .......................................................................................... 114 Storage for Cloud Computing ............................................................................... 114 Best Practice – Implement an Object-based Data Management Infrastructure to Address Interoperability and Transparency Requirements .......................................................... 115 Best Practice – Implement CDMI Metadata Properties Consistent with the CDMI Specification .................................................................................................................. 117 Data Portability ..................................................................................................... 117 Best Practice – Implement Data Portability techniques into the NGDC architecture ...... 118 Storage Tiering ....................................................................................................