Cloud Storage: Key to Future of IT Infrastructure Anil Vasudeva President & Chief Analyst [email protected] 408-268-0800 Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack Storage Usage in Data Centers & in Cloud Mega-Centers Applications mapped by Key Workload & Storage Metrics (2) Cloud Infrastructure Technology Enablers Virtualization: Key to Cloud Storage Infrastructure SSDs: A New SCM Filling Price Performance Gaps Networks: Fast March to 100Gb/sec Hadoop: Meeting Big Data/BI/Data Base Paradigm (3) Drivers for Cloud Storage Adoption Storage Efficiencies and Costs Reduction Tier 4 Storage for Data Protection Real-Time Online Analytics Technology Innovations HW, SW 2005-2015 (3) Cloud Segments: Challenges & Solutions Adopting Public/Private/Hybrid Clouds: Pros & Cons (4) Cloud Storage Requirements Adopting Public/Private/Hybrid Clouds: Pros & Cons Best Practices: Getting Ready for the Cloud (5) Key Take-Aways Data Centers & Cloud Infrastructure Public CloudCenter© Enterprise VZ Data Center On-Premise Cloud Vertical Clouds Servers VPN Switches: Layer 4-7, IaaS, PaaS Layer 2, 10GbE, FC Stg SaaS Supplier/Partners ISP ISP Internet ISP FC/ IPSANs ISP Core Optical ISP ISP Edge Caching, Proxy, Database Servers, Remote/Branch Office ISP FW, SSL, IDS, DNS, LB, Middleware, Data Web Servers Mgmt Tier-1 Application Servers Tier-3 Edge Apps HA, File/Print, ERP, Data Base Web 2.0 SCM, CRM Servers Servers Social Ntwks. Facebook, Tier-2 Apps Cellular Twitter, YouTube… Directory Security Policy Management Wireless Cable/DSL… Middleware Platform Home Networks 4 Source:: IMEX Research - Cloud Infrastructure Report ©2009-12 IT Industry Journey - Roadmap Analytics – BI IT Industry Predictive Analytics - Unstructured Data Roadmap From Dashboards Visualization to Prediction Engines using Big Data. IMEX Research Cloudization On-Premises > Private Clouds > Public Clouds DC to Cloud-Aware Infrast. & Apps. Cascade migration to SPs/Public Clouds. Automation Automatically Maintains Application SLAs (Self-Configuration, Self-Healing©IMEX, Self-Acctg. Charges etc) Virtualization Pools Resources. Provisions, Optimizes, Monitors Shuffles Resources to optimize Delivery of various Business Services Integration/Consolidation Integrate Physical Infrast./Blades to meet CAPSIMS ®IMEX Cost, Availability, Performance, Scalability, Inter-operability, Manageability & Security Standardization Standard IT Infrastructure- Volume Economics HW/Syst SW (Servers, Storage, Networking Devices, System Software (OS, MW & Data Mgmt SW) Cloud Computing Ecosystem Cloud Services Providers Examples Service Public - BT, Telstra, T-Systems France Telecom Public – Mutitenancy,OnDemand Private – Providers Private - On Premises, Enterprise Hybrid – IBM/Cloudburst, Hybrid – Interoperable P2P Software-as-a-Service Examples eMail - Yahoo!,Google… SaaS - Servers, Network, Storage Collaboration - Facebook,Twitter … - Management, Reporting Bus.Apps - SalesForce, GoogleApps, Intuit… Examples Platform Tools & Services Amazon EC2 PaaS - Deploy developed platforms Force.com ready for Application SW on Navitaire Cloud Aware Infrastructure Examples Infrastructure HW & Amazon S3 IaaS Services Nirvanix - Servers, Network, Storage - Management, Reporting Virtualized Cloud Infrastructure Cloud Computing Private Cloud Hybrid Public Cloud Service Enterprise Cloud Providers SaaS Applications SaaS SLA SLA SLA SLA App App SLA App App App Platform Tools & Services ….. ….. EJB .Net PHP Ruby PaaS Python Operating Systems Management Virtualization IaaS Resources (Servers, Storage, Networks) Application’s SLA dictates the Resources Required to meet specific requirements of Availability, Performance, Cost, Security, Manageability etc. Data Storage Usage – In Corporate Data Centers I/O Access Frequency vs. Percent of Corporate Data 100% 95% 75% Disk Tape 65% Arrays Libraries • Back Up Data • Tables • Archived Data • Indices • Offsite DataVault Cache • Hot Data • Logs • Journals • Temp Tables • Hot Tables SSD % of I/O Accesses of I/O% 1% 2% 10% 50% 100% % of Corporate Data 8 Source:: IMEX Research - Cloud Infrastructure Report ©2009-12 Applications Best Suited for Cloud • Temporary Storage – Test & Dev Data, Excess Storage from Data Centers • Internet Web Content – Web 2.0 user-generated content, Digital Catalogs, Public Files, iPhone App Content • Data Archiving & Records Retention – Regulatory compliance, Corporate governance, Audits & searches, Legal, Intellectual Property, Risk Management • Data Back Up & Restore – Offsite data protection, Historical Versioning, Online Access, Automated Retrieval • Intranet File Storage – Employee Portals, Digital Libraries, Document Collaboration, Training Media, File Shares, Historical Versioning, Online Access, Automated Retrieval Source: IMEX Research SSD Industry Report ©2010-122 9 NextGen Data Storage Usage in Cloud I/O Access Frequency vs. Percent of Corporate Data 95% 75% Cloud 65% FCoE/ Storage SAS SATA Arrays • Back Up Data SSD • Tables • Archived Data • Logs • Indices • Offsite DataVault • Journals • Hot Data • Temp Tables • Hot Tables % of I/O Accesses of I/O% 1% 2% 10% 50% 100% % of Corporate Data 10 Source:: IMEX Research - Cloud Infrastructure Report ©2009-12 Market Segments by Applications 1000 K OLTP Transaction Processing eCommerce 100 K 1) - Business (RAID - 0, 3) (RAID - 1, 5, 6) Data Intelligence Warehousing 10K (*Latency OLAP IOPS* Scientific Computing 1K HPC Imaging TP 100 Audio Web 2.0 HPC Video 10 1 5 10 50 100 500 *IOPS for a required response time ( ms) MB/sec *=(#Channels*Latency-1) Source:: IMEX Research - Cloud Infrastructure Report ©2009-12 Cloud Storage Requirements Data Protection Storage Efficiency DRAM Back Up/Archive/DR Auto Tiered Storage Flash RAID – 0,1,5,6,10 Storage Virtualization SSD Virtual Tape Thin Provisioning Performance Replication Deduplication Disk MAID Capacity Disk Tape Auto Tiering System using SSDs Data Class (Tiers 0,1,2,3) Storage Media Type (Flash/Disk/Tape) Policy Engines (Workload Mgmt) Transparent Migration (Data Placement) File Virtualization (Uninterrupted App.Opns.in Migration) Source: IMEX Research SSD Industry Report ©2011 Workloads Consolidation using VZ • A single server 1.5x larger than standard 2-way server will handle consolidated load of 6 servers. • VZ manages the workloads + important apps get the compute resources they need automatically w/o operator intervention. • Physical consolidation of 15-20:1 is easily possible • Reasonable goal for VZ x86 servers – 40-50% utilization on large systems (>4way), rising as dual/quad core processors becomes available • Savings result in Real Estate, Power & Cooling, High Availability, Hardware, Management Source: Dan Olds & IMEX Research 2009 TCO Savings with Virtualization 995 Pre-Virtualization (VZ) Servers 78 VZ $16,000 Servers 0800 - Provisioning Downtime (408)268 $12,000 Disaster Recovery DC Real Estate Power & Cooling $8,000 Network SAN Hardware Cost over Cost over 3 years imex@ imexresearch.comimex@ $4,000 VZ SW & $- For TCO Analysis,EM: TCOFor Support w/o VZ w VZ Cloud Storage Requirements Data Protection Storage Efficiency DRAM Back Up/Archive/DR Auto Tiered Storage Flash RAID – 0,1,5,6,10 Storage Virtualization SSD Virtual Tape Thin Provisioning Performance Replication Deduplication Disk MAID Capacity Disk Tape Auto Tiering System using SSDs Data Class (Tiers 0,1,2,3) Storage Media Type (Flash/Disk/Tape) Policy Engines (Workload Mgmt) Transparent Migration (Data Placement) Source: IMEX Research SSD Industry Report ©2011 File Virtualization (Uninterrupted App.Opns.in Migration) SSD Technology – Filling Price/Perf Gaps Price $/GB CPU SDRAM DRAM getting DRAM NOR Faster (to feed faster CPUs) & Larger (to feed Multi-cores & NAND Multi-VMs from Virtualization) PCIe SCM SSD HDD SSD segmenting into SATA PCIe SSD Cache Tape SSD - as backend to DRAM & SATA SSD HDD becoming - as front end to HDD Cheaper, not faster Source: IMEX Research SSD Industry Report ©2010-12 Performance I/O Access Latency 16 SSD in SAN Storage - Perf & TCO SAN StorageSAN Performance – Performance & TCO Improvements using SSD Improvements using SSD SAN TCO using HDD vs. Hybrid Storage 300 10 250 IOPS $/IOPS 9 250 Improvement Improvement 8 200 475% 800% 7 200 145 6 150 HDD-FC 0 150 5 IOPS $/IOP HDD- 36 Cost $K 4 100 SATA 100 0 3 SSD 64 2 50 50 75 1 RackSpace 28 0 0 14.2 Pwr/Cool 0 5.2 FC-HDD Only SSD/SATA-HDD HDD Only HDD/SSD Performance (IOPS) $/IOP Power & Cooling RackSpace SSDs HDD SATA HDD FC Source: IMEX Research SSD Industry Report ©2010-12 17 SSD Challenges & Solutions: Goals & Best Practices Concerned about SSD Adoption in your Enterprise ? 20% Be aware of Tools & Best Practices … And you should be OK !! 15% Best Practices 10% – By leveraging Error Avoidance Algorithms, and Best Practices of Verification Testing, to keep total functional failure rate <=3% (with defects and 5% wear-outs issues combined) – In practice, endurance ratings are likely to be 0% % of Drives Failing (AFR Failing % of (AFR Drives %) significantly higher than typical use, so data errors 0 1 2 3 4 5 JEDEC and failures will be even less. Years of Use SSD – Capacity Over-provisioning will provide large Std. increases in random performance and endurance. <=3% – Select SSD based on confirmed EVT Ratings – Use MLC within requirements of Endurance Limits Using Best-of-Breed Controllers to achieve <=3% AFR and JEDEC Endurance Verification Testing should allow Enterprise Capabile SSDs Source: Intel IDF’10 & IMEX Research SSD Industry