An Economic Study of the Hyperscale Data Center Ericsson Hyperscale Datacenter System 8000 Transforms the Economics of the Ente

An Economic Study of the Hyperscale Data Center

Ericsson Hyperscale Datacenter System 8000 transforms the economics of the enterprise data center, enabling a new generation of hyperscale architecture

January 2016 An Economic Study of the Hyperscale Data Center White Paper

TABLE OF CONTENTS

Executive Summary 3

Welcome to the Networked Society 4

Transforming the Enterprise Data Center 4

Virtualization and Software-Defined Infrastructure 5

Hyperscale Pioneers 6

Ericsson Hyperscale Datacenter System 8000: Hyperscale Data Center Architecture for the Enterprise 6

Higher Utilization 7

Optimized Data Center Infrastructure Management: From “Procure-to-Provision” to “Pool-to-Provision” 9

Business Impact and TCO Analysis 10

Operating Expense (OPEX) Savings 10

Capital Expenditure (CAPEX) Savings 12

Revenue and Other Strategic Impacts 13

5-Year TCO Impact for Enterprise Data Centers 13

TCO and ROI Summary 14

TCO Model Approach and Assumptions 15

The Journey to Hyperscale 16

About This Study 17

About Mainstay 17

Appendix 18

Endnotes 19

EXECUTIVE SUMMARY

This paper examines the economics and performance of a new data center infrastructure solution from Ericsson called Ericsson Hyperscale Datacenter System 8000 that is poised to reshape how enterprise data centers are architected to meet the challenges of the hyper-connected Networked Society. Based on the paradigm-shifting Intel® Rack Scale Architecture, Ericsson Hyperscale Datacenter System 8000 delivers a hyperscale cloud solution for enterprise data centers that will enable quantum leaps in efficiency, infrastructure elasticity, and economics.

Our research found that companies implementing Ericsson’s hyperscale platform can capture significant TCO savings compared to traditional data center infrastructures — generating CAPEX savings of up to 55%, OPEX savings of up to 75%, and a return on investment (ROI) of up to 138% for large enterprise data center operators over a five-year period.

Ericsson Hyperscale Datacenter System 8000 generates these savings by delivering performance and scalability on par with data center leaders such as Facebook, Google and Amazon in a commercially available platform. By creating a pool of infrastructure components leveraging Intel’s new RSA technology, SDI techniques, and optical networking, data center operators can better match infrastructure capacity with application and network workload requirements, thereby improving utilization of these resources. In this white paper we detail the range of value drivers that support these results and estimate the impacts across different data center scale scenarios.

In an economy where companies face exponentially rising data traffic and demands for new cloud computing capabilities, data center operators need to take a close look at new ways to provide IT services that are more flexible, cost-effective and vastly more scalable than what they offer today. Operators need to ask themselves whether they are ready for the Networked Society.

WELCOME TO THE NETWORKED SOCIETY

We live in a world where five billion people are connected and where mobile, broadband, and cloud are transforming the fabric of society. Ericsson calls it the Networked Society and it is impacting our everyday lives in profound ways, not unlike the titanic shifts of the industrial revolution more than a century ago. Today, Internet connectivity and mobile communications are taken for granted.1 And with the rise of the Internet of Things, more products are becoming “smart” and connected with other things across an expanding global cloud network. Meanwhile information streams from these networks are proliferating, spawning opportunities to harness Big Data for competitive advantage as well as social progress.

CHALLENGES FACING The world’s data traffic is exploding as a result. Analysts estimate that global workload growth will DATA CENTERS increase by 81% between 2015 and 2018, with the highest growth (113%) being cloud-based work- TODAY loads. And Ericsson is currently forecasting compounded annual growth of mobile traffic of 45% • Scaling to support between 2015–2021.2 All of this is placing enormous pressure on businesses and data centers to keep exponential growth of pace. Indeed, one of the most critical challenges facing enterprise data centers in the next few years application and network will be how to scale their computational capacity to meet exponentially growing digital workloads workloads with minimal in an economically efficient way. additional costs to the business (OPEX/CAPEX) To thrive in the Networked Society, businesses will need to seize fast-emerging digital business • Transforming service opportunities that require immediate deployments of compute, storage and networking resources. capabilities from back Demand can also recede or shift to new projects just as quickly, so enterprise data centers need to be office operations to a able to turn capacity on and off at short notice. This means data centers will need to provide business strategic revenue driving, stakeholders with the same “infrastructure on tap” capabilities that modern cloud-service providers customer-facing asset like Amazon Web Services have pioneered.

• Increasing flexibility and Indeed, as McKinsey & Company has observed, one of the top priorities for enterprise data centers speed of delivery of will be establishing a commercial-style business relationship with internal customers. This will require infrastructure services to an ability to standardize the data center’s service offerings, provide a solid “bottom-up” service cost meet the dynamics of digital industrialization structure, implement robust infrastructure management solutions, and reinvent the way they serve customers.3

TRANSFORMING THE ENTERPRISE DATA CENTER

The great majority of enterprise data centers today are ill-equipped to help businesses succeed in the Networked Society. Designed primarily to deliver back-office applications to internal business users, the traditional data center is burdened with an inefficient and costly architectural design that prevents it from scaling and flexing rapidly to meet the changing workloads of a modern digital enterprise.

One of the biggest design inefficiencies of the conventional data center is the creation of separate silos of hardware and software for each application workload. Because resources are not shared between workloads, investments in capacity buffers (over-provisioning) and high availability need to be made separately for each workload, adding to the overall data center cost burden.

Figure 1. Traditional heterogeneous application workload management

High Availability Buffer Overprovisioned Overprovisioned Overprovisioned Overprovisioned Capacity Capacity Capacity Capacity

Required Required Required Required Long Procure-to-Pay Capacity Capacity Capacity Capacity Process per Workload Creates Overprovisioning

Application Application Application Application Workload Workload Workload Workload for BU 1 for BU 1 for BU 2 for BU 2

Increases Proportionally Reduces hardware capital increases operating resource expenses expenses utilization

Moreover, because traditional data centers typically follow a lengthy “procure-to-provision” process tied to the business’s annual capital investment cycle, system architects will typically over-provision the infrastructure to weather these lengthy procurement cycles. Enterprise architects we interviewed said they typically design 30% over-capacity for a standard workload and up to 60–70% for mission- critical workloads.

Adding to data center complexity and cost is the wide variety of proprietary infrastructure components associated with each application stack. The bill of materials includes database appliances, storage solutions, servers, and networking solutions — all procured on a per-application, per-department basis. Heterogeneous environments like this always require more people to administer, adding to OPEX.

Virtualization and Software-Defined Infrastructure

To address these inefficiencies, the traditional data center is evolving. Server virtualization technology has allowed system administrators to have more flexibility in provisioning infrastructure resources. According to Forrester Research, 77% of x86 servers in enterprise data centers have been virtualized.4 Analysts estimate that virtualization has lifted x86 server utilization from 5–10% to over 30%.5 SOFTWARE-DEFINED INFRASTRUCTURE The success of server virtualization has opened the door to virtualizing other data center components, (SDI) such as network and storage, and leading to the next wave of enterprise data center optimization — Software-defined infrastructure software-defined infrastructure (SDI) — which seeks to virtualize the entire data center infrastructure. is the definition of technical Accelerating the spread of this new architecture is an industry-standard technology blueprint developed computing infrastructure by Intel called Intel® Rack Scale Architecture (RSA). entirely under the control of software. It operates Intel RSA is a logical architecture that disaggregates compute, storage, and network resources and independent of any hardware- introduces the ability to pool these resources for more efficient utilization of assets. It simplifies specific dependencies and are resource management and provides the ability to dynamically compose resources based on programmatically extensible. workload-specific demands.6

Hyperscale Pioneers

GOING HYPERSCALE Technology giants Google, Amazon, and Facebook were among the first to exploit the new architectural Hyperscale is the ability of paradigm, developing what are called hyperscale data centers — highly scalable platforms built to an architecture to scale by handle the massively complex workloads of the Networked Society. seamlessly adding compute, memory, networking, and These pioneering hyperscale platforms have set the modern standard for performance, efficiency, storage resources to and manageability. In fact, our research found that these custom-built hyperscale data centers, when accommodate increasing compared to traditional data centers, deliver four times the level of CPU utilization, increase network volumes of data and work- utilization by as much as 26%, and dramatically lift productivity by enabling a single IT administrator loads. Hyperscale computing to manage thousands of servers versus just hundreds, as shown in Figure 2. is often associated with cloud computing and the These efficiencies have enabled large public cloud providers to drive down data center infrastructure very large data centers and management costs, making it economically feasible to accommodate the massive computing and owned by Facebook, storage demands of the Networked Society. The next step in the evolution of the data center is to Google and Amazon. make hyperscale cloud architectures commercially available to enterprises worldwide.

Figure 2. Hyperscale data center efficiencies

Network Utilization1 CPU Utilization2 Servers per Admin Resource3 (Traditional vs. SDN Indexed) (Traditional vs. Google/Facebook) (Traditional vs. Google/Facebook) 5,000 1.26 65% 1.0

15% 300 Network CPU Servers/Admin

Traditional DC Hyperscale DC Traditional DC Hyperscale DC Traditional DC Hyperscale DC

26% improvement in 4x improvement in CPU Utilization 10x+ improvement Network Utilization (2x vs. virtualized workloads) in Resource Scaling

Sources: 1 Juniper Networks & Wakefield Research Survey for 400 U.S.-Based IT Decision Makers, 2014 2 Traditional DC benchmark based on 2014 Data Center Efficiency Assessment, NRDC; Google/Facebook benchmark based on extensive Mainstay and Ericsson research (see endnote #7 for a full list of sources) 3 Delfina Eberly, Director of Datacenter Operations, Facebook

Source: Mainstay Partners ERICSSON HYPERSCALE DATACENTER SYSTEM 8000: HYPERSCALE DATA CENTER ARCHITECTURE FOR THE ENTERPRISE

Recognizing the game-changing potential of Intel Rack Scale Architecture and software-defined infrastructure, technology leader Ericsson developed a data center solution — Ericsson Hyperscale Datacenter System 8000 — that brings the benefits of hyperscale computing to enterprises in a commercially available solution. For the first time, enterprises can deliver cloud computing performance and manageability on par with hyperscale leaders Google, Facebook and Amazon.

As discussed above, traditional data centers typically build dedicated infrastructures for each workload. Server virtualization has helped break down some of these silos, but in practice this technique can only be applied to a portion of enterprise workloads. Many enterprise applications, for example, are too costly to virtualize, while other legacy applications can present performance and tuning challenges

in virtualized environments. In addition, the performance of certain business-critical workloads — particularly high-speed, revenue-generating, and customer-facing applications — can suffer from the overhead latency effects of virtualization, and are excluded for that reason.

Ericsson Hyperscale Datacenter System 8000 overcomes these limitations by creating a pooled set of physical layer infrastructure components to support the virtualized and application layers, enabling businesses to flexibly support the dynamic demands of heterogeneous workloads.Administrators use management software — the HDS Command Center portal (or via APIs to an existing enterprise management system) — to define the optimal infrastructure components for each workload. Figure 3 compares a traditional data center architecture with the Ericsson platform, showing the advantages of deploying a centralized and homogenous bare metal layer of resources managed by a centralized software controls system (HDS Command Center) across all workloads.

Figure 3. Traditional data center vs. Ericsson Hyperscale Datacenter System 8000 higher utilization

Traditional Data Center Infrastructure Architecture

Legacy Specialized Business-Critical Workloads Workloads Workloads

• Siloed workload infrastructure OS OS OS • Heterogeneous bare-metal complexity Virtualization Virtualization Network Network Network • Virtualization only applicable for select workload segments Servers OS OS Servers Servers

Disk Network Network Disk Disk Servers Servers

Disk Disk

Ericsson Hyperscale Datacenter System 8000 Infrastructure Architecture

Legacy Specialized Business-Critical Workloads Workloads Workloads

IaaS (e.g., Openstack, VMWare) • Pooled workload infrastructure • Homogeneous bare metal components Command Center Portal • Utilization improved across workloads

Operating System Operating System

Pooled Hardware Components CPU Memory Disk Network

Higher Utilization

The ability to rapidly match workloads with hardware components in a common pool of bare-metal infrastructure enables data centers to increase the average utilization of components. Our research indicates that system administrators could routinely maintain server utilization rates of approximately 60% across most workloads.7 Disk Disk

We found three critical drivers of utilization improvements. By replacing the “procure-to-provision” planning process with a streamlined, software-defined “pool-to-provision” planning process (see next section), system administrators can better match infrastructure to workload demands, removing the overprovisioning allowance that is prevalent in current workload planning decisions. In our discussions with data center experts, we found this driver could improve utilization by up to 60%. Another utilization booster is the ability to dynamically allocate CPU resources across servers and racks, allowing administrators to quickly migrate resources to address shifting demand. We found this can drive 100–300% greater utilization for virtualized workloads and 200–600% for bare-metal workloads.

Finally, HDS Command Center’s open platform approach allows data center operators to manage their entire data center infrastructure (other vendor solutions as well as Ericsson) through a single portal. This enables system administrators to identify opportunities to allocate infrastructure to multiple workloads, improving utilization by up to 50%. For example, infrastructure used to support Workload A, which experiences peak demand during the day, could be shifted to Workload B, which experiences peak demand during the night. The visibility and reporting capabilities of HDS Command Center also allow operators to audit all infrastructure components, enabling them to locate under-utilized “ghost servers” (those not procured or managed by IT) and bring these resources under IT management. By leveraging the management and reporting capabilities of HDS Command Center, data center operators could improve utilization by up to 10%, our research shows. Figure 4 summarizes the range of utilization improvements that could be achieved by implementing the Ericsson Hyperscale Datacenter System 8000 platform.

Figure 4. Summary of Utilization Improvements

Utilization Ericsson Hyperscale Virtualized and Improvement Datacenter System Bare Metal Workload Driver 8000 Impact Improvement Improvement Driver Description “Pool-to- Up to 60% Virtualized Impact — Traditional “procure-to-provision” processes Provision” 0.25X improvement combined with high availability and other per- Planning in utilization formance considerations drive overprovision- Process ing for infrastructure. The Ericsson system Bare Metal Impact — Impact streamlines this process, creating a “pool-to- 0.5X improvement provision” capability, including auto-provision- in utilization ing, that better matches infrastructure requirements with workload demands. CPU 1X – 3X for virtualized Virtualized Impact — HDS Command Center software-defined Re-provision- servers 1.5X improvement infrastructure management enables simple ing and in CPU utilization CPU re-assignment, maximizing the utilization Infrastructure 2X – 6X for bare-metal of available resources across workloads Tuning (non-virtualized) Bare Metal Impact — (e.g., Workload A is not meeting planned servers 3X improvement demand metrics so infrastructure is moved in CPU utilization to Workload B). CPU Multi- Up to 50% Virtualized Impact — With the Ericsson system’s greater automation, Provisioning 0.25X improvement monitoring, and resource discovery capabili- in utilization ties, ghost servers can be eliminated. Ghost Servers Up to 10% The hyperscale platform also allows for Discovery and Bare Metal Impact — re-allocation of resources across multiple Avoidance 0.5X improvement workloads that are migrated based on need in utilization (e.g., Workload A during peak daytime hours; Workload B during peak nighttime hours). AVERAGE 2X (virtualized) Based on the drivers above, we modeled the median of each factor, which gets us to 4X (non-virtualized) approximately 2X CPU utilization improvement for virtualized workloads and 4X CPU improvement for non-virtualized workloads *Please note that all these factors will vary based on the unique operating practices of the enterprise data center.

By contrast, server utilization rates for traditional data center workloads typically range between 6% and 15%, with hardware virtualization benchmarks averaging closer to 30%.8 Whether applications are virtualized or running on bare metal, the ability to pool infrastructure at the component level using the Ericsson Hyperscale Datacenter System 8000 platform creates a homogeneous data center infrastructure across all workloads.

Optimized Data Center Infrastructure Management: From “Procure-to-Provision” to “Pool-to-Provision”

Ericsson’s software-defined data center platform will modernize the way operators manage infrastructure planning and procurement. The focus of planning cycles, for example, will move from procuring infrastructure for individual workloads (“procure-to-provision”) to managing the disposition and deployment of the data center’s entire of pool of hardware (“pool-to-provision”).

This shift to more holistic planning will eliminate traditional over-provisioning practices and enable rapid reallocations based on actual usage patterns. To effectively manage the new planning approach, operators may find they need to modify their organizational structures somewhat, relying less heavily on individual component solution specialists (server, storage, and networking experts) and more heavily on cross-component designers and SDI experts.

INSIDE THE ERICSSON HYPERSCALE DATACENTER SYSTEM 8000 PLATFORM Ericsson’s new platform introduces unique capabilities to enable companies to build a software-defined hyperscale data center.

• Resource pooling. Leveraging RSA, Ericsson’s platform reinvents the way system administrators provision, manage and monitor data center infrastructure assets through the pooling of infrastructure components. Resource pooling results in significant improvements in asset utilization.

• HDS Command Center. This system management portal gives IT administrators full visibility into all data center infrastructure elements (including non-Ericsson system elements via APIs) and enables composing a system using pooled resources that include compute, network and storage based on workload requirements. Command Center also provides auto-provisioning, real-time performance tracking, and triggers to adapt to workload shifts.

• HDS Optical Backplane. Ericsson’s state-of-the-art optical interconnect removes the traditional distance and capacity limitations of electrical connections, enabling maximum pooling of resources and flexibility in physical placement of data center assets, while increasing the lifespan of racking chassis and cabling by 3-4x.

BUSINESS IMPACT AND TCO ANALYSIS

What specific benefits and savings can businesses expect to achieve if they implement the Ericsson Hyperscale Datacenter System 8000 platform? Our analysis, based on extensive discussions with data center engineers and industry experts, estimated a range of impacts spanning three key business-value categories, as summarized in Figure 59: Figure 5. Ericsson Hyperscale Datacenter System 8000 Economic Value Tree: Impact Categories

• System administration savings via Command Center workload lifecycle management Operating Expense • Power savings from improved asset utilization, density, PDU architecture (OPEX) Savings • Optical backplane wiring and lifecycle cost savings • Server/CPU consolidation and improved utilization reduces component costs • SDN Architecture + Ericsson hyperscale networking solution reduces cost per port Capital Expense • SDS Architecture + Ericsson hyperscale storage solution reduces total capacity required (CAPEX) Savings • Facility infrastructure density improvement • SDI agility to migrate infrastructure reduces total capacity requirements • Enable revenue generating services Revenue and • Deliver Networked Society strategic opportunities Other Strategic • Create “infrastructure on tap” or “AWS-like” asset nimbleness Impacts • Improve “green” data center measures

Estimates of TCO and business impact will vary depending on the existing maturity and operating practices of the enterprise data center being studied. To simplify our analysis, we used third-party benchmarks to estimate the current operating practices of data centers today. For example, virtualization of enterprise data centers has helped to drive additional cost savings and higher efficiencies but our research has found that the range of virtualization can vary from 25% to 75% of data center workloads.

Operating Expense (OPEX) Savings

Our analysis showed that the Ericsson Hyperscale Datacenter System 8000 solution offers clear opportunities for reducing operating costs. These drivers of OPEX savings can be grouped into five categories, as shown in Figure 6 and summarized below.

Figure 6. Ericsson Hyperscale Datacenter System 8000 economic value tree — OPEX savings

Benefit Category Benefit Description Business Impact

Operating Deployment Optical Backplane simplifies deployment Expense Administrative Savings and increases rack lifespan by 3x+ (OPEX) Savings Simplified System SDI scales key management activities Admininstration Processes across all components

Reduced Cooling/Power Increases utilization + improves data center Consumption density, maximizing PUE

Reduced Software Workload efficiencies reduces License Support Costs software license support costs

Data Center Reduced Higher density and capacity utilization Footprint OPEX reduces facility costs

Benefit Category Benefit Description Business Impact

Capital Increased Capacity Utilization Increasing utilization of CPUs by pooling components of Server/CPU infrastructure and advanced capacity management practices Copyright © 2016 Mainstay, LLC Expense 10 (CAPEX) Savings Increased Storage Efficiency Increased storage efficiency gained from erasure of Unstructured Data coding storage dispersal management approach

Reduced Network Costs by Reduced spend by moving to a fabric/SDN solution Fabric-Based Approach increasing utilization with commodity components

Reduced Operating Reduced CPUs per workload lowers operating System Licenses system license purchase requirements

Reduced Virtualization Licenses Reduced CPU per workload lowers the virtualization (e.g., Hypervisor) software license purchase requirements

Strategic Impacts An Economic Study of the Hyperscale Data Center White Paper

• Physical deployments savings. Ericsson’s innovative racking Figure 7. Ericsson Hyperscale system, combined with the efficiency improvements delivered by Datacenter System 8000: HDS, can significantly reduce infrastructure physical deploy- Physical deployment cost savings ment costs per year. Our analysis showed that these savings Physical Deployment Costs* (60-Rack Data Center — annual spend) could reach as high as 74% over traditional physical deployment 1.00 annual costs, as shown in Figure 7. The savings are driven by the platform’s advanced optical backplane technology, as well as greater CPU utilization which together dramatically reduce the hardware and the labor required to build and maintain racks and

increases the average lifespan of racks from about 3 years to 10 0.26 years or more. Further, Ericsson’s high-utilization, component-

based architecture changes the traditional approach of re-racking Traditional Ericsson and re-cabling after each systems-replacement cycle (every 3 to Hyperscale * Assumes a 60-rack scale data center with 5 years). The Ericsson Hyperscale Datacenter System 8000 electrical, standard rack architecture; platform allows data center operators to maintain the existing Savings derived from: 1. Increased lifespan of Optical Backplane; 2. Reduced footprint from optical backplane rack and cabling and replace component sleds improved capacity utilization; 3. Reduced footprint from less over-provisioning; and into that racking system, reducing effort and minimizing costs. 4. Simplified cabling requirements.

• Simplified system administration. Companies, especially large Figure 8. System administration Source: Mainstay Partners enterprise-scale data center operators, will realize significant management savings

labor productivity savings using Ericsson’s command center Infrastructure System Administrators* software to streamline the management of data centers. Common (at 60-Rack Data Center) system administration tasks such as monitoring, notifications, 35 quota management and resource allocation, ghost server discovery, and provisioning across workloads are now automated and orchestrated by HDS Command Center. Further, because the data center is now managed as a homogeneous infrastructure, fewer component and vendor experts are required. As a result, our research indicates that large data center operators can expect to 2 Traditional Ericsson reduce system asset management tasks and reduce FTE require- Hyperscale ments by as much as 94% for managing storage, server and * Assumes Ericsson system best practices, 10 unified infrastructure simplification, and networking environments, as shown in Figure 8. SDI benefits are achieved

• Power and cooling savings. Our research shows that Ericsson’s high-utilization platform will enable data centers to operate with substantially less physical infrastructure, as detailed in the section on CAPEX savings. Based on these estimates, the average data center could reduce overall power and Source: Mainstay Partners cooling costs up to 72%.11

• Reduced software licensing and support costs. As organizations boost component utilization and cut back on data center hardware — particularly CPUs — the cost of software licensing would fall proportionally. Based on our earlier estimates, companies could realize as much as a 30% reduction in ongoing virtualization software license costs.12 However, these savings are offset by software license investments in HDS Command Center Software. Our analysis found that this TCO category would be a net investment for operators.

• Lower real estate costs. In most cases, real estate cost savings will take the form of cost avoidance as companies find they can accommodate growth more efficiently and thus use existing data center space longer. Ericsson’s optical backplane also facilitates better space utilization, allowing for more flexible

Benefit Category Benefit Description Business Impact

Operating Deployment Optical Backplane simplifies deployment Expense Administrative Savings and increases rack lifespan by 3x+ placement of(OPEX) racks. Our analysis estimates that by switching to Ericsson’s hyperscale infrastructure, Savings Simplified System SDI scales key management activities the typical enterprise could reduceAdmininstration its data Processes center footprint by up to 50%.across all components

Capital Expenditure (CAPEX)Reduced Savings Cooling/Power Increases utilization + improves data center Consumption density, maximizing PUE

Ericsson’s hyperscale cloud solutionReduced can helpSoftware companies significantlyWorkload reduce efficiencies or avoid reduces data center capital expenditures as each componentLicense Support of the Costs data center becomes softwaremore efficient license support and costs easily scalable.

Figure 9 details the range of capitalData savings Center Reduced companies can potentiallyHigher density realize. and capacity utilization Footprint OPEX reduces facility costs Figure 9. Ericsson Hyperscale Datacenter System 8000 economic value tree — CAPEX savings

Benefit Category Benefit Description Business Impact

Capital Increased Capacity Utilization Increasing utilization of CPUs by pooling components Expense of Server/CPU infrastructure and advanced capacity management practices (CAPEX) Savings Increased Storage Efficiency Increased storage efficiency gained from erasure of Unstructured Data coding storage dispersal management approach

Reduced Network Costs by Reduced spend by moving to a fabric/SDN solution Fabric-Based Approach increasing utilization with commodity components

Reduced Operating Reduced CPUs per workload lowers operating System Licenses system license purchase requirements

Reduced Virtualization Licenses Reduced CPU per workload lowers the virtualization (e.g., Hypervisor) software license purchase requirements

• Server CAPEX savings. As enterprises shift workloads to Ericsson’s software-defined infrastructure, they can expect to reduce capital expenditures on servers (CPUs) in line with our earlier 2-4x utilization improvement estimate. In estimating the total savings potential for an enterprise data center with Strategic 75% virtualizedImpacts workloads, we calculated a 65% TCO savings for server-related CAPEX spending. Companies can also economize on CPU costs by optimizing how they are deployed in the hardware pool. For example, as a CPU ages and newer components are brought online, administrators can match older components to less critical workloads, extending the average life of a CPU from three to five years, according to data center experts we interviewed. Many companies will see additional savings in lower software license costs tied to CPU cores or sockets.

• Storage savings. By employing an advanced erasure coding storage Figure 10. Traditional vs. algorithm, Ericsson Hyperscale Datacenter System 8000 storage Ericsson hyperscale platform’s minimizes the replication of unstructured data across the data center, unstructured data storage enabling cost reductions in disk storage by as much as 35% (Figure Amount of Storage Required for Similar Workloads in Petabytes 10). In addition to lowering physical disk CAPEX, the enterprise 3.6 should also see reductions in storage-related software license costs.

• Network CAPEX savings. Ericsson’s software-defined networking topology, combined with the physical advantages driven by its optical 1.7 backplane racking solution, will increase the overall efficiency and effectiveness of the data center’s physical switch infrastructure, enabling capital cost reductions in this area. In addition, our analysis Traditional Ericsson Hyperscale

Source: Mainstay Partners An Economic Study of the Hyperscale Data Center White Paper

shows that network-layer savings will increase exponentially as the scale and complexity of the data center increases.13

• Power and cooling equipment savings. A side benefit of migrating to Ericsson’s streamlined infrastructure is a corresponding reduction in the amount of power backup systems and cooling equipment required. Since these savings are likely to represent only a small portion of overall cost savings, they are documented here for completeness but not included the TCO calculations that follow.

Revenue and Other Strategic Impacts

Although the specific purpose of this study was to detail the potentialTCO savings of the Ericsson hyperscale platform, our research and discussions with data center and industry experts also uncovered a range of strategic and revenue-generating opportunities that enterprises could capture when they are equipped with hyperscale data center capabilities. Although the sheer diversity and unpredictability of these opportunities makes it impossible to quantify for the average company, they may well represent the greatest potential source of value for many enterprises, especially those seeking business transformation through digital initiatives.

For example, a hyperscale platform like Ericsson’s is ideally suited for enabling IoT and Big Data initiatives, both of which require massively scalable compute, networking and storage capabilities. McKinsey estimates that the IoT could have a total economic impact of $11.1 trillion per year by 2025,14 the equivalent of 11 percent of the world economy. However, to exploit IoT opportunities, data centers will need to bring new capabilities into play, such as enabling interoperability and the ability to analyze huge volumes of data. Ericsson Hyperscale Datacenter System 8000 makes all of this possible and at a lower cost basis than the competition. For “hyperscale-enabled” companies, billion-dollar opportunities become addressable markets.

Other capabilities enabled by the Ericsson hyperscale platform, such as rapid workload provisioning, optimal hardware utilization, and better infrastructure visibility and manageability, will enable enterprise data centers to provide services comparable to what’s available from public cloud providers such as AWS. Furthermore, by increasing data center density, conserving space, and improving power usage effectiveness, or PUE, the Ericsson hyperscale platform will help companies meet environmental sustainability goals, an important strategic priority for many companies.

5-YEAR TCO IMPACT FOR ENTERPRISE DATA CENTERS

What is the total economic benefit enterprises could gain from implementing Ericsson Hyperscale Datacenter System 8000 in a typical data center? To estimate the impact, we created a total cost of ownership (TCO) model, factoring in the range of value drivers described in the previous section. The model analyzes operating scenarios for three sizes of data centers: small (1 rack), medium (6 racks), and enterprise-scale (60 racks), recognizing that infrastructure components can vary greatly in each case depending on industry, geographic coverage, the types of applications being run, and other factors.

TCO and ROI Summary

The results of our TCO analysis are clear: Enterprise data centers that adopt the Ericsson Hyperscale Datacenter System 8000 platform can expect to capture significant savings — as much as 55% in CAPEX savings and 75% in OPEX savings for a 60-rack data center (see below for more details on our TCO modeling), with an estimated overall TCO savings of 63%. This translates into an estimated ROI of 138% over five years.

One of the key drivers of TCO savings is greater CPU utilization, which allows for the deployment of fewer servers and CPUs, less network hardware, and less storage disk. The Ericsson platform’s ability to transform the “procure to provision” process into a vastly more efficient “pool to provision” process enables more frequent, collaborative planning efforts that help avoid over-investment in infrastructure. For these utilization and labor efficiencies to be fully realized, however, organizational changes may be required to facilitate collaborative planning across departments and optimize compute, storage and network capacity.

Ericsson’s Optical Backplane solution drives more efficient cabling, resulting in fewer cables procured and ease of physical deployment resulting in lower OPEX spending. The automation and orchestration with HDS Command Center, results in being able to scale up the amount of infrastructure managed per systems administrator. Less infrastructure additionally reduced power and cooling, software licenses and hardware and software support costs.

Figure 11 summarizes the estimated magnitude of cost savings and ROI that could be achieved at different scales of deployment. Figure 12 shows a more detailed breakdown of savings by data center cost category.

Figure 11. Ericsson Hyperscale Datacenter System 8000: Summary of estimated savings and ROI (five years)16

Scale of Deployment Financial Category 60 Racks 6 Racks 1 Rack CAPEX Savings 55% 53% 50% OPEX Savings 75% 62% 0.4% Total Cost of Ownership (TCO) Savings 63% 57% 26% Return on Investment 138% 109% 29%

Figure 12. Full comparison of TCO cost elements

Scale of Deployment Cost Element 60 Racks 6 Racks 1 Rack Hardware CAPEX 59% 57% 53% Software CAPEX 30% 29% 29% Hardware OPEX 44% 41% 35% Software OPEX* 30% 29% 29% System Administration OPEX 94% 75% 0% Power, Cooling and Space OPEX 62% 62% 36% *Software OPEX does not include investment in licensing costs for HDS Command Center software.

Figure 13 shows the 5-year economic impact of moving to Ericsson Hyperscale Datacenter System 8000 at a hypothetical 60-rack data center currently operating in a non-hyperscale environment. Figure 14 depicts the relative percentages of each type of savings. See Figure 15 in the Appendix for additional notes and calculations on each category of savings and investment cost.

Figure 13. 5-Year TCO Impact Summary (60-rack data center)

MORE CAPACITY… $63,565,634 Assuming equivalent hardware, Ericsson Hyperscale Datacenter $19,878,060 System 8000 can deliver $1,326,353 $476,176 $84,887 double the compute capacity of a traditional data center. $23,551,384 $16,500,000 $3,953,774 $2,205,000 AND LESS TCO… A 25% improvement in data center asset utilization Current Hardware Software Hardware Software System Power, Cooling HDS Ericsson (non-HDS) Capital Capital Support Support Administration and Space Investment Platform can translate into a 10% TCO Cost Savings Cost Savings Cost Savings Savings Effort Savings Cost Savings TCO TCO savings. Savings Investment Note: the Ericcson Hyperscale Datacenter System 8000 is abbreviated as “HDS” in this table.

Figure 14. Ericsson Hyperscale Datacenter System 8000: Savings by cost category

Hardware Capital 47% System Administration 39%

Power, Cooling and Space 10% Software Support <1% Software Capital 3% Hardware Support 1%

TCO Model Approach and Assumptions

To establish realistic assumptions and benchmarks used in the model, we worked closely with data center and industry experts, including operators at Fortune 500 companies, and took a conservative view of the potential cost savings of the Ericsson hyperscale platform. For example, wherever possible, we have assumed a conservative basis for calculations, such as using discounted “market” prices for hardware and other components.

For the same reason, we chose not to include the potential revenue or strategic benefits that might result from providing the enterprise with greater agility, speed or environmental sustainability, as outlined in the previous section. Although these benefits could be substantial, even exceeding the other categories of benefits, we judged them to be too variable and unpredictable to include in theTCO model.

The relative percentage of virtualized vs. non-virtualized (or bare metal) workloads can vary on a case by case basis. We chose to present a conservative view of the impact by assuming that 75% of the workloads had already been virtualized, thus using a relatively efficient baseline for our analysis. Based on our research, we set the average CPU utilization in non-virtualized environments at 15%, and virtualized environments at 30%. Because of the Ericsson platform’s superior resource pooling and disaggregation capabilities, its utilization rate was set at 60%, which is consistent with performance levels achieved at leading hyperscale data centers.

The TCO analysis covers a 5-year period and factors in the depreciated costs of the infrastructure. The following CAPEX elements were considered: server hardware, storage HW, network hardware, cable hardware, physical deployment effort, virtualization software, management software. The following OPEX elements were considered: hardware support, software support, HDS Command Center software support, system administration, power and cooling, and space utilization.

Our analysis assumed a greenfield approach to implementing the Ericsson platform. In our model, the compute-storage-network environment is completely converted to the Ericsson hyperscale platform on Day One and compared to a traditional, non-hyperscale operating model. Since the infrastructure is assumed to be fully-depreciated, there is no time-lag or phasing of impacts. A phased approach model would need to be created for cases that are not greenfield implementations. Our recommendation is to use the framework, benchmarks and research provided in this paper as a “starter kit” to develop a custom ROI analysis that takes into account the unique features and opportunities of your individual data center.

THE JOURNEY TO HYPERSCALE

For most enterprise IT organizations, moving to an Ericsson hyperscale platform won’t happen overnight. In reality, it will be more of a journey than a flip of the switch. Enterprise data center management will be tasked with not only transforming the physical architecture of the data center, but also changing work processes and job descriptions in order to maximize TCO savings — a process that will likely take months, if not years.

EARLY RESULTS: GOING HYPERSCALE AT ERICSSON’S STATE-OF-THE-ART DATA CENTER Ericsson is committed to being its first and best customer with its hyperscale data center platform. As part of its 2016 IT planning efforts, Ericsson has prioritized its engineering systems infrastructure as the best place to start. In our interviews, IT executives explained that the Ericsson Hyperscale Datacenter System 8000 solution was particularly well suited for the engineering systems environment given the very high CPU demand of its application workloads.

Executives said that the platform’s HDS Command Center will help the IT team more effectively manage the infrastructure and meet the unique requirements of its engineering applications. With Ericsson’s hyperscale platform, the IT team is expected to achieve an estimated 400% improvement in system utilization across more than 30,000 servers.

The team expects to launch the hyperscale project in 2016 and then expand the platform’s capacity over several years. Results from the project will be analyzed and shared widely within the company to help improve the product for internal users as well as Ericsson’s customers.

In most cases, enterprises will be best served by following an incremental approach to hyperscale transformation. That could mean starting with a pilot on a selected workload in order to demonstrate the benefits of pooling resources and delivering “infrastructure on tap.”Another initial step, which Ericsson itself took internally, is to leverage the audit capabilities of the HDS Command Center to gain full visibility into the utilization of the existing infrastructure and find dormant or shadow infrastructure that may be unknown or poorly leveraged. This activity can provide an early return by eliminating dormant or underproductive assets.

After successfully proving the platform’s capabilities in the pilot, most organizations will be ready to plan for a wider rollout. Ericsson, for example, decided to leverage the visibility gained from its HDS Command Center audit to identify areas of the data center that could benefit most from the hyperscale platform and then start the deployment there (See callout: “Early Results: Going Hyperscale at Ericsson’s State-of-the-Art Data Center”).

Capital depreciation schedules will likely influence the pace of change, since most companies will want to avoid write-offs of existing equipment and prefer to phase in the hyperscale platform only when legacy hardware has been fully depreciated. This financial constraint would suggest a three- to five-year journey for most enterprises.

Considering the comprehensiveness of Ericsson’s hyperscale platform and its potential for transforming the business, companies should take advantage of Ericsson’s experts in their planning efforts. This could include the development of a multi-year business case, a phased implementation approach, and a technical design evaluation that explores the range of opportunities for leveraging this next-generation data center architecture.

About This Study

Research and analysis for this study was conducted by Mainstay, the leading IT management consulting firm focused on quantifying and communicating the business value of technology. This study was based on reviews of primary research and interviews with business and IT executives at Ericsson and numerous industry leaders, including experts from one of the world’s largest systems integrators; a data center architect with experience in server and storage virtualization, provisioning and networking, and storage management; and an executive from a leading gaming and entertainment company with experience in data center virtualization and performance. TCO and ROI calculations use industry standard assumptions regarding the time value of money. For more information about the study’s methodology and model assumptions, see Figure 15 in the Appendix. Information contained in the publication has been obtained from sources considered reliable, but is not warranted by Mainstay.

About Mainstay

Research and analysis for this study was conducted by Mainstay, a leader in quantifying and communicating the value of business and technology investments. Mainstay serves as an independent advisor to eight of the top ten global high-tech companies.

Appendix

Figure 15. Comparison of non-HDS and HDS environments and cost calculation methodologies (Note: the Ericcson Hyperscale Datacenter System 8000 is abbreviated as “HDS” in this table.)

Non-HDS Servers HDS Servers Non-HDS environment has 60 racks, which have 900RU/servers CPU utilization improves by 2x for virtualized server workloads, (and 3600 ports). Price per server is multiplied by total number from 30% to 60% utilization. For non-virtualized the improve- of servers and depreciated. During refresh cycle, 20% of the ment is 4X, going from 15% to 60% utilization. The weighted server hardware can be reused, therefore 80% of new server average improvement is 2.5X for the 75% scenario modeled in cost is applied to year 5. this analysis. The 900 legacy servers are reduced to 394RU. For physical deployment efforts (racking/stacking), we considered Physical deployment of HDS requires 2 FTE x 5 hrs per server. 2 FTE x 10 hrs per server + installation + other hardware and Installation plus other hardware and software deployment software deployment activities (e.g., testing: 2 FTE x 70 hours per including testing is estimated at 2 FTE x 40 hours per physical physical server). These costs are depreciated in the model. server.

Non-HDS Storage HDS Storage Centralized storage, 7.2k HDD (72 TB / 1 RU), 250W Assuming that 70% of this storage environment is unstructured Centralized storage, 10k HDD (24 TB / 1 RU), 250W data, HDS software-defined storage provides 50% reduction in Centralized storage, SSD (40TB / 1RU), 250W replication of unstructured storage, resulting in a 35% savings. Total of 230RU make up 10 PB (i.e.,130 of SSD + 50 of 10K+ 50 of 7.2k)

Non-HDS Network HDS Network 96 port fabric interconnect switches, 950W. Total of 80 44 Ericsson leaf switches, 800W per switch plus EAM. switches, with an additional 8 routers for inter-rack routing.

Non-HDS Cables HDS Cables Standard 10G electrical copper cables: Ericsson optical cabling: Compute-Network: Assumes 4 10G ports are connected. CSU-NSU: Per server price per cable, assuming 4 10G ports 90U x 4 port x price per cable are connected. 394U x 4 port x price per cable Storage-Network: 23 U x 2 ports x price per cable SSU-NSU: 150U x 2 ports x price per cable Intra-Network: 8 U x 2 ports x price per cable NSU-NSU: 44U x 1.5 ports x price per cable

Virtualization Software: Non-HDS Virtualization Software: HDS Virtualization/Hypervisor Software (75% of the environment is Number of servers reduced to 394RU; this expense is reduced virtualized x 900 physical servers) priced per CPU. by fewer CPUs. 75% of environment x 900 servers x 2 CPUs.

Management and Orchestration: Non-HDS Management and Orchestration: HDS Assuming 15 virtual servers per physical server. Calculated at Expenses do not change (enterprise needs for the number of price per virtual machine. virtual servers does not change). Note: HDS Command Center costs are listed under OPEX.

Software and Hardware Support Costs: Non-HDS Software and Hardware Support Costs: HDS Assumed at 8% per year (first year $0) for each hardware cost Same calculation as non-HDS. above (server, storage, network). Additionally, HDS Command Center costs are considered at: Price per box x (394U + 150U + 44 U) per year.

Systems Administration: Non-HDS Systems Administration: HDS 1 FTE per 300 virtual servers. This represents server, storage Ratio is increased to 5000 virtual servers per FTE. and network administration composite resource estimation.

Power, Cooling & Space: Non-HDS Power, Cooling & Space: HDS Standard cost of power 10 c per KWH, 8736 hours total avail- Same calculation method, fewer server hardware and fewer able hours per year x total wattage of servers, storage and total number of racks reduces OPEX expenses for power, network. For space, each rack occupies 30 sq. ft. Data center cooling and space. space cost is $310 per sq. ft. Assumes 60 racks are occupied.

Sources for benchmarks: Recent research from Mainstay and Ericsson data center analysts and research publications from Gartner, VMWare, Uptime Institute and IDC.

Endnotes

1 Worldwide mobile data traffic is projected to grow at a compound annual rate of 57% for the next four years, reaching a throughput of 24.3 exabytes per month by 2019 (Source: Statistica: “Global data center IP traffic in exabytes per year, 2015). 2 “Ericsson Mobility Report: On the pulse of the Networked Society,” Ericsson, November, 2015. 3 “The enterprise IT infrastructure agenda for 2014,” McKinsey & Co, 2014. 4 “Strategic Benchmarks 2013: IT Infrastructure,” Forrester Research, May 2013. 5 Multiple sources: “The Economics of Virtualization: Moving Toward an Application-Based Cost Model,” IDC, Nov, 2009; “Chapter 20: Data Center IT Efficiency Measures,” National Renewable Entergy Laboratory, US Department of Energy, Jan 2015. 6 Intel Rack Scale Architecture is a set of architecture guidelines and principles that are being implemented by Ericsson and other vendors. 7 Based on interviews with leading SI data center experts, enterprise data center management, and publicly available Google and Facebook benchmarks (see list of sources below). Administrators may choose to maintain some critical workloads at lower averages to manage spikes, while maintaining higher rates for non-critical workloads. (Our research found that >65% utilization is possible for certain workloads.) http://www.epanorama.net/newepa/2011/04/10/facebook-datacenter-secrets/comment-page-1/ http://www.datacenterdynamics.com/content-tracks/servers-storage/how-facebook-deals-with-constant-change/83715.fullarticle http://www.singjupost.com/facebook-fb-presents-credit-suisse-2014-annual-technology-conference-transcript/ http://www.vmware.com/solutions/consolidation.html http://www.slideshare.net/dellenterprise/forrester-the-economic-impact-of-server-virtualization http://ieeexplore.ieee.org/document/6529276/?reload=true http://www.slideshare.net/sgurnam73/server-virtualization-by-vmware https://www.linkedin.com/pulse/grace-hopper-2015-jade-chu?articleId=6062981612944056320 https://turbonomic.com/blog/on-turbonomic/optimal-cpu-utilization-depends/ 8 Based on the diversity of workloads in data centers, we are assuming (based on available research) that a best-in-class data center may have as high as 60–75% of their workloads virtualized today. To be conservative, the analysis assumed 75% of the workloads are virtualized, leading to a 69% improvement in server utilization across the data center. 9 Given that the solution has not been released commercially at the time of this paper, all quantifications of impact are estimates based on available research and assumptions made in concert with the experts interviewed. 10 These savings are dependent on enterprise data center operators transforming the people, process and technology practices as described in this piece. 11 Assumes $0.10 per KWH @ 8,736 hours per year; wattage per component (storage, network, server) estimated based on available benchmarks; assumes rack occupies 30 sq. ft. of data center space at $310/sq. ft. 12 Data center infrastructure TCO includes software license costs and annual support costs for operating system software, virtualization software, disaster recovery software, and the like. 13 This is due to two major factors: (1) As the complexity and requirements for the network increase the costs of the switching environment grow exponentially; (2) The fabric-based network environment, coupled with the fiber-based racking environment, allow for more flexibility in how you physically deploy the network switches into the racking environment. Due to electrical connection limitations, typical top-of-rack switches are required to manage very dense environments. With the Ericsson hyperscale platform, these physical switches can be better utilized because of the distance and low-latency capabilities of optical connections. 14 Mckinsey Global Institute’s June 2015 survey. 15 Cost savings are estimates are based on full adoption the Ericsson Hyperscale Datacenter System 8000 solution as well as implementing new organizational best practices discussed earlier, such as holistic infrastructure planning that reduces hardware overprovisioning. 16 CAPEX includes: server, storage, network, cabling, power and cooling equipment hardware costs, and virtualization and orchestration software license costs. 17 OPEX includes: server, storage, network hardware support costs, virtualization and orchestration software support costs, system administration labor costs, power, cooling and space costs.