white paper

SoftBank Tests Next-Generation IT Platform Based on Intel® Rack Scale Architecture in Pursuit of Information Revolution

Intel® Rack Scale Architecture for Data Center Efficiency

Executive Summary SoftBank Corp. is pursuing an “information revolution” that will spur the latest innovations in technol- ogy, including the of Things (IoT), artificial intelligence (AI), and smart robots. The company needs the IT platforms which support its rapidly growing businesses to have agility to optimize real-time performance and high efficiency to utilize all available resources. The consolidation of its Japanese companies is what led SoftBank to consider the Intel® Rack Scale Ar- chitecture, which pools resources at the rack level and allocates them via software. During workload- based testing, a development test demonstrated a potential reduction in the number of servers, a great reduction in staff workload, and a significant electrical conservation. The test result also showed “Intel® Rack Scale Architecture is that both server and network delivery times and fault response times could be significantly reduced. Having concluded that Intel® Rack Scale Architecture is suitable as a next-generation platform for an integral piece of architecture real-time computing, SoftBank is now planning to test expansion functions and functional improve- for real-time computing at ments in preparation for the launch of Intel® Rack Scale Architecture. SoftBank as we accelerate our Consolidation of Japanese to give them the competitive advantage and global strategy including the IoT, Telecommunications Companies differentiate themselves, they must adopt Poses Challenge for IT Infrastructure a common IT platform to achieve synergies artificial intelligence, Consolidation between the four merged companies and improve efficiency. and smart robots.” • Testing Background Another factor behind SoftBank’s desire to In April 2015, SoftBank Mobile, SoftBank BB, – Mr. Amane Kito consolidate its IT platforms is to create an “in- SoftBank Telecom, and Y!Mobile were con- Senior Vice President, CISO and formation revolution”, a policy they pursue in solidated into a single new company, SoftBank Head of Information Systems Unit recognition of the important role that the IoT, Corp., that handles both mobile and fixed SoftBank Corp. AI, and smart robots will play in future growth. line telecommunications. This created a new In the near future, network connections will be organizational structure under which the bulk commonplace not only for mobile devices but of SoftBank Group’s Japanese telecommuni- also other devices such as industrial equip- cation businesses are consolidated. ment, robots, and different type of sensors, Mr. Amane Kito One issue that comes with the structural with a forecast of 50 billion online devices in Senior Vice President, CISO and reorganization is how to consolidate the IT 2020. This could lead to the emergence of a Head of Information Systems Unit platforms. While the business activities of steady stream of new IoT-based businesses, and in turn, the potential to change the fun- Mr. Masahiro Sekiya the four old companies have merged, they Senior Director retain the same IT infrastructure as before. damental role of these Internet “things.” If AI- System Infrastructure Division Furthermore, the gradual acquisition of a integrated smart robots enter widespread use, Information Systems Division variety of other companies has created a silo the need to enable robots to interpret human structure. Among the server, storage, and emotions by using large amounts of data will Mr. Toshiyuki Sugiyama rise. Accordingly, there will be a demand for System Infrastructure Development Section network hardware from different vendors and faster and more scalable computing resources System Infrastructure Division operating under different standards were a Information Systems Division range of general-purpose systems. In order to support the real-time processing and vi- SoftBank Tests Next-Generation IT Platform Based on 2 Intel® Rack Scale Architecture in Pursuit of Information Revolution

sualization of the high volume of transactions Intel® Rack Scale Architecture is one of the 5. Top-of-rack (ToR) switch arising from this proliferation of devices. architectures used to implement the concept 6. Computing resources drawer of software-defined infrastructure (SDI) being These are some of the considerations that • Each drawer was fitted with an Intel® Xeon promoted by Intel, which abstracts the vari- piqued SoftBank’s interest in Intel® Rack Scale or Intel® Atom™ processor module Architecture, a vertically integrated architec- ous hardware used at a data center, including • Fabric switch with Intel® Ethernet Switch ture for consolidating server, storage, and processors, memory, storage, and network- FM5224 silicon base network hardware at the rack level and inter- ing, and uses software to modify its setup. connecting them via high Ethernet Use of Intel® Rack Scale Architecture enables • Each drawer and ToR switch connected Fabric. Senior Vice President, CISO and Head hardware configurations to be modified as via 40GbE optical cable of Information Systems Unit of SoftBank, Mr. necessary. Whereas resources in the past were 7. Intel® Xeon® E5 v3 processor modules and Amane Kito, put it as follows. only available after servers, storage, or other Intel® Atom™ C2000 SoC modules (each devices were added individually, with Intel® “Intel® Rack Scale Architecture has the po- module has 12 Intel® Atom™ C2000 SoCs) Rack Scale Architecture, they can be made tential to dramatically improve the utilization 8. JBOD storage(object storage) available by software setting in much the same of computing resources that in the past have way as selecting a “component” from a pool not been efficiently utilized. SoftBank Group • Testing dates of available resources organized on the rack is currently working on a unified platform From May to September 2015 (effective test- level. Mr. Sekiya commented, “Because an for real-time computing called “Chronos.” ing time of 10 weeks) architecture like Intel® Rack Scale Architecture We have been conducting tests on an Intel® is such a groundbreaking development, this Rack Scale Architecture development system • Tests testing will have considerable impact on our because we expect it to play an important role SoftBank tested its basic performance and future IT infrastructure strategy.” in implementing this new platform. functions by using the web UI for resource “One of SoftBank Group’s strengths is that Prototype Evaluation System Tested monitoring; composing servers from the com- we are quick to adopt advanced Information over an Effective 10-Week Period pute pool; assigning storage and networking, and Communication Technology(ICT), and installing and booting the OS on the server; ever since our formation we have consistently • Testing environment installing and executing the application; and testing the Intel® Rack Scale Architecture been among the first to test promising new The system configuration used for testing API. The following tests were conducted with technologies. This attitude has allowed us to is listed below. The evaluation system was an emphasis on verifying whether the basic take the first steps toward implementation a prototype used by Intel for development, concept of Intel® Rack Scale Architecture had of advanced ICT by using the only evaluation with hardware put together from off-the- been achieved. system in , therefore learning about shelf components. Intel® Rack Scale Architecture before anyone 1. Intel® Rack Scale Architecture rack [Test 1] Intel® Rack Scale Architecture pod else. One of our future goals is global deploy- management function using Pod Manager ment of this system as a common global • 26U rack platform for SoftBank Group.” 2. Rack was fitted with a common power sup- Testing confirmed that Pod Manager, the con- ply trol center for Intel® Rack Scale Architecture could detect key components, such as com- Confirming Functions Provided by Intel® 3. Rack was fitted with a common cooling puting and storage on the rack and man- Rack Scale Architecture and Operation system under Actual Workloads age them centrally. It also confirmed when 4. Pod Manager (also incorporating rack man- computing resources were added or removed, • Testing Objectives agement functions) the change was automatically detected by the The aim of testing was to affirm that the • A small desktop PC kit (Intel® NUC) system and the resource was recognized as Intel® Rack Scale Architecture’s functions mounted at the bottom of the rack available or unavailable. can optimize resources in real time. The first Pod objective was to confirm that the proces- Pod Manager by domain Pod Manager sors, memory, storage, and network could be configured on the hardware platform. The Rack second was to verify the ability to execute the TOR RMC by rack RMC actual workload on top of the platform OS and Drawer

PCIe middleware used by SoftBank. This work also FM6324 PCIe CPP included testing network coordination and the switch by drawer PSME optimization of compute, storage, and other PCIe

SMBus resources to assess the effectiveness of the Xeon automation essential for the next-generation module PCH BMC by module Xeon SOC Xeon SoC DMI BMC MMC BMC MMC platform. Mr. Masahiro Sekiya, Senior Director, MMC Xeon uServer

System Infrastructure Division, Information DDR DDR DDR E5 DDR DDR DDR DDR module Systems Division, commented that, “The over- QPI riding objective of the testing was to confirm Avoton SOC DDR whether or not Intel® Rack Scale Architecture, Xeon ••• DDR DDR DDR DDR DDR DDR DDR DDR E5 M.2 Drive Avoton Compute

as a next-generation architecture, was suitable Blade Compute Avoton Blade Compute Avoton Blade Compute Avoton Blade Compute Avoton Blade Compute Avoton Blade Fig. 1 Hardware Configuration and in practice for deployment as a platform, and M.2 Drive Architecture of Evaluation System could be put to beneficial use.” SoftBank Tests Next-Generation IT Platform Based on 3 Intel® Rack Scale Architecture in Pursuit of Information Revolution

[Test 2] Assignment from resource pool The testing demonstrated the ability to per- using Pooled System Management Engine form provisioning using the configuration they Intel® Rack Scale Architecture Pooled System Management Engine is a man- set up, including networking, with resources agement tool for drawers. Testing confirmed in the rack being detected automatically, and Intel® Rack Scale Architecture is a that it could be used to allocate resources the required computing resources and storage reference architecture devised as part appropriately. assigned by software alone using the Web UI. of Intel’s work on its future data center The testing represents a major step forward strategy (software-defined infrastruc- [Test 3] Intra-pod storage management toward SDI, as Mr. Sekiya noted by saying, ture). A logical architecture that sub- function and allocation through integration “What is significant is that this is not virtu- divides processors, memory, storage, with distributed block storage (Ceph) alization, but an actual physical, bare-metal and networking to provide highly ef- Testing confirmed that storage disk capacity environment.” ficient resource pools, it is designed to could be allocated appropriately for the dis- That trouble-free execution of the workload allocate resources automatically and tributed storage (Ceph) used for testing. was achieved on a configuration in which dynamically in response to applica- the drawers had a mix of both Intel® Xeon® tion demand. Use of Intel® Rack Scale [Test 4] Division and allocation of network and Intel® Atom™ processor modules. This Architecture enables the implemen- resources demonstrated Intel® Rack Scale Architecture’s tation of a software-controlled data Testing confirmed that the pool of network re- responsiveness in permitting the reallocation center without a large investment. sources on the rack could be correctly divided of processor resources during use, even in Specifically, the processors, memory, up and allocated to the selected nodes. mixed configurations that, in the future, may storage, and networking in a rack are also include even faster programmable LSIs designed for flexible allocation and [Test 5] Efficiency of cabling using optical (FPGAs) and coprocessors (such as the Intel® pooled via a fabric switch. Rack draw- cable connection from networking Xeon Phi™ product family). ers can hold multiple boards on which integrated into drawer The current testing only went as far as the processors, memory, and networking In a typical Intel® Rack Scale Architecture manual allocation of resources. In the future, are mounted. Processors are con- Platform, the compute nodes within a drawer the intention is to test a fully automatic nected to the switch in the drawer, are connected to a disaggregated switch which mechanism for maintaining capacity without while the top-of-rack (ToR) switch (so then connects to a TOR. This allows fewer any human intervention whatsoever. The called because it is located at the top cables and prevents the tangle of cables that software should identify system overload- of the rack) connects to these drawer might otherwise be required. (Note: Based on ing automatically and respond by adding just switches allowing fewer cables. disaggregated switch cost and performance the right amount of additional resources, and requirements, some systems may decide to likewise, conserve resources when they are connect from compute node to TOR directly.) no longer needed. This should lead to higher Testing confirmed that when a server was resource utilization and further optimization of 3. Review of Testing setup for testing, a network connection and IP IT infrastructure. address were automatically provisioned. Testing of Intel® Rack Scale Architecture was 2. Improved operational efficiency a series of new discoveries. Looking back at [Test 6] Run and test actual workloads Testing demonstrated that Intel® Rack Scale the work, Mr. Toshiyuki Sugiyama of System The same OS and middleware as used on the Architecture significantly reduced administra- Infrastructure Development Section, System actual IT platforms were installed and applica- tion workloads by pooling resources in a rack Infrastructure Division, Information Systems tions run to conduct workload testing to verify and managing them centrally using Pod Man- Division commented that, “This was the first that Intel® Rack Scale Architecture could be used ager. Whereas adding processors or memory time I had ever been involved in building IT in SoftBank’s commercial systems. The installed under previous administrative environments, infrastructure from scratch, and while it was OS was an Ubuntu* Linux* distribution and the for example, required servers to be purchased a big challenge to test actual operation in middleware was a scalable key-value store (KVS) and added one at a time, this can be accom- parallel with getting the prototype up and for column-based data storage. Testing con- plished with Intel® Rack Scale Architecture running, it left me with a sense of achieve- firmed the ability to run the KVS on Ubuntu* and by adding the required processor modules, ment.” For SoftBank, this work on a new ar- that the workload could be executed on the KVS. memory, or storage to a computing resources chitecture also represented an opportunity drawer in the rack. This was found to be a to gain a major advantage. much more efficient means of operation, and [Test 7] Intel® Rack Scale Architecture Mr. Sekiya commented that, “The arrival of also eliminated the need to physically recon- API testing a new technology constitutes a change in nect network cables. This involved evaluating the standard API the world, and success or failure is deter- provided with Intel® Rack Scale Architecture. The Routine resource monitoring has previously in- mined by how quickly you can gain experi- testing provided feedback on APIs that SoftBank volved individual monitoring of physical server ence from it. We were able to advance our would like added for use in commercial systems. resources, but resource optimization is impos- IT strategy by being the first to work with a sible when physical servers are independent technology that has the potential to become Confirmation of Basic Operation in of each other, and can be influenced by a standard some years in the future, and to Preparation for Fully Automated workloads. Changing to the rack-based Intel® find out what it is capable of. This testing Operation in Future Rack Scale Architecture makes it possible to should represent a major turning point for centrally manage the physical environment. SoftBank. It seems significant to me that this • Testing results It was also found that maintenance could be new technology was offered not by a manu- 1. Testing of basic functions for software- streamlined by conducting it at the compo- facturer or system integrator but by Intel, a defined infrastructure (SDI) nent rather than the server level. device vendor.” SoftBank, a Company that Seeks to Bring about an Information Revolution, Tests an Next-Generation IT Platform Based on Intel® Rack Scale Architecture

Confirmation of Cost and Workload driving force for accelerating SoftBank “For SoftBank, Intel® Rack Scale Reductions Group’s strategy of halving non-personnel costs and achieving a two-fold increase in Architecture is a potential oppor- • Remarks productivity. tunity for changing the world.” The test results demonstrated benefits in terms of both cost and administration. Testing of Expansion Functions and – Mr. Masahiro Sekiya Results indicated a potential reduction of Functional Improvements in Preparation Senior Director, servers compared to past architectures. for Commercial Deployment of Intel® System Infrastructure Division Along with this, the company believes it will Rack Scale Architecture Systems Information Systems Division be possible to expect a significant conserva- tion of electric power and great reduction • Future Activities in operational staff workload. In terms of As the testing was conducted using a administration, tests indicated that required pre-release prototype, it did not extend to made the point that, “A world in which ma- resources could be provided more quickly, considerations of actual operation. However, chine learning, image recognition, and voice with greatly shortened delivery times of based on the test results, the intention is to recognition are performed in real time will servers and networking, while dealing with use a commercial Intel® Rack Scale Archi- need processors, I/O, and other technology faults promptly. tecture system, once available, to conduct with higher performance. Accordingly, we Mr. Kito commented on these results by further testing on expansion functions for anticipate the integration of Intel® Rack Scale saying, “As the scale of SoftBank’s business use in commercial operation, integration Architecture with new types of processors, is expanding to encompass the IoT, AI, and with OpenStack, and testing of functional such as FPGAs and co-processors, and also smart robots, the ability of Intel® Rack Scale improvements. The aim of testing on a advanced memory and storage technolo- Architecture to implement automation with commercial Intel® Rack Scale Architecture gies that use next-generation non-volatile high performance and achieve a major re- system will be to execute the applications memory. We also recognize its potential duction in IT costs not only facilitates faster used by the business, not just the work- as a next-generation IT platform that will decision making, it also has the potential to load execution testing conducted this time. support the global strategy of the SoftBank become an important platform for growth.” Furthermore, with a view to utilizing future Group, including in operational terms and Furthermore, IT is seen as providing the new technologies released by Intel, Mr. Kito through the testing of the latest technology.”

Expectation for Intel® Rack Scale Architecture Business Scale IT Cost

Growing businesses Using Existing Architecture +IoT +AI Cost +Robot Reduction +Contents

Using Intel® Rack Scale Architecture - High Performance Mr. Amane Kito Mr. Masahiro Sekiya Mr. Toshiyuki Sugiyama - Automation Senior Vice President, CISO Senior Director, System Infrastructure Devel- - Optimization and System Infrastructure opment Section Head of Information Systems Division System Infrastructure Fig. 2 (Anticipated) Benefits of Adoption Unit Information Systems Division Division Source: SoftBank Group Corp. Information Systems Division

Please note that the attribution and position are confirmed at the time of customer interview.

Intel does not control or audit the design or implementation of third party benchmarks or Web sites referenced in this document. Intel encourages all of its customers to visit the referenced Web sites or others where similar performance benchmarks or data are reported and confirm whether the referenced benchmarks or data are accurate and reflect performance of systems available for purchase. All dates and products specified are for planning purposes only and are subject to change without notice. Information in this document is provided in connection with Intel® products. No license, express or implied, by estoppel or otherwise, to any intellectual property rights is granted by this document. Except as provided in Intel’s terms and conditions of sale for such products, Intel assumes no liability whatsoever, and Intel disclaims any express or implied warranty, relating to sale and/or use of Intel products including liability or warranties relating to fitness for a particular purpose, merchantability, or infringement of any patent, copyright or other intellectual property right. Intel products are not intended for use in medical, life-saving, or life-sustaining applications. Intel may make changes to specifications and product descriptions at any time, without notice. Intel, the Intel logo, Intel Atom, Intel Atom Inside, Xeon, Xeon Phi, and Xeon Inside are trademarks of Intel Corporation in the U.S. and/or other countries. SoftBank and the SoftBank name and logo are registered trademarks or trademarks of SoftBank Group Corporation in Japan and other countries. Copyright © 2015 Intel Corporation. All rights reserved. * Other company or product names are names, trademarks, or registered trademarks of their respective companies. JPN/1605/PDF/SE/BDG/YS 334307-001US