Networking Multicore Processing: Virtualization and Data Center Syed Shah, Nikolay Guenov Virtualization and Its Impact Virtualization helps to: Figure 1: Virtualization is a combination of • Run multiple operating systems on a Benefits of VirtualizationBenefits of Virtualization software and hardware features that single computer, including Windows®, creates virtual CPUs (vCPU) or virtual Linux® and more In-Service Software Upgrades systems-on-chip (vSoC). These vCPUs • Increase energy efficiency, reduce Dynamic Resource Allocation or vSoCs are generally referred to as hardware requirements and thereby virtual machines (VM). Each VM is reduce overall capital expenditure an abstraction of the physical SoC, Cost Flexibility • Determine highest availability OEM and Functional ConsolidationReduction Value Scalability complete with its view of the interfaces, and performance for enterprise Board-Level Consolidation memory and other resources within the applications Reliability physical SoC. Virtualization provides and • Use computing resources efficiently Protection the required level of isolation and Sandboxing Applications Multiple OSs in a Single SoC partitioning of resources to enable Virtualization in each VM. Each VM is protected from Embedded Space Virtualization and interference from another VM. The virtualization layer is generally called the Although they are not mainstream, these Multicore Processing virtual machine monitor (VMM). IT industry trends are trickling down into With multicore SoCs, given enough the embedded space as well. Observed processing capacity and virtualization, Why Virtualization? trends for virtualization in the embedded control plane applications and data space include: plane applications can be run without Virtualization has already impacted the one affecting the other. Data plane server and IT industries in a significant • The concept of having a sea of and control plane applications, in most way. IT organizations are using it to processors, and the associated cases, will be mapped to different reduce power consumption and building processing capacity sliced and diced cores in the multicore SoC as shown in space, providing high availability for among applications and processes, is Figure 2. critical applications and streamlining no longer science fiction. application deployment and migration. • The challenges of extracting higher Control and data plane applications The trends to adopt virtualization in the utilization from the processors, and are not the only application- server space also are being driven by consolidation triggered by cost level consolidation that will occur. the desire to support multiple operating reduction, are driving the adoption of Virtualization and partitioning will allow systems and consolidation of services virtualization in embedded systems. OEMs to enable their customers to on a single server by defining multiple • With virtualization, one can merge the customize service offerings by adding VMs. Each VM operates as a stand- control and data plane processing their own applications and operating alone device. Because multiple VMs onto the same SoC. Previous systems to the base system on the can run on a single server (provided the approaches used separate discrete same SoC, rather than using another server has enough processing capacity), devices for these functions. discrete processor to handle it. Data IT gains the advantage of reduced or control traffic that is relevant to the Virtualization offers three major benefits server inventory and better server customized application and operating to the embedded industry: utilization. system (OS) can be directed to the 1. Cost reduction via consolidation appropriate virtualized core without 2. Flexibility and scalability impacting or compromising the rest of 3. Reliability and protection the system. 24 Beyond Bits Power Architecture Edition Figure 2: Control and Data Plane Application 1) OS-hosted and 2) bare-metal hypervisor. Each approach has its pros ControlConsolidation and Data Plane inApplication a Virtualized Multicore SoC and cons and the choice would depend Data Plane Processing Control Plane Processing SoC SoC on the applications and the market µp µp µp µp segments. OS-Hosted Hypervisor Data Plane Traffic Switch Control Plane Traffic The hypervisor schedules the guest operating systems based on the Mixed Control and Data Plane Traffic Application scheduling policies in the hypervisor. Consolidation Scheduling of the VMM and guest Data Plane Control Plane operating systems is dependent on µp µp µp µp Single Multicore SoC Partitioned and Virtualized the scheduling policies of the host OS, Mixed Control and because they run on top of the host OS. Data Plane Traffic Switch Mixed Control and Figure 3: Data Plane Traffic OS-hostedOS-Hosted Hypervisor Hypervisor Another example of consolidation of Addressing Challenges with Virtual Virtual functions is board-level consolidation. Virtualization Machine 1 Machine 2 Functions that were previously Addressing Partitioning Challenge implemented on different boards now Partitioning can be defined as Hypervisor can be consolidated onto a single card subdividing resources of a SoC in a and a single multicore SoC. Virtualization Device Host OS manner that allows the partitioned Drivers can present different virtual SoCs to the resources to operate independently of applications. With increasing SoC and one another. Partitioned resources can Hardware application complexity, the probability be mapped either explicitly to the actual of failures due to software bugs and hardware or to the virtualized hardware. SoC mis-configuration are greater than Note that the system can be partitioned purely hardware-based failures. In Bare-Metal Hypervisor without being virtualized. For example, such a paradigm, it may make sense The bare-metal hypervisor approach in a SoC that allows partitioning but not to consolidate application-level fault does not depend on the host OS and virtualization, each Ethernet interface can tolerance onto a single multicore SoC, runs directly on the physical hardware be assigned to a partition but a single where a fraction of the cores are set (bare metal). The hypervisor fully Ethernet interface cannot be assigned aside in hot standby mode. While such controls the SoC, enabling it to provide to two different partitions at same time. a scheme will save the cost of having quality of service guarantees to the However, if the SoC also provides to develop a standby board or at the guest operating systems. virtualization capabilities, then a single very least another SoC, it would require Ethernet interface can be virtualized and the SoC to be able to virtualize not only Figure 4: each virtual Ethernet interface can be the core complex but also the inputs/ Bare-Metal Hypervisor presented to a different partition. Bare-Metal Hypervisor outputs (I/Os). Although virtualization has its Addressing Fair Sharing and Virtual Virtual advantages, it comes with new Protection Challenge Machine 1 Machine 2 challenges and considerations, including A hypervisor is a software program partitioning, fair sharing and protection that works in concert with hardware Guest OS 1 Guest OS 2 of resources between multiple/ virtualization features to present the VM Device Hypervisor competing applications and operating to the guest OS. It is the hypervisor that Drivers systems. The following sections will creates the virtualization layer. There discuss how virtualization technology are two broad architectural approaches Hardware addresses these challenges. when it comes to virtualizing the system: freescale.com 25 Networking Different I/O handling approaches can Advantages of Virtualization Based on the system and the be used by the hypervisor, including fully application, memory associated with On the processor, a new hypervisor virtualized I/O, dedicated I/O and para- a VM may need to be protected and state introduced by Freescale and some virtualized I/O. In the fully virtualized partitioned from the other VMs. For traditional compute-centric companies approach, the hypervisor virtualizes example, in a mixed control and data automatically traps privileged and the I/O by emulating the devices in plane system where control and data sensitive calls by the guest OS to the software. The software overhead plane applications run on the same hypervisor, removing the need for binary thus created reduces the efficiency SoC, it is imperative that memory code rewriting or para-virtualization. This of the system. In the para-virtualized be protected and partitioned so that reduces the complexity and overhead approach, the I/O interfaces are not fully these applications run completely introduced by the hypervisor and also virtualized. independently and the state of one improves overall performance of the partition cannot erroneously or The key difference between full I/O system. maliciously corrupt the state of the other virtualization and para-virtualization is The second area where hardware partition. that not all functions are emulated in support improves hypervisor para-virtualization; hence, this approach Finally, the last major area where performance is memory management. reduces software overhead at the cost hardware support for virtualization will In an un-virtualized system, a virtual of OS portability. In the dedicated greatly reduce hypervisor overhead memory address is formed by using the approach, each VM is assigned a is the I/O. With increasing desire to effective address. The virtual address