vCPU Sizing Considerations Accurately Determine how many vCPUs to Allocate to Virtual Machines

Written by David Davis, Evangelist & Alex Rosemblat, Product Marketing Manager

Abstract The challenge: maximizing server hardware ROI while The number of vCPUs is one of the most important considerations ensuring high application performance when sizing virtual machines. But getting the right balance— Virtualization administrators should avoid over-allocating neither over-allocating nor under-allocating—is a challenge. This vCPUs because doing so wastes expensive server resources white paper offers the guidance you need to get vCPU sizing right. and will minimize ROI on that infrastructure. In fact, over- allocation of vCPUs in some VMs will actually cause Introduction performance problems for that VM and other VMs. On the vCPU sizing is one of the most important considerations when other hand, the business critical applications running in virtual sizing virtual machines machines also need to maintain high performance and they When sizing virtual machines, a virtualization administrator need processing power to do so. The last thing a virtualization must select the number of vCPUs, the size of the virtual disk, administrator wants is to have performance complaints from number of vNICs and the amount of memory. Out of all those end users. VM administrators face a difficultly in balancing potential variables, the two most difficult to determine are the need to maximize ROI of the server hardware and the always the number of vCPUs and the amount of memory to requirement for applications to perform optimally. allocate. This is because CPU and memory are the most finite resources that a server has, and these resources are also the Fortunately, with the right tools in place and the guidance from most dynamically demanded resources by the guest OS in each this whitepaper, virtualization administrators will be able to VM. For a discussion of memory sizing, see “VM easily make the right decisions about vCPU sizing. Memory (vRAM) Sizing Considerations”; here, we’ll cover proper vCPU sizing in your virtual machines. Understanding CPU usage in To summarize, the difference in CPU virtual servers usage with physical and virtual servers Physical and virtual servers are different is this: when it comes to CPU usage. • In the physical world, applications are With the traditional physical server or scheduled by the OS onto the physical desktop PC, you have an entire CPU or CPUs. multiple CPUs (each with multiple cores) • In the virtual world, applications running dedicated to the OS and applications on each OS, in each VM, make requests of running on it. The virtualization virtual CPUs that are then scheduled by the adds an additional layer hypervisor. between OS and the physical CPU, allowing multiple virtual machines to Adding CPUs benefits multi-threaded share the hardware. Instead of the CPU applications only. requests from applications going to the OS and then the OS them Not only do today’s servers have on the physical CPU, the OS in the VMs multiple CPU cores in each CPU, but talks to virtual CPUs (which it thinks are they also use multi-threading (known real physical CPUs). Requests from the as “hyper-threading” if you are using an Only multi-threaded multiple virtual CPUs are scheduled, Intel CPU). This means that each CPU by the hypervisor, across the multiple core can execute “threads” in parallel. applications will physical CPU cores. Just like with You can think of a as a , benefit from having memory sharing in virtualization, with so envision a CPU core being able to run CPU sharing in virtualization, there is the multiple processes all at the same time more than one CPU. traditional OS CPU scheduler and the (in parallel). However, having more than hypervisor CPU scheduler. one CPU will benefit only applications that are multi-threaded. In other words, All of this enables greater utilization dedicated task-based servers (such as and massive sharing of the server’s database servers and web servers) that physical CPUs. run a single-threaded application will see no performance benefit by adding more CPUs.

Determining whether an application is single-threaded or multi-threaded To know if an application is single- threaded or multi-threaded, you have two options: • Ask the software manufacturer if the application is multi-threaded and supports SMP (symmetric multi-processing). • When you have more than one processor or core on a server, see whether the program’s multiple processes are using

Figure 1. The virtualization stack

2 more than one processor or core at a time. application but that app isn’t multi- For example, if you have a quad-core CPU, threaded, you are wasting the cost watch the application as the CPU demands of those CPUs. However, in a virtual increase. If the application can use only infrastructure, if you over-allocate 25 percent of total CPU capacity (one of multiple vCPUs to a VM and the primary the four cores), then it is a single-threaded app on that VM isn’t multi-threaded, you application that can’t use more than one could actually cause performance issues core. On the other hand, if the application for that VM and others. This is because, is using 50, 75 or 100 percent of the total with multiple vCPUs, the hypervisor’s CPU capacity (four of the four cores), then CPU scheduler must wait for multiple it is multi-threaded. physical CPU time slots to become available before it can process requests Whether you are running an application from the multi-vCPU VM. on a physical server or a virtual machine, you want to have multiple CPUs In other words: available only if the application running • With one vCPU, CPU requests are quickly on that host can take advantage of those processed (or they are waiting on pCPU if no CPUs with multiple threads. pCPU is available), as illustrated in Figure 2 • With multiple vCPUs, the hypervisor CPU If your virtual hosts Having multiple vCPUs when they are scheduler must wait for multiple pCPUs not needed will slow down VMs. to be available, as shown in Figure 3. don’t have the On a physical server, if you have multiple Therefore, having multiple vCPUs when CPU capacity that CPUs available for the server’s primary they are not needed will slow down VMs. today’s applications demand, you should solve that problem before analyzing vCPU allocations.

Figure 2. Application command processing with one vCPU

3 100 percent CPU usage for short periods isn’t cause for changing CPU configuration. Instead, look at Figure 3. Application command processing with multiple vCPUs CPU statistics

over a slightly Things to know before you begin the CPU was utilized, over the time range longer sample analyzing vCPU allocations selected. Note that the total CPU includes Before you begin your analysis, resolve all vCPUs on a VM or all pCPUs on a host. time to ensure that any virtual host and VM performance If you have greater than 80 percent CPU problems. utilization on a VM or host over a one- you are seeing hour time period, you should find ways to trends instead of Deciding the proper number of vCPUs solve that problem. Those solutions could for a VM should be a long-term include reducing the number of vCPUs on instantaneous performance planning exercise that you your VMs, migrating active VMs to another perform periodically. Before starting that host, adding more pCPU to a host, or usage. vCPU planning exercise, you should first dealing with misbehaving applications. determine if there is already CPU over- utilization in your virtual machines or on If your virtual hosts don’t have the your virtual host. Typically, this is done CPU capacity that today’s applications by checking the following: demand, you should solve that problem • CPU ready (cpu.ready.summation) in first. It should be noted that one of the milliseconds (ms)—This statistic is available potential solutions for solving high per virtual machine or per vCPU. It shows virtual host CPU utilization is to reduce the number of milliseconds that the VM the number of vCPUs allocated to VMs if was ready to execute requests on the virtual they were dramatically over-allocated. host’s CPU but there was no pCPU available to do so. An increasing CPU Ready value for VMware performance metrics to use a VM indicates that there is an ongoing lack for virtual host pCPU and VM vCPU of CPU cores on the virtual host. usage analysis • CPU usage (cpu.usage.average) in Once you’ve ensured that your physical percent—This CPU usage statistic is server isn’t over-utilized, it’s time to available per host, per VM or per resource move on to a structured analysis of pool. It shows the percentage of time that vCPU allocation on one or across

4 all virtual machines. We’ll provide a How to view and modify vCPUs for VMs detailed process below but first, let’s Viewing and modifying vCPU cover two statistics you need to know configurations on your VMs isn’t before you start that process: something VMware admins need to do • CPU usage maximum (cpu.usage. every day; however, you do need to maximum) in percent—This shows the know where to do it and when you can maximum amount of CPU that was used make changes: at any one time, as a percentage. Keep • To view VM vCPU configuration—Click on in mind that if you have two vCPUs, a 50 a VM and looking at the Summary tab. percent utilization value is 100 percent of • To edit VM vCPUs—Go to a VM’s properties one vCPU and 0 percent of another, or 25 and then click on the CPU in the hardware percent of one and 25 percent of another. tab. This value will be used to determine if we • To view vCPU configurations across all hit the maximum amount of CPU already VMs—Go to a higher level in the virtual You can configure allocated to that VM during the time range. infrastructure (from a VM, go up to the a Mhz limit on a • CPU usage average (cpu.usage.average) ESXi server or up to the resource pool or in percent—This value shows the average cluster level) and then click on the Virtual virtual machine’s CPU utilization over the time range. Note Machines tab. that this is based on the total number of vCPU, but it rarely vCPUs; for example, if you have two vCPUs, Additional factors to keep in mind makes sense. a 100 percent utilization over the time when allocating VM CPUs range is 100 percent of both vCPUs. Limits Instead, set a CPU By configuring a limit on a virtual Understanding time ranges and machine, you are placing an artificial Mhz limit across all sample periods cap on the maximum amount of CPU VMs in a particular To be able to accurately understand that the VM can use. Without a limit, the statistics that you are viewing, you maximum CPU that can be used is the resource pool. must fully understand that these stats Mhz of the pCPUs on the server, per are measured over the default sample vCPU allocated to a VM. Keep in mind periods or whatever sample period you that while a VM has full access to the specify. For example, if you measure total Mhz of a pCPU (per vCPU allocated), real-time or one-minute CPU usage, it those pCPUs are still shared between is very likely that you will see periodic that VM and all others on that virtual CPU utilization peaks of 100 percent. host. Note that limits configured in the Short periodic CPU usage at 100 percent hypervisor aren’t visible to the guest OS, isn’t cause for alarm or changes to CPU which can further cause unexpected configuration. Instead, I recommend application performance issues. that you look at CPU statistics over a slightly longer sample time to ensure While functionality exists to configure a that you are seeing trends instead of Mhz limit on a virtual machine’s vCPU, instantaneous usage. it rarely makes sense. Instead, it makes more sense to set a CPU Mhz limit across Some number of the statistical samples all VMs in a particular resource pool. are pulled out and stored in a historical database for use when you run a Many times, VM CPU limits are put in performance report over a specific place by an administrator who did not time period. Understanding both the fully understand the poor performance sample time and the time range are and wasted resources that limits can important when it comes to interpreting create. When looking at CPU utilization, performance graphs. CPU limits skew performance metrics and can cause confusion while

5 troubleshooting. In other cases, VM CPU Special considerations when changing limits have also been put in place by the vCPU configurations VMware admin to limit physical resource When you determine that the number consumption by applications (often of vCPUs that a VM has configured unbeknown to the application owner). needs to be changed, there are a few considerations: Reservations • When going from one vCPU to many, or CPU reservations artificially set the from many to one, vCPU kernel changes minimum amount of CPU that a virtual are required in the VM guest operating machine (or resource pool) has access system. For example, with Windows Server to. Even though those CPU resources 2003, you need to make a change to the may not be needed by the VM now, a HAL (see VMware KB article 1003978 for reservation pulls those CPU resources more information). However, with Windows Take the time away from other virtual machines that Server 2008, you can switch between to understand may need it. Like limits, reservations are single and multiple CPUs without making better set on resource pools instead any HAL changes. how and when of individual virtual machines since • Virtual machines need to be shut down they can hurt the performance of other to remove vCPU, and most VMs need to applications are virtual machines and skew CPU metrics be shut down to change vCPU unless used. Those due to the artificial requirements being the vSphere “hot plug CPU” feature was put in place. enabled before boot AND the VM meets insights will help the requirements to use The application’s needs that feature. you properly size Taking the time to look inside the virtual vCPU for the VM, machine and analyze the CPU resources The VM CPU sizing process that an application uses can yield a great Overview configure resource deal of information about that VM’s CPU Not only is a virtual environment dynamic, needs. When evaluating a Windows but the usage of the applications in the controls (if needed), operating system, you can run Resource VMs will be in constant flux. As a VMware and understand Monitor and Performance Monitor to admin working on a critical production expose which processes use the most virtual infrastructure with hundreds or the priority of each CPU and how it varies. thousands of constantly changing virtual machines, you need a formal process application. Also, speaking to the business owner for for proper vCPU sizing, not just a “rule an application helps determine when of thumb.” that application is used, who uses it, what would happen if it were unavailable, This process, ideally, involves the and how the use of the application is application admins and will have to be growing (or shrinking). undertaken both when a new VM is being created for an application and on By taking time to understand these a periodic basis. factors, VM administrators can draw further insights into properly sizing The workflow diagram on the next page vCPU for the VM, configuring resource introduces a best practice for how to controls (if needed), and understanding execute this process. While this process the priority of the application as may need slight modification for certain compared to other apps. companies, it will work as is for most VMware administrators.

6 Make sure you identify the true peak by investigating

Figure 4. The VM CPU sizing workflow occurrences of 100 percent CPU to see The vCPU sizing process for an have a common application that you existing VM know is relatively predictable, you can if they are one-time Let’s look at the steps in this vCPU sizing safely look at time frames between one process if we are not dealing with a week and one month. anomalies. newly deployed VM (the branch to the right in the figure). Step 2: Find the peak (maximum) value. So what peak value do you choose? Step 1: Determine the time frame for Peak values are interesting because they analysis. show whether the vCPUs were ever 100 As I mentioned earlier in this guide, percent utilized during your time period. when performing your analysis, you However, you need to investigate exactly need to take the perspective of the when the 100 utilization occurred time frame into consideration. It’s easy (for example, was it during backups?) to interpret statistics incorrectly and and weigh your findings with the make improper vCPU configurations by average utilization. For example, an looking at a poorly selected time range. instantaneous value of 100 percent CPU utilization for a single vCPU VM over a While it is difficult to pick one time frame week means nothing if it happened only that is perfect for all situations, I generally once during the backup window and the recommend a time frame that is no less average utilization is just 25 percent. than one week. However, in some cases, that time frame may be as much as a year In other words, don’t just look at the (such as a retail company that has high current value and use that. Use the tips utilization around the holiday season or a above in determining a time frame to university that has high utilization around make sure that you identify the true peak. registration). On the other hand, if you

7 Step 3: Compare the peak usage value for both current and expected future to allocated CPU. usage, or a VM may face vCPU-related Once you find the true average and peak performance issues. (maximum) percent of vCPU utilized during your sample, compare that to Once you establish the expected growth the number of vCPUs configured for rate, add a buffer to ensure that the the virtual machine. Then use these vCPU allocation is accurate. While guidelines: historical peaks have likely been taken • If the average is less than 38 percent and into account, it is still good to factor the peak is less than 45 percent, consider in a buffer to ensure that the virtual downsizing vCPUs. machine’s vCPU has headroom to grow • If the average is greater than 75 percent into and does not max out. For less Be sure to provision and the peak is above 90 percent, consider critical applications, a buffer may not adding vCPUs. be necessary, and can be avoided to sufficient vCPU for retain more CPU capacity for other VMs. For example, if the VM is configured with However, for multi-threaded critical both current and two vCPUs and the maximum utilization applications whose CPU utilization is 100 percent, you are maxing out both fluctuates, adding an additional vCPU is expected future vCPUs at some point. If the average recommended. This way, in the event usage; otherwise, utilization is consistently greater than 80 that an unexpected, business-critical percent, then consider adding vCPUs. demand on CPU happens, end-user the VM may face application performance won’t suffer. On the other hand, if you find that that vCPU-related vCPUs never hit their maximum, average Step 5: Make kernel-level changes if performance issues utilization is low, and you have more needed, change the vCPU allocation, than one vCPU, then consider migrating and document all your changes. later on. the VM down to one vCPU (or reducing Once you’ve completed the first four the VM’s vCPUs by one). steps, it is time to change the virtual machine’s number of vCPUs. Step 4: Forecast the expected growth rate and factor in a buffer to determine First, shut down the guest OS. Unless the the right allocation. correct guest OS is installed and CPU hot Before you decide to downsize a VM’s plug is enabled (assuming you need to vCPU configuration, be aware that the add additional vCPU), the guest OS must vCPU demands of the VM (and, more be shut down to add or remove vCPU specifically, its applications) could from a virtual machine. increase by the time that you repeat the vCPU sizing process. Ask yourself Next, resize the virtual machine’s vCPU. the following questions: Has the Right-click on the VM (in the hosts application owner been contacted? Was and clusters inventory) and then click an increasing trend in vCPU demand Properties. At that point, you’ll see a noted based on historical analysis? It is window (see Figure 5) where you can important to provision sufficient vCPU resize the VM’s CPUs on the Hardware tab.

8 Figure 5. Configuring vCPU on a vSphere VM Once you have

Once the VM’s vCPU has been resized, On the other hand, if a new VM is being deployed a new you need to take guest OS kernel and created for a new application or use VM, monitor CPU HAL changes into account. Typically, case, and this is no indication how many the servers that need HAL or kernel vCPUs should be allocated for that app usage closely to see changes when going from one CPU to and its users, you could just go with the several, or from several to one, are the recommendations as if the app were if more vCPUs are older Windows Server operating systems being installed on a physical server. Or, needed. like Windows Server 2003. Without the you could start with just one vCPU and proper changes, Windows 2003 Server see how it goes from there. systems will perform slowly or not at all. Windows Server 2008 and -based It may be tempting to go with a gut systems don’t require any HAL or kernel feeling (such as the SharePoint admin’s changes when their number of virtual preference that his VM to have six vCPUs) CPUs is changed. or to use the same number of vCPUs that a physical server is configured Finally, you can document the change to with. But with those methods, it’s likely the VM’s vCPU and power back on the VM. that the resulting allocation will either grossly overshoot the number of vCPUs The vCPU sizing process for a new VM needed or, even worse, underestimate Now, let’s look at the vCPU sizing the number of vCPUs required by the workflow in Figure 4 again but this time, application and its users. let’s say that a new virtual machine is being provisioned, so we’ll follow the path Step 2: Monitor CPU metrics intensely. to the left. There won’t be any historical Once a new VM has been deployed with performance data for this new virtual a given number of vCPUs, CPU usage machine. Because of this, the vCPU should be monitored intensely. Just how configuration made for this VM could be frequent and in-depth this monitoring much less accurate and the VM will need needs to be depends on how critical the to be monitored more closely. VM is. Administrators may also be able to rely on vSphere alarms to alert them if the Step 1: Initially, allocate the minimum new VM hits high CPU utilization. vCPUs possible. When monitoring, look at the vSphere In many cases, there will be some idea VM CPU usage level and also look inside of how many vCPUs a new VM should the guest OS to see how much CPU is in have based on the same application (or a use and by what application. Is the VM similar app) running on another VM. But maxing out the base quantity of vCPUs even if this information is available, the you configured? If so, then more vCPUs process of monitoring and reviewing are likely needed. described below should still be followed.

9 Step 3: Add more vCPUs one at a time. new VM has an appropriate number of If the new VM is running low on CPU vCPUs, be sure to include the new VM in capacity, add more vCPUs—one at a the periodic vCPU review process. time. Remember, you want to find the right number of vCPUs for the VM, not Keep SLA requirements in mind. just throw a bunch of resources at it and As administrators, we strive to give potentially cause performance problems applications what they need for peak later. When you add an additional vCPU, performance. However, in the real world, be sure to monitor very carefully. due to service level agreements or cost models, this isn’t always possible. In Set up a regular CPU sizing review these cases, we have to intentionally give based on an appropriate time frame. fewer vCPUs than would be ideal or use Because application usage changes, vCPU resource controls like limits in order to Because application must be continuously evaluated to ensure give a VM (and its underlying CPU-hungry usage changes, you that performance remains high through application) only what should be allocated any additional changes in the dynamic for business reasons. should regularly virtualized infrastructure. Thus, keep service level agreements in review vCPU Accordingly, you should set up a regular mind when analyzing vCPU allocations usage to ensure vCPU review process at appropriate and realize that there are cases where we intervals. My recommendation is to hold have to allocate a vCPU value other than that performance reviews once per month. However, some what is optimal. organizations that are growing quickly remains high. may want to perform reviews more Auto-size vCPUs in your frequently. Once a VM sizing process environment has been conducted a few times on the You can automatically calculate virtual machines in an environment, the vCPU allocation using the techniques steps will become more familiar and described in this white paper with a administrators will be able to build on their 30-day free trial of Dell vOPS Server experience to streamline the process. Standard™. The free trial of vOPS Server Whenever you add a new VM, once the Standard™ installs as a virtual appliance vCPU usage has been stabilized and the in 20 minutes.

Figure 5. Automatically calculate vCPU with vOPS Server Standard™.

10 Conclusion In additional, regular monitoring of Besides memory, CPU is the most CPU usage will help VM administrators finite computing resource that a virtual proactively spot problem areas in VMs infrastructure has. By implementing that are under-provisioned, so they can the workflow detailed in this paper, prevent performance issues. Ultimately, virtualization administrators can more accurately sizing vCPU in virtual machines efficiently use their computing resources. benefits everyone: the organization can Data centers will be able to reclaim realize a better return on investment CPU resources for other VMs, enabling for a virtualization initiative, and both them to increase VM density and defer application owners and end users see hardware purchases. better application performance.

11 For More Information © 2013 Dell, Inc. ALL RIGHTS RESERVED. This document DELL ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS contains proprietary information protected by copyright. No ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING part of this document may be reproduced or transmitted in TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE any form or by any means, electronic or mechanical, including IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR photocopying and recording for any purpose without the A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO written permission of Dell, Inc. (“Dell”). EVENT SHALL DELL BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL Dell, Dell Software, the Dell Software logo and products—as DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES identified in this document—are registered trademarks of Dell, FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS Inc. in the U.S.A. and/or other countries. All other trademarks OF INFORMATION) ARISING OUT OF THE USE OR INABILITY and registered trademarks are property of their respective TO USE THIS DOCUMENT, EVEN IF DELL HAS BEEN ADVISED owners. OF THE POSSIBILITY OF SUCH DAMAGES. Dell makes no representations or warranties with respect to the accuracy or The information in this document is provided in connection completeness of the contents of this document and reserves with Dell products. No license, express or implied, by estoppel the right to make changes to specifications and product or otherwise, to any intellectual property right is granted by descriptions at any time without notice. Dell does not make this document or in connection with the sale of Dell products. any commitment to update the information contained in this EXCEPT AS SET FORTH IN DELL’S TERMS AND CONDITIONS AS document. SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT,

About Dell Dell Inc. (NASDAQ: DELL) listens to customers and delivers worldwide innovative technology, business solutions and services they trust and value. For more information, visit www.dell.com.

If you have any questions regarding your potential use of this material, contact:

Dell Software 5 Polaris Way Aliso Viejo, CA 92656 www.dell.com Refer to our Web site for regional and international office information.

Whitepaper-vCPUsizingConsideration-US-SW-02-27-2013