Intel® Cloud Builders Guide Intel® Xeon® Processor‐based Servers StackIQ Rocks+ Intel® Cloud Builders Guide to Cloud Design and Deployment on Intel® Platforms Creating Private Clouds from Bare Metal using Rocks+ Management Software Audience and Purpose This paper will discuss Rocks+, a deployment and management solution from StackIQ that makes building highly scalable clouds simple and efficient. We will walk users through the end‐to‐end cloud building process, starting from bare metal physical machines, and ending with a complete cloud environment well purposed for enterprise Intel® Xeon® Processor 5500 Series IT environments and service providers. Intel® Xeon® Processor 5600 Series StackIQ Rocks+ We’ll frame the discussion by using BigCorp as an example customer, which has multiple departments with varying data center needs. First, we’ll set up a group of LAMP nodes for BigCorp’s Web Services Department. Second, we’ll set up a Hadoop cloud for BigCorp’s Analytics Department. In this process, we’ll show how these Virtual Private Clouds can remain completely isolated, while retaining a central repository of software by leveraging the Rocks Cloud Development Kit. Using Rocks+ and the contents of this paper, which includes detailed commands and screen shots, should significantly reduce the learning curve for building and operating your first cloud computing infrastructure. Intel® Cloud Builders Guide: Creating Private Clouds using Rocks+ Management Software Table of Contents Executive Summary ............................................................................................................................................................... 3 Product Overview .................................................................................................................................................................. 3 Test Bed Blueprint ................................................................................................................................................................. 4 Technical Review and Use Cases ............................................................................................................................................ 5 Use Case 1: Install and Configure Your Physical Infrastructure as a Cloud ........................................................................ 5 Use Case 2: Create an Airboss for Added Security and Control ........................................................................................ 13 Use Case 3: Setup Your First Virtual Private Cloud Inside Your Physical Cloud running BitNami LAMP Stacks ................. 14 Use Case 4: Scale Physical Cloud Up ............................................................................................................................... 17 Use Case 5: Setup Your Second Virtual Private Cloud in the same Physical Cloud running Hadoop MapReduce .............. 18 Things to Consider ................................................................................................................................................................. 20 Conclusion ............................................................................................................................................................................. 20 About StackIQ ....................................................................................................................................................................... 20 Glossary and Additional Information ..................................................................................................................................... 20 2 Intel® Cloud Builders Guide: Creating Private Clouds using Rocks+ Management Software The Xen Roll installs and configures inside of Amazon EC2. We’ll briefly Executive Summary virtual machines (VMs) on Rocks+ discuss how the same Rolls used in this Rocks+ is a deployment and management clouds. A physical frontend can paper can we leveraged in Amazon solution that makes building highly scalable configure VMs on client nodes (VM EC2. clouds simple and efficient by encapsulating container appliances). A VM container and delivering the complete cloud stack in a is a physical machine that houses and Product Overview single Linux distribution. By using Rocks+, you runs VMs. The Xen Roll also supports Rocks+Cloud sets up a cluster of physical can quickly deploy a physical cloud with a building virtual clusters. The frontend machines as a private cloud by first central point of management and any number can be installed as a VM server provisioning a “frontend” appliance and then of virtual container nodes. The physical cloud appliance and the client nodes can be provisioning the remaining servers as “vm‐ can then be used to host any number of installed as VM containers. Then a container” appliances. Rocks+Cloud Virtual Private Clouds (VPCs), which gives virtual frontend can be installed on the subsequently provisions, manages, and users and departments security and control of VM server while virtual nodes can be controls VMs and virtual clusters (VCs). their own infrastructure. The Rocks+ Cloud installed on the VM containers. All network traffic is encapsulated within a Development Kit enables modular Scalable Provisioning components at each layer in the stack and unique VLAN, that is, each virtual allows users to define custom appliance cluster has its own VLAN. Rocks+ provides a parallel, automated types. The Hadoop Roll is a simple to use, software installation process to accelerate your time to solution. You can network‐boot The Software: Rocks+ highly scalable solution for deploying Hadoop clusters of all shapes and sizes. (PXE‐boot) physical nodes or launch virtual Rocks+ dramatically reduces the time and Typically, the installation and nodes in parallel from the command‐line. All cost of setting up and managing clusters of management of a Hadoop cluster nodes are provisioned from the ground up cloud‐enabled servers through intelligent requires a long, hands‐on process in and are ready to run applications on first boot. multi‐server software automation (both which the end‐user or a deployment physical and virtual machines). The entire Rocks+ includes the Avalanche Installer to team has to install and configure each cloud stack is packaged and deployed as a address the bottleneck of delivering packages component of their data center by monolithic yet modular Linux distribution from a central distribution server to multiple hand. The setup time for these systems (based on Red Hat Enterprise Linux or simultaneously‐installing cloud nodes. When and continued management can be CentOS). The system provides for a single an installing node downloads a package, the burdensome, which has a direct affect step install, and is capable of dynamically node stores the package locally then sends a on reliability and security. As more provisioning heterogeneous appliance types message to an Avalanche tracker. The enterprises look to utilize Hadoop, the across massive scale data centers (in parallel, package is now in a peer‐to‐peer package need for a supported and robust leveraging BitTorrent‐style package sharing). cache. When a subsequent installing node platform that is easy to deploy, A key differentiator that separates Rocks+ wants to download the same package, the manage, and operate has become from other "complete stack" paradigms, is a node sends a message to the Avalanche essential. Rocks+ completely modular framework called Rolls, which are tracker and the tracker responds with a list of automates this process. optional, automatically configured, cloud‐ peers. The installing node will download the aware software systems. The BitNami Roll enables users to package from a peer, not the central select pre‐packaged binaries from distribution server. This dramatically reduces Rocks+ Rolls BitNami.org and deploy them across the network pressure on the central Rolls allow customers to take advantage of an Rocks+ clouds. The BitNami project distribution server and enables large‐scale end‐to‐end packaged cloud stack and still has support for a number of commonly simultaneous node installations. choose which software components are used machine types including a LAMP included to meet site‐specific requirements. stack, which we’ll demonstrate in this Here are a few of the Rolls that will be paper. demonstrated in this paper: The AWS Roll enables Rocks+ users to use StackIQ’s management solution 3 Intel® Cloud Builders Guide: Creating Private Clouds using Rocks+ Management Software Management and Control Machine Architecture The Airboss Rocks+ makes ongoing management easy. The following illustration is a guide to help In Rocks+, there is a service known as the Cluster and cloud administrators manage the keep track of the names of the physical "Airboss" that resides on the physical entire system from the fronted. If a physical machines and the virtual machines that will frontend (in Dom0) and it allows non‐root or virtual node needs an update, it will be be built during the technical review. It’s too users to control their VMs. The motivation for completely re‐provisioned by the frontend to small to read here, but will be explained in this service is that libvirt (a virtualization API ensure it boots into a known‐good state. New detail throughout the paper. included in Red Hat Enterprise Linux that can nodes are also configured automatically— control several different virtualization with a single command—without the need for implementations)