The Eucalyptus Open-source Cloud-computing System Daniel Nurmi, Rich Wolski, Chris Grzegorczyk Graziano Obertelli, Sunil Soman, Lamia Youseff, Dmitrii Zagorodnov Computer Science Department University of California, Santa Barbara Santa Barbara, California 93106 Abstract ability, application performance profiling, software ser- Cloud computing systems fundamentally provide ac- vice requirements, and administrative connections. While cess to large pools of data and computational resources great strides have been made in the HPC and Grid Com- through a variety of interfaces similar in spirit to exist- puting communities [15, 7] toward the creation of resource ing grid and HPC resource management and program- provisioning standards [14, 18, 33, 38], this process re- ming systems. These types of systems offer a new pro- mains somewhat cumbersome for a user with complex re- gramming target for scalable application developers and source requirements. have gained popularity over the past few years. However, For example, a user that requires a large number of most cloud computing systems in operation today are pro- computational resources might have to contact several prietary, rely upon infrastructure that is invisible to the different resource providers in order to satisfy her re- research community, or are not explicitly designed to be quirements. When the pool of resources is finally deliv- instrumented and modified by systems researchers. ered, it is often heterogeneous, making the task of per- In this work, we present EUCALYPTUS – an open- formance profiling and efficient use of the resources diffi- source software framework for cloud computing that im- cult. While some users have the expertise required to ex- plements what is commonly referred to as Infrastructure ploit resource heterogeneity, many prefer an environment as a Service (IaaS); systems that give users the ability to where resource hardware, software stacks, and program- run and control entire virtual machine instances deployed ming environments are uniform. Such uniformity makes across a variety physical resources. We outline the ba- the task of large-scale application development and de- sic principles of the EUCALYPTUS design, detail impor- ployment more accessible. tant operational aspects of the system, and discuss archi- Recently, a number of systems have arisen that at- tectural trade-offs that we have made in order to allow tempt to convert what is essentially a manual large-scale Eucalyptus to be portable, modular and simple to use on resource provisioning and programming problem into a infrastructure commonly found within academic settings. more abstract notion commonly referred to as elastic, util- Finally, we provide evidence that EUCALYPTUS enables ity, or cloud computing (we use the term “cloud com- users familiar with existing Grid and HPC systems to ex- puting” to refer to these systems in the remainder of this plore new cloud computing functionality while maintain- work). As the number and scale of cloud-computing sys- ing access to existing, familiar application development tems continues to grow, significant study is required to de- software and Grid middle-ware. termine directions we can pursue toward the goal of mak- ing future cloud computing platforms successful. Cur- rently, most existing cloud-computing offerings are either 1 Introduction proprietary or depend on software that is not amenable There are many ways in which computational power to experimentation or instrumentation. Researchers in- and data storage facilities are provided to users, rang- terested in pursuing cloud-computing infrastructure ques- ing from a user accessing a single laptop to the alloca- tions have few tools with which to work. tion of thousands of compute nodes distributed around the The lack of research tools is unfortunate given that world. Users generally locate resources based on a va- even the most fundamental questions are still unanswered: riety of characteristics, including the hardware architec- what is the right distributed architecture for a cloud- ture, memory and storage capacity, network connectivity computing system? What resource characteristics must and, occasionally, geographic location. Usually this re- VM instance schedulers consider to make most efficient source location process involves a mix of resource avail- use of the resources? How do we construct VM instance 1 networks that are flexible, well-performing, and secure? system” that is, by design, hypervisor agnostic. However, In addition, questions regarding the benefits of cloud com- the current implementation of the system uses Xen-based puting remain difficult to address. Which application do- virtualization as its initial target hypervisor. mains can benefit most from cloud computing systems and Grid computing must be acknowledged as an intellec- what interfaces are appropriate? What types of service tual sibling of, if not ancestor to, cloud computing [7, 15, level agreements should cloud computing provide? How 33, 38]. The original metaphor for a computational utility, can cloud-computing systems be merged with more com- in fact, gives grid computing its name. While grid com- mon resource provisioning systems already deployed? puting and cloud computing share a services oriented ap- Cloud computing systems provide a wide variety proach [16, 17] and may appeal to some of the same users of interfaces and abstractions ranging from the ability (e.g., researchers and analysts performing loosely-coupled to dynamically provision entire virtual machines (i.e., parallel computations), they differ in two key ways. First, Infrastructure-as-a-Service systems such as Amazon EC2 grid systems are architected so that individual user re- and others [4, 12, 27, 9, 30]) to flexible access to hosted quests can (and should) consume large fractions of the software services (i.e. Software-as-a-Service systems total resource pool [34]. Cloud systems often limit the such as salesforce.com and others [37, 20, 21, 29]). All, size of an individual request to be tiny fraction of the to- however, share the notion that delivered resources should tal available capacity [4] and, instead, focus on scaling to be well defined, provide reasonably deterministic perfor- support large numbers of users. mance, and can be allocated and de-allocated on demand. A second key difference concerns federation. From its We have focused our efforts on the “lowest” layer of cloud inception, grid computing took a middleware-based ap- computing systems (IaaS) because here we can provide proach as a way of promoting resource federation among a solid foundation on top of which language-, service-, cooperating, but separate, administrative domains [14]. and application-level cloud-computing systems can be ex- Cloud service venues, to date, are unfederated. That is, a plored and developed. cloud system is typically operated by a single (potentially In this work, we present EUCALYPTUS: an open-source large) entity with the administrative authority to mandate cloud-computing framework that uses computational and uniform configuration, scheduling policies, etc. While storage infrastructure commonly available to academic re- EUCALYPTUS is designed to manage and control large col- search groups to provide a platform that is modular and lections of distributed resources, it conforms to the design open to experimental instrumentation and study. With EU- constraints governing cloud systems with regards to fed- CALYPTUS, we intend to address open questions in cloud eration of administrative authority and resource allocation computing while providing a common open-source frame- policies. work around which we hope a development community will arise. EUCALYPTUS is composed of several compo- Thanks in part to the new facilities provided by virtu- nents that interact with one another through well-defined alization platforms, a large number of systems have been interfaces, inviting researchers to replace our implementa- built using these technologies for providing scalable In- tions with their own or to modify existing modules. Here, ternet services [4, 1, 8, 10, 11, 19, 37], that share in com- we address several crucial cloud computing questions, in- mon many system characteristics: they must be able to cluding VM instance scheduling, VM and user data stor- rapidly scale up and down as workload fluctuates, support age, cloud computing administrative interfaces, construc- a large number of users requiring resources “on-demand”, tion of virtual networks, definition and execution of ser- and provide stable access to provided resources over the vice level agreements (cloud/user and cloud/cloud), and public Internet. While the details of the underlying re- cloud computing user interfaces. In this work, we will source architectures on which these systems operate are discuss each of these topics in more detail and provide a not commonly published, EUCALYPTUS is almost cer- full description of our own initial implementations of so- tainly shares some architectural features with these sys- lutions within the EUCALYPTUS software framework. tems due to shared objectives and design goals. In addition to the commercial cloud computing of- 2 Related Work ferings mentioned above (Amazon EC2/S3, Google Ap- Machine virtualization projects producing Virtual Ma- pEngine, Salesforce.com, etc.), which maintain a propri- chine (VM) hypervisor software [5, 6, 25, 40] have en- etary infrastructure with open interfaces, there are open- abled new mechanisms for providing resources to users.