A CLOUD ARCHITECTURE FOR REDUCING COSTS IN LOCAL PARALLEL AND DISTRIBUTED VIRTUALIZED CLOUD ENVIRONMENTS by JEFFREY MICHAEL GALLOWAY Dr. SUSAN VRBSKY, COMMITTEE CHAIR Dr. XIAOYAN HONG Dr. JOHN LUSTH Dr. RANDY SMITH Dr. FEI HU A DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Computer Science in the Graduate School of The University of Alabama TUSCALOOSA, ALABAMA 2013 Copyright Jeffrey Michael Galloway 2013 ALL RIGHTS RESERVED ABSTRACT Deploying local cloud architectures can be beneficial to organizations that wish to maxi- mize their available computational and storage resources. Many users are reluctant to move their computational and storage needs to a public cloud vendor. While designing scalable local cloud architectures, power requirements should be given adamant attention. This dissertation focuses on several challenging concerns relating to cloud computing architectures, specifically lowering the power requirements of Infrastructure-as-a-Service (IaaS) local cloud architectures. These chal- lenges include power efficient computational resource load consolidating, power efficient persis- tent cloud storage consolidating, and deploying a local IaaS cloud architecture with limited net- working resources. The design of a load consolidation approach to Infrastructure-as-a-Service cloud archi- tectures that is power efficient is presented in this dissertation. A proposed Power Aware Load Consolidation algorithm, PALC, maintains the state of all compute nodes, and based on utiliza- tion percentages, decides the number of compute nodes that should be operating. Results show that PALC provides adequate availability to compute node resources while decreasing the overall power consumed by the local cloud architecture. Persistent storage is a necessity in cloud computing architectures. Since the goal of this local cloud architecture design is to deploy resources using minimum power consumption, a power aware persistent storage consolidation algorithm is presented in this dissertation. The Power Aware Storage Consolidation algorithm, PASC, dynamically determines the number or active persistent ii storage nodes based on the number of active users. This algorithm, combined with the PALC algorithm will significantly decrease the power consumed by the local cloud architecture. Realizing the implications of deploying a local cloud system in an environment with limited networking resources (IP addresses), a solution is needed to allow users to connect with only one public IP address. Users will be able to access cloud resources through a simple web interface and maintenance of the cloud will be contained with private networking resources. Also introduced is the ability to scale to have multiple geographically distributed clusters in the local cloud using only one IP address per cluster. This dissertation provides a comprehensive solution for deploying a local cloud architecture that is cost efficient to maintain. iii LIST OF ABBREVIATIONS 3DES Triple Data Encryption Standard 3G 3rd Generation 4G 4th Generation AES Advanced Encryption Standard AJAX Asynchronous JavaScript and XML AMD-V AMD Vitualization Extensions APAC Asia Pacific API Application Programming Interface ARPANET Advanced Research Projects Agency Network ATX Advanced Technology Extended AWS Amazon Web Services CC Cluster Controller CDN Content Delivery Network CD-ROM Compact Disc Read Only Memory CGI Common Gateway Interface CIFS Common Internet File System CLC Cloud Controller CPU Central Processing Unit CPUutil CPU Utilization (%) iv CRUD Create, Read, Update, Repeat CSS Cascading Style Sheet DES Data Encryption Standard DHCP Dynamic Host Control Protocol DOM Document Object Model DPr Dirty Page Rate DRY Don’t Repeat Yourself DSL Digital Subscriber Line Ew Power Consumped During VM Migration EBS Elastic Block Storage Controller EC2 Elastic Compute Cloud EioP Ethernet over IP EU European Union Eucalyptus Elastic Utility Computing Architecture Linking Your Programs to Useful Systems FAH Folding at Home FTP File Transfer Protocol GB Gigabyte Gbps Gigabits per second GFS Google File System GPS Global Positioning System GUI Graphical User Interface HaaS Hardware as a Service v HDD Hard Disk Drive HTML Hyper-text Markup Language I/O Input/Output IaaS Infrastructure as a Service ID Identity IMFS Internet Media File System IP Internet Protocol IPv4 Internet Protocol Version 4 IPv6 Internet Protocol Version 6 IT Information Technology KVM Kernel-based Virtual Machine kWh KiloWatt-hours Lab Link Bandwidth L20 20 large virtual machines L30 30 large virtual machines LAN Local Area Network LTS Long Term Support M Mean Time Between Two Migration of the Same Virtual Machines Mtms Total Time for Migration MB/s Megabytes per second Mbps Megabits per second MeMmig Total amount of Memory transmitted from source to destination vi MySQL My Structured Qurey Language Nu Network utilization NAS Network Attached Storage NAT Network Address Translation NC Node Controller NFS Network File System NIST National Institute of Standards and Technology OCCI Open Cloud Computing Interface OSI Open Systems Interconncetion Pt Increase in Process Execution Time During VM Migration Pth Decrease in Process Throughput During Vm Migration PaaS Platform as a Service PALC Power Aware Load Consolidation PASC Power Aware Storage Consolidation PC Personal Computer PDA Personal Data Assistant Pflops Peta Floating Point Operations per Second PHP-CLI PHP-Command Line Interface PHP PHP: Hypertext Preprocessor QOE Quality of Experience QOS Quality of Service R20 20 random sized virtual machines vii R30 30 random sized virtual machines RAM Random Access Memory RDP Remote Desktop Protocol REST Representational State Transfer RPM Revolutions Per Minute RRLB Round Robin Load Balancer RSA Rivest, Shamir, Adleman (encryption algorithm) S20 20 small virtual machines S3 Simple Storage Service S30 30 small virtual machines SaaS Software as a Service SATA Serial Advanced Technology Attachment SDK Software Development Kit SDN Storage Delivery Network SOAP Simple Object Access Protocol SSD Solid State Disk SSH Secure Shell SSL Secure Sockets Layer TB Terabyte UEC Ubuntu Enterprise Cloud UFW Uncomplicated Firewall URI Uniform Resource Identifier viii US United States USD United States Dollar Vims Total Inaccessable Time for VM VM Virtual Machine VMmem Virtual Machine Memory Size VPN Virtual Private Network VT Virtualization Extensions W Watt WAN Wide Area Network Wh Watt-hour WS3 Walrus Storage Controller XHTML Extensible Hyper-text Markup Language XL20 20 extra large virtual machines XL30 30 extral large virtual machines XML Extensible Markup Language XSLT Extensible Stylesheet Language Transformations ix DEDICATION To my mother, father, and sister. & To the pioneers of computer science that paved the way before me. “We can only see a short distance ahead, but we can see plenty there that needs to be done.” — Alan Turing, Father of Computer Science “If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility.” — John McCarthy, Father of Cloud Computing x ACKNOWLEDGMENTS My appreciation goes to my professor and adviser, Dr. Susan Vrbsky. I am very thankful for Dr. Vrbsky’s involvement in my academic and professional life for the past three years. During my time here Dr. Vrbsky helped me become a better computer science researcher and teacher. When I applied to the Ph.D. program, Dr. Vrbsky was the first to contact me about joining a research group that would evolve into the Cloud and Cluster Computing Group here at the University of Alabama. As I was progressing through the program, Dr. Vrbsky was able to send me to many great conferences around the United States. Even though most of our interaction involved research topics, I also learn much about the aspects of working in the academic environment. Those insights are invaluable and would be difficult to find anywhere else. As I progress through my career, I plan on remaining in contact with the person that most influenced the direction of my life. Thank you, Dr. Vrbsky, for motivating and helping me through these years of research and teaching. I would like to express my gratitude to Dr. Xiaoyan Hong. Dr. Hong served as my advisor during my time here as a master’s student. My undergraduate focus was in computer networking so it was obvious that I would take her as an adviser. During my time as a master’s student, Dr. Hong introduced me to many computer network tools and applications. As I moved to local cloud computing infrastructure research, computer networking is still a major factor to master. Dr. Hong has always been a great source of information when I faced tough networking problems. To Dr. Randy Smith, who was my first contact arriving at the University of Alabama. Thank you for teaching the Software Engineering course and Human Computer Interaction course, both xi of which helped me prevail in my studies of cloud computing. I will always remember the first remark of noticing me in Houser Hall with my orange and blue backpack. To Dr. John Lusth, my teaching advisor. I am glad to have had the opportunity to teach the Introduction to Programming class here for so many semesters. This has undoubtedly been one of the best experiences I had during my time as a Ph.D. student. I believe my approach in teaching students has become significantly improved by Dr. Lusth. Also, for some unknown reason, I now believe that every text editor and web page should respond to VIM commands. To my other committee members, I appreciate your cooperation and support during in my time here as a Ph.D. student. Dr. Jeffrey Carver, thank you for giving me the insight of applying an empirical approach to my research of cloud computing. Dr. Marcus Brown, thank you for always providing a positive aspect on any situation we discussed. Dr. Yang Xiao, thank you for your insights on wireless networks and security. To David Engle, thank you for sharing your knowledge in systems administration. This is an interesting field to me, which is shown by the fact that this dissertation is inherently an applied systems research.