Vmware Technical Journal
Total Page:16
File Type:pdf, Size:1020Kb
VOL. 1, NO. 1–APRIL 2012 VMWARE TECHNICAL JOURNAL Editors: Steve Muir, Rita Tavilla and Ben Verghese VMWARE TECHNICAL JOURNAL TECHNICAL VMWARE SPECIAL THANKS TABLE OF CONTENTS Stephen Alan Herrod, Ph.D. Chief Technology Officer and Senior Vice President of Research & Development 1 Introduction Steve Herrod, CTO Julia Austin Vice President Innovation Programs 2 VisorFS: A Special-purpose File System for Efficient Handling of System Images Olivier Cremel PROGRAM COMMITTEE Banit Agrawal 3 A Software-based Approach to Testing VMware® vSphere® VMkernel Public APIs Jim Chow Lan Xue, Sreevathsa Sathyanarayana, Thorbjoern Donbaek, Ramesh Pallapotu, James Truong, Keith Farkas Sriram Sankaran, Eric Lorimer Steve Muir Javier Soltero 4 Providing Efficient and Seamless Desktop Services in Ubiquitous Computing Environments Rita Tavilla Lizhu Zhang, Wenlong Shao, Jim Grandy Ben Verghese 5 Comprehensive User Experience Monitoring Lawrence Spracklen, Banit Agrawal, Rishi Bidarkar, Hari Sivaraman Questions and Comments can be sent to [email protected] 6 StatsFeeder: An Extensible Statistics Collection Framework for Virtualized Environments Vijayaraghavan Soundararajan, Balaji Parimi, Jon Cook 7 VMware Distributed Resource Management: Design, Implementation, and Lessons Learned Ajay Gulati, Anne Holler, Minwen Ji, Ganesha Shanmuganathan, Carl Waldspurger, Xiaoyun Zhu 8 Identity, Access Control, and VMware Horizon Will Pugh, Kyle Austin 9 VMworld 2011 Hands-On Labs: Implementation and Workflow Adam Zimman, Clair Roberts, Mornay Van Der Walt VOL. 1, NO. 1 2012 1, NO. VOL. VMware, Inc. 3401 Hillview Avenue Palo Alto CA 94304 USA Tel 877-486-9273 Fax 650-427-5001 www.vmware.com Copyright © 2012 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents. VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. Item No: VMW-TJ-VMWARE-TECH-JOURNAL-VOL1-NO1-USLET-106-cover VMWARE’S RESEARCH PARTNERSHIPS VMWARE ACADEMIC PROGRAMS The VMware Academic Program (VMAP) supports a number of academic research projects across VMware has created rich global education and research offerings with programs to a range of technical areas. We conduct an annual Request for Proposals (RFP), and also support allow access and support to the latest virtualization technologies for both research and a small number of additional projects that address particular areas of interest to VMware. The topic instruction purposes: for our Spring 2012 Request for Proposals was Security for Virtualized and Cloud Platforms, VMware Academic Programs with funding awards to be announced in May 2012. Our previous RFPs addressed Performance If you represent an academic institution, the VMware’s Academic Programs group can provide Management Challenges in Virtualized Environments (Spring 2011) and Cloud Computing (Spring access to cutting-edge virtualization and cloud technology, along with access to experts for 2010). A selection of our past and present research partnerships include: research and teaching purposes. • Data-Mining Virtual Machines for Resource Optimization • Research Funding Una-May O’Reilly, Saman Amarasinghe, Skye Wanderman-Milne, MIT, Computer Science and • Conference Sponsorships Artificial Intelligence Laboratory (CSAIL) • Research Collaboration • Managing Datacenter Assets under Capacity and Power Constraints • Guest Lectures Karsten Schwan, Ada Gavrilovska, Georgia Institute of Technology, Center for Experimental • Scholar in Residence Program Research in Computer Systems (CERCS) • VMware Graduate Fellowship • A Robust Framework of Conversational Assurance for Distributed Systems based on Multiparty Session Types Academic Affiliate Partners Nobuko Yoshida, Rumyana Neykova, Imperial College, London; Kohei Honda, • Algorithms, Machines, People Laboratory (AMP) Laboratory at the University of Queen Mary, University of London California, Berkeley • Parallel Data Laboratory (PDL) at Carnegie Mellon University Performance Management Challenges in Virtualized Environments (Spring 2011 RFP) • Center for Experimental Research in Computer Systems (CERCS) at Georgia Institute • Energy Efficient and Reliable Server Consolidation for the Cloud of Technology Ayse Coskun, Boston University • Computer Science and Artificial Intelligence Laboratory (CSAIL) at Massachusetts Institute of Technology • Hardware Performance Monitoring of Scientific Applications in VMware Environments Shirley Moore, Dan Terpstra, University of Tennessee Visit labs.vmware.com to learn more about VMware’s education and research opportunities. • Flexible Computing with VM Fork Eyal de Lara, University of Toronto Cloud Computing (Spring 2010 RFP) • A vCloud at Carnegie Mellon: Establishment, Automation, Measurement, and Evolution Greg Ganger, Carnegie Mellon University Hello and welcome to the inaugural issue of the VMware Technical Journal! This journal is completely driven by VMware engineers who enjoy telling the details of interesting projects and products, sharing their passion for virtualization and cloud computing, and celebrating the innovation occurring all around our industry. It’s definitely an exciting time to be at VMware. We’re at a point in time when nearly 60% of all server applications in the world are running virtualized, the large majority of those on VMware’s virtualization platform. We’re also becoming very popular in the “desktop computing” space as enterprises rapidly move into the Post-PC era characterized by mobile devices, SaaS applications, and anywhere/anytime computing. This popularity means that many great partners want to integrate with our products, that our innovations can have a dramatic impact, and that we have an opportunity to extend our products into many adjacent technology areas. As a result, our engineering team ends up working in almost every area of system software. And the VMware Technical Journal appropriately reflects this breadth of focus. This inaugural issue includes articles on our core vSphere technologies as well as on our newer SaaS-related management tools. We’re also providing a look behind the product creation process with articles that cover the testing of vSphere quality, measuring performance, and building great user-experiences. We also close the journal with a great look at using all of these products in a very high-stress, high-demand environment… the VMworld Labs! We sincerely hope you enjoy this first edition. We welcome your comments and ideas for future articles. Until next time, happy computing! Steve Herrod CTO, VMware 1 VisorFS: A Special-purpose File System for Efficient Handling of System Images Olivier Cremel VMware [email protected] Abstract In reaction to the complexity of classic VMware® ESX®, simplicity Two kernels vying for the same hardware, and needing to be was an overarching design driver for VMware® ESXi™. A very simple synchronized through upgrade cycles, led to inherent complexity runtime file system with specific features was needed to allow the in maintaining the code base and managing the ESX system. To use case—stateless and embedded—that was envisioned for ESXi facilitate the deployment and management of thousands of ESX at the time. Those features turned out to be quite useful, as ESXi systems at one extreme, and the availability of very small turnkey evolved beyond its original intent. This paper reveals what is special configurations at the other, simplifying the ESX architecture about visorfs and how it enables the efficient use of memory, clean became paramount. handling of system images, and proper definition of system state. The obvious angle of attack was to see how the Service Console could be excised and have all of its roles handled by the VMkernel. 1. Introduction It became clear this required a radical rethinking of how ESX worked. The ESX architecture relies on a Service Console to handle three ESX needed to change from being seen as just another server to a main tasks: clean platform that delivers virtualization simply and easily. In practice, that meant changes, such as the absence of installation (system • Booting. The Service Console is booted in total control of the images baked offline) and the push for statelessness (external host, and is responsible for loading the ESX kernel (VMkernel) authority instead of local configuration) that drove the design and handing control to it. of what became ESXi. • Handling some devices, such as USB or BMC. Once the VMkernel 1.1 Taking Over the Service Console Roles is in charge of the machine, the Service Console is lifted up as a Booting was handled by adding a thin layer to the VMkernel to make pseudo virtual machine that has direct access to the host. it multi-boot compliant. In addition, an existing bootloader was used • Providing a user-level environment. The VMkernel offers limited that understood the multi-boot standard, while handling additional user-level support for the exclusive benefit of the virtual machine devices required writing the appropriate drivers for the VMkernel. executable (VMX), the user-level side of the virtual machine monitor. All other user-level needs are handled by the Service Console, ESX already provided user-level support for the benefit of VMX such as agents and shells for management. with a combination of kernel code and the GNU