MASARYK UNIVERSITY FACULTY}w¡¢£¤¥¦§¨ OF I !"#$%&'()+,-./012345<yA|NFORMATICS Virtualization and repository management in academic environment DIPLOMA THESIS Bc. Jakub Hadvig Brno, Spring 2013 Declaration Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Advisor: Mgr. Marek Graz´ iii Acknowledgement I would like to thank Mgr. Marek Grac´ for supervising this master thesis. Mainly I want to thank my colleague Ivan Necasˇ for all his advice and feedback during the applica- tion development. I would also like to thank the whole Katello and Foreman develop- ment team and community for their effort and work. Furthermore, I would like to thank Marek Mahut, Jozef Zigmund,ˇ Luka´sˇ Votypka´ and my whole family, for their help and support during the writing of this thesis. Finally I would like to thank Red Hat company for the internship, during which I was able to write this thesis. v Abstract This thesis deals with usage of virtualization and repository management in academic environment. Its goal is to make application which will provide virtual machines and their management for students and teachers in their class courses. The systems virtu- alization and their management will be handled by open source projects Katello and Foreman dedicated for system provisioning and management. Their REST API will be used at the back-end of the application to provide all the necessary services. vii Keywords Rest API, virtualization, libvirt, provisioning, smart proxy, puppet, ruby, ruby on rails, foreman, katello, open-source ix Contents 1 Introduction .......................................1 1.1 Goals ........................................1 1.2 Why virtualization in laboratory IT courses ..................2 1.3 Structure of work .................................3 2 Virtualization ......................................5 2.1 Types of virtualization ..............................5 2.2 Virtualization APIs ................................7 2.2.1 Libvirt . .8 3 Puppet .......................................... 11 3.1 Agent-Master model ............................... 11 3.2 How Puppet works ................................ 12 3.3 Puppet architecture ................................ 14 3.3.1 Puppet modules . 14 3.3.2 Module structure . 16 3.3.3 Puppet environments . 16 4 Project Foreman ..................................... 19 4.1 Foreman architecture ............................... 20 4.1.1 Puppet . 20 4.1.2 Smart proxy . 21 4.1.3 Foreman Rest API . 23 4.2 Provisioning .................................... 26 5 Project Katello ...................................... 29 5.1 Katello components ................................ 29 5.1.1 Pulp . 30 5.1.2 Candlepin . 31 5.2 System registration and basic managemet ................... 33 6 Katello Scholar ..................................... 35 6.1 Project Specification ................................ 35 6.1.1 Used Technologies . 35 xi 6.1.2 Project configuration . 36 6.2 Structure of project ................................ 40 6.2.1 Project workflow . 41 6.2.1.1 Preparation of application environment .......... 41 6.2.1.2 Application usage ....................... 42 6.2.2 Model Mapping . 44 6.3 Present application limitations ......................... 45 7 Conclusion ........................................ 47 7.1 The theoretical part ................................ 47 7.2 The practical part ................................. 47 7.3 Personal experiences ............................... 48 7.4 Future development plans ............................ 48 8 Terminology ....................................... 49 A Application source code ................................ 51 xii List of Figures 2.1 Implementing an application without and with libvirt[10] 9 3.1 Difference between system configuration without(left) and with(right) puppet master. 12 3.2 How Puppet works.[5] 13 4.1 Foreman architecture.[15] 20 4.2 Smart Variable example 21 4.3 Smart proxy distributed services scenario 22 4.4 Smart proxy all in one scenario 23 4.5 Foreman REST API example of a JSON input, for creating a Hostgroup object 24 4.6 Available REST API calls for Foreman Hostgroup object 24 4.7 Foreman REST API call example together with returned values 25 5.1 Project Katello features and dependencies.[7] 30 5.2 Pulp diagram.[3] 31 5.3 Candlepin diagram. 32 6.1 Katello Scholar entity-relationship model 40 xiii Chapter 1 Introduction In today’s education environment there is a lack of resources that could provide stu- dents with an opportunity to experiment with operating systems, their tools, so they can study without worrying of breaking some part of the system, the advantages and disadvantages of the system, to better understand it thoroughly and master its admin- istration. Also an desirable idea is that students will come to their lab lessons, and all the needed lesson resources and tools will be pre-installed and up and running, so they don’t have to worry about any missing dependencies associated with the lessons cur- riculum, and in case something goes wrong everything can be restored to initial state in a few minutes. They just log into the system and practice the lesson curriculum, even if that means learning by trial and error. This chapter is meant to provide a basic overview of what this thesis is about, spec- ify its goals and expected outcomes. The text part of this thesis will give overview of individual parts which the practical part consists of. The practical output of this thesis will be an application used in academic environment for virtual machine provisioning and management. 1.1 Goals The main goal is to create an open source web-based application capable of providing fully configured virtual machines and their management for lectors and their students for their laboratory IT courses. The application in its basic form will be able to provide operating system from Red Hat operating system family (Red Hat Enterprise Linux, Fe- dora, CentOS, Scientific Linux) and settings specified by the lector and software pack- ages, that will be pre-installed. In addition the application will be able to assign needed repositories to the virtual machines of the desired lesson. The application will be built upon two open-source projects, whose RESTful API will 1 1. INTRODUCTION be used at the back-end of the application to provide all the necessary services. These two projects are The Foreman project which will be responsible for the virtual machine provisioning, configuration and management, and project Katello, which is a system life cycle management tool developed by Red Hat company and will be responsible for the repository management. The text part of this thesis will describe virtualization and introduce these two projects, their functionality and usage. Also tools, which these projects are using, will be pre- sented and described because of their important role. The text part will therefore consist of four main fields that will be looked into: • Virtualization • Puppet • Project Foreman • Project Katello 1.2 Why virtualization in laboratory IT courses Virtualization in laboratory IT courses has a lot of advantages compared to classic ap- proach, where each class has predefined operating system or collection of operating systems, on all the class computers and students log in with their credentials. In this case students have to prepare their environment at the beginning of each lesson in case they need extra packages as lesson requirements. They also have to be careful not to disrupt their environment. On the other hand, virtualization and virtual machine pre- configuration in these courses come with number of benefits: • All students in the class have identical, easily pre-set environments when using virtual machines. • All students will have a clean virtual machine for each lesson without any old dependencies that could cause problems (This kind of approach is also used in software development environment). • All students can be given an administrator access to the virtual machines, where they won’t affect the physical host machine. 2 1. INTRODUCTION • In case something goes wrong, due to the students’ administrator rights or other circumstances, the virtual machine can be easily restored (re-installed) to the initial state. • Every student can be given more virtual machines to work with. • Lector can prepare environments with necessary operating system and software packages for each course lesson, so no further package installation and configu- ration will be needed during the lesson. • Virtual machines will install themselves automatically before the lesson begins so when the lesson starts everything is prepared and set up for the lesson cur- riculum. 1.3 Structure of work The second chapter will summarize basic types and aspects of virtualization. Third chapter will describe the Puppet, tool designed to manage system configuration, which is utilized in both projects. Fourth and fifth chapters describe the two used projects and their usage. Chapter six discusses the practical part of this thesis and describes the design and architecture of the created web-based application. 3 Chapter 2 Virtualization Virtualization has its roots in early 1960s, when IBM corporation came up with and time-sharing solution so their users and customers can more efficiently use their expen- sive computer resources. Virtualization in this days is a common