MASARYK UNIVERSITY FACULTY}w¡¢£¤¥¦§¨ OF I !"#$%&'()+,-./012345<yA|NFORMATICS Foreman plugin for Jenkins CI MASTER’S THESIS OndˇrejPraz´akˇ Brno, 2015 Declaration Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Advisor: Mgr. Marek Grac,´ Ph.D. ii Acknowledgement I would like to give thanks to Mgr. Marek Grac,´ Ph.D. for supervising my thesis and provided counsel. I would like to thank Petr Chalupa and Ivan Necasˇ for their help, advices and suggestions during my work on the practical part of this thesis. My thanks also belongs to my parents who supported me in my studies. iii Abstract The main goal of this thesis is to analyze and design a plugin for Foreman that would allow cooperation with Jenkins CI. The textual part describes the motivation, current state of the art. The practical part deals with my proposed solution of the outlined task. iv Keywords The Foreman, Jenkins, CI, automation, configuration management, DevOps v Contents 1 Introduction .......................................3 2 Automation in Software Development .......................5 2.1 Build Automation .................................5 2.2 Continuous Integration ..............................7 2.3 Continuous Deployment and Continuous Delivery .............9 2.4 Configuration Management ........................... 10 3 State of the Art ..................................... 13 3.1 The Foreman .................................... 13 3.2 Katello ....................................... 15 3.3 Puppet ....................................... 17 3.4 Pulp ......................................... 20 3.5 Candlepin ..................................... 22 3.6 Bastion ....................................... 23 3.7 Elastic Search ................................... 24 3.8 Jenkins ....................................... 25 3.9 Jenkins API Client ................................. 27 3.10 Dynflow ...................................... 27 3.11 Foreman Tasks ................................... 28 4 Project Design & Analysis .............................. 29 4.1 Project Structure .................................. 29 4.2 Models ....................................... 29 4.2.1 ForemanPipeline::Job . 30 4.2.2 ForemanPipeline::JenkinsInstance . 32 4.2.3 ForemanPipeline::JenkinsProject . 32 4.2.4 ForemanPipeline::JenkinsProjectParam . 33 4.2.5 ForemanPipeline::JenkinsUser . 33 4.3 Views ........................................ 33 4.4 Controllers ..................................... 33 4.5 Dynflow Actions ................................. 34 1 4.5.1 General Workflow . 35 4.6 Challenges and Obstacles ............................ 37 Conclusion .......................................... 39 Bibliography ........................................ 40 Index ............................................. 46 A User Guide ........................................ 47 A.1 Jobs ......................................... 47 A.1.1 Create a Job . 47 A.1.2 Configure a Job . 47 A.1.3 Run a Job . 52 A.1.4 Promote a Content View . 52 A.2 Jenkins Projects .................................. 52 A.2.1 Create Jenkins Project . 52 A.2.2 Configure Project Parameters . 53 A.3 Jenkins Instance .................................. 55 A.3.1 Create Jenkins Instance . 55 A.3.2 Configure Jenkins Instance . 56 A.4 Jenkins Users ................................... 56 A.4.1 Create Jenkins User . 57 A.4.2 Configure Jenkins User . 58 A.4.3 Change user’s details . 58 2 Chapter 1 Introduction When a certain level of technological progress is reached, automation is an idea that offers itself naturally thanks to its apparent advantages. Speed, cost effectiveness and decreased error rate are the most obvious ones. Automation is hardly a new idea since we have proofs of its practical application as early as 1930’s [31]. But even after 80 years, we still concern ourselves with it because it is an ongoing process rather than a problem with one solution that can be reapplied each time when circumstances require it. Furthermore, automation succeeds in finding new fields of application for itself thus creating new challenges and invades even our homes [10]. That is possibly a reason why there are no signs of the ultimate solution on the horizon. Even this thesis concerns itself with automation, but is limited to the software deve- lopment. There is a plethora of aids that try to automate various aspects of interaction with software systems and running automated tests during development is only a tip of the proverbial iceberg. There are already tools that can provision large number of ma- chines, regardless whether virtual or bare metal, with a few mouse clicks. Also content and configuration management has made a great leap forward during the last decade. There are also high-powered build engines that can be used for continuous integration and subsequent deployments to various environments. But there are so far no promi- nent attempts to combine these powerful standalone tools and make them collaborate closely. Content of my thesis humbly tries to exploit this niche. It aims to bring The Foreman with its plugin Katello and Jenkins together and make them cooperate in a useful way. Foreman is Red Hat’s solution to host provisioning and configuration, one of Katello’s main features is content management of machines provisioned by Foreman. Jenkins is one of the leading open-source continuous integration servers. General idea is to make Foreman provision a new system based on user’s predefined configuration, sup- ply Jenkins CI server with content from Katello and enough information about newly 3 1. INTRODUCTION provisioned machine to allow an optional deployment of builds. The whole process should run without supervision and be triggered by a certain type of events. In chapter 2, I try to elucidate the tools and principles that have a practical impact in the areas mentioned in the previous paragraphs. Chapter 3 describes various projects that are either reification of the topics discussed in chapter 2 or they are in some way important for the practical part of this thesis. They are subsequently brought together in chapter 4, which presents my solution of the outlined task. 4 Chapter 2 Automation in Software Development 2.1 Build Automation When it comes to interaction with a software, human user is almost exclusively a liabi- lity. He is much slower and more prone to make mistakes in comparison to a running program. Sometimes problems may arise solely from the fact that different people do the same thing differently [13]. Growing complexity of the project increases the count of manual steps that are ne- cessary to include all the components required for successful compilation when change is made. To keep track of all dependencies may be more than challenging, if not impos- sible, and the more manual steps are required the higher is a chance of a mistake. This was recognized long time ago and UNIX world has had Make tool for several decades now [17]. To use Make, a user needs to write a Makefile listing the rules. Each rule con- sists of target, target’s prerequisites and optionally a set of commands. When invoking Make with a target, it tries to resolve its dependencies and produce desired artifact. GNU Automake takes things even further with its automatic creation of portable Makefile. It depends on GNU Autoconf to supply shell script that adapts the pack- age to the user’s system. Autoconf is essentially a bundle of M4 macros that create the configure script which in turn runs series of tests to determine the target system’s con- figuration. Because different systems handle shared libraries differently and C compiler versions may vary, GNU Libtool was created to hide the library complexities behind portable interface [45]. Libtool, Autoconf and Automake are often used together and referred to as GNU build system, although Libtool may be used independently [23]. Make’s popularity was an inspiration for derived tools in other programming lan- guages. One of the most prominent examples is Rake (Ruby Make), which Ruby world relies on to do its automated tasks. Unsurprisingly, Rake tasks are defined in a Rakefile and they use pure Ruby syntax as a bonus [59]. 5 2. AUTOMATION IN SOFTWARE DEVELOPMENT Ruby developers rely on two additional tools in their daily tasks. To keep track of dependencies, Bundler is the weapon of choice. Project dependencies, which are com- monly called gems and are simultaneously units of code distribution in Ruby, are de- fined in Gemfile in project’s root folder. Bundler takes care of tracking, resolving and installing needed gem versions with simple bundle install command [32]. RVM (Ruby Version Manager) manages whole Ruby development environment and can be used to set identical self-contained environments in Development, Testing and Produc- tion environments. It has a flexible gem management system known as Named Gem Sets that prevent duplicate gem versions to be installed in the system and confines all gems exclusively into user space thus providing higher level of system security. RVM also allows having various Ruby versions with their own Gem Sets and provides com- fortable way of switching between them [47]. Although Make is widely used, it also has a wide array of undesirable attributes. Difficulty to maintain readability for large projects, not raising errors for undeclared variables and obligatory tabs at the beginning