Evaluation of Infrastructure As a Code for Enterprise Automation

Masaryk University Faculty of Informatics

Evaluation of Infrastructure as a Code for Enterprise Automation

Master’s Thesis

Michal Hagara

Brno, Spring 2018

Declaration

Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and liter- ature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source.

Michal Hagara

Advisor: Bruno Rossi, PhD

Acknowledgements

I would like to thank Dr. Bruno Rossi for being my adviser during the thesis, as without his advice and help I would not be able to finish the thesis. I would like to thank him also for his patience and understanding on the personal level.

iii Abstract

Expansion of cloud and virtualization is present every day, and it has to be handled accordingly, as there is an emphasis on speed, consistency and time to market. The thesis deals with this phenomenon and provides insight into terms such as automation, Infrastructure as a Code, DevOps or configuration management (CM). The thesis aims to introduce how the selection process of configuration management tool for enterprise automation should be carried out, as the integration of the right tool to the infrastructure is critical. The selection process is divided into two parts. First part evaluates mechanics and different aspect of configuration management tools from the practical point of view. In the second part of the thesis evaluation of public survey of configuration management tools is presented. The primary contribution of this thesis is aimed at providing insight on how the selection process of CM tool should be carried out, and also to provide a comprehensive review of CM tools for enterprise automation.

iv Keywords

Configuration management, automation, DevOps, Infrastructure asa Code, Ansible, Chef, Puppet, SaltStack

Contents

1 Introduction 1

2 Infrastructure as a Code 3 2.1 History and DevOps background of IaC ...... 3 2.2 Infrastructure as a Code as a part of DevOps ...... 4 2.3 Configuration management ...... 5

3 Comparison of configuration management tools 7 3.1 Ansible ...... 9 3.1.1 Communication and security ...... 9 3.1.2 Ansible specific working mode ...... 9 3.1.3 Ansibles mechanics and terminology ...... 10 3.1.4 Ansible DSL and templating engine ...... 12 3.2 Chef ...... 13 3.2.1 Communication and security ...... 13 3.2.2 Chefs specific working mode ...... 14 3.2.3 Chefs mechanics an terminology ...... 14 3.2.4 Chef DSL and templating engine ...... 16 3.3 Puppet ...... 17 3.3.1 Communication and security ...... 17 3.3.2 Puppet specific working mode ...... 18 3.3.3 Puppets mechanics an terminology ...... 18 3.3.4 Puppet DSL and templating engine ...... 20 3.4 SaltStack ...... 21 3.4.1 Communication and security ...... 21 3.4.2 Salts specific working mode ...... 21 3.4.3 Salts mechanics an terminology ...... 22 3.4.4 Salt DSL and templating engine ...... 25 3.5 Intermediate result of evaluation ...... 25

4 Public survey of configuration management tools 27 4.1 Installability ...... 30 4.2 Configurability ...... 31 4.3 Stability ...... 33 4.4 Scalability ...... 34 4.5 Usability/learning curve ...... 36

vii 4.6 ACL capabilities ...... 38 4.7 Documentation ...... 39 4.8 Support ...... 41 4.9 Training ...... 42 4.10 Supported Operating Systems ...... 43 4.10.1 OS Support of Ansible ...... 44 4.10.2 OS Support of Chef ...... 45 4.10.3 OS Support of Puppet ...... 46 4.10.4 OS Support of SaltStack ...... 46 4.11 Advantages ...... 47 4.12 Disadvantages ...... 48 4.13 Satisfaction ...... 49 4.14 Deployment properties ...... 50 4.14.1 Portability ...... 50 4.14.2 Installability ...... 51 4.14.3 Adaptability ...... 51 4.14.4 Configurability ...... 51 4.14.5 Distributability ...... 51 4.14.6 Scalability ...... 51 4.14.7 Stability ...... 52 4.15 Specification management properties ...... 52 4.15.1 Language ...... 52 4.15.2 Monitoring ...... 53 4.15.3 Versioning ...... 53 4.15.4 Integration ...... 53 4.15.5 Access Control ...... 53 4.16 Result of the public survey of CM tools ...... 54

5 Conclusion 59 5.1 Results in regards to the comparison of CM tools ...... 59 5.2 Results in regards to the public survey of CM tools ..... 59 5.3 In retrospect ...... 60

A Online survey 69

B CM tools code examples 77 B.1 Ansible ...... 77 B.1.1 Playbook example ...... 77 viii B.2 Chef ...... 78 B.2.1 Recipe example ...... 78 B.3 Puppet ...... 79 B.3.1 Manifest example ...... 79 B.4 SaltStack ...... 80 B.4.1 State file example ...... 80 B.5 Jinja template example ...... 81 B.6 ERB template example ...... 82

List of Tables

3.1 Overview of practical CM tools comparison 26 4.1 Overview of respondents occupation 29 4.2 Overview of public survey results 58

List of Figures

4.1 Installability graph 30 4.2 Configurability graph 32 4.3 Stability graph 34 4.4 Scalability graph 35 4.5 Usability graph 37 4.6 ACL graph 38 4.7 Documentation graph 40 4.8 Support graph 41 4.9 Training graph 42 4.10 OS Support graph 44 4.11 Advantages graph 47 4.12 Disadvantages graph 48 4.13 Satisfaction graph 49 4.14 Deployment properties graph 50 4.15 Specification Management properties graph 52

xiii

1 Introduction

In the enterprise of the present, when a large number of services is involved in the business of companies, there is a need for structure and consistency as well as a need for rapid deployments. Companies services usually consist of several parts, two of them being software and network infrastructure, e.g., servers. Many companies tend to use cloud services either for themselves or to provide their services to another enterprise. There is also part of an enterprise which for different reasons, for example, security, decides to host their servers in-house. Each of these worlds deals with the same challenges and are sig- nificantly affected by modern IT demands. The client usually wants to have their service up and to run as fast and stable as possible. As part of this trend, the term DevOps arose not limited to principles of agile development. DevOps allows companies to be able to compete on fast-growing market of nowadays. In practice, there are two approaches how one can look at this term. DevOps engineer might be a developer or a Software engineer, who is also able to perform system engineers task and vice versa [1]. "DevOps is a term given to a set of practices, ideals, and tools that are helping organizations of various sizes deliver value on their IT investments at quicker rate[2, p. 30]." As a part of DevOps, or more precisely one of the steps in this practice is methodology defining network infrastructure called Infras- tructure as a Code or shortly IaC. IaC resolves around the term of idempotence, which means the ability to set up and configure the same environment every time it is applied. Not only providing consistency to new systems but also keeping already running systems according to desired definitions as the way the environment should look and behave is defined. The administrator isn’t forced to manage a large number of servers, where each becomes unique during some period, but can keep the systems in the same stat [3]. When implementing IaC in the enterprise, it’s not possible to un- derestimate selection of the right tool, as wrong decision might result

1 1. Introduction in management dissatisfaction, delayed projects, and other unplanned complications. Thus, the thesis goals and objectives are to give insight on Infras- tructure as a Code, its history, principles, and characteristics. To discuss DevOps as a base for Infrastructure as a Code. The thesis also gives insight and explores modern trends in infrastructure automation, e.g., configuration management. The following text also evaluates configuration management tools as users around the world see them. Last but not least the thesis practically discusses different configuration management tools. The structure of the thesis is following:

Chapter 2, Infrastructure as a Code provides information on Infras- tructure as code, DevOps principles, and configuration management tools in general. Chapter 3, Comparison of configuration management tools com- pares different aspects of configuration management tools such as security, means of communication, etc. Chapter 4, The public survey of configuration management tools gives insight on how users perceive their CM tools in regards to properties such as installability, configurability, etc.

2 2 Infrastructure as a Code

2.1 History and DevOps background of IaC

In the past number of IT professionals was not as large as it is today. Over the years the complexity of systems and IT grew. Professions as a System administrator, System engineer as well as more specialized jobs as Quality assurance specialists, Database specialists or Security specialist arose. With this trend the need for automated processes became apparent. System administrators as a part of operations were responsible for keeping systems up and running. DevOps was in a way a result of this continual fast expansion of IT but isn’t as new as it might seem at first glance. The term DevOps is splittable into two separate words "Development" and "Operations". This particular methodology is a convergence of several principles, as the Lean movement, Agile principles, Theory of constraints, change management, quality assurance (QA), product management and others, two of the most notable being Lean and Agile. Agile arose around the end of the second millennium and is in a way successor to waterfall model, with its incremental releases, small changes, and highly self- motivated teams. There is an emphasis on delivering software in short time scale, coping with requirements changes, customer satisfaction, and continuous quality delivery. Application of these principles and processes on infrastructure level, as opposed to application code itself, contributed to DevOps creation [4, 5, 6]. Another core principle of DevOps lays in lean manufacturing. Lead time is of great importance when practicing lean, as it is considered to be an indicator of quality and satisfaction of the customer. The short delivery date is to be achieved mostly by working in small batches. Holding onto those principles should result in value creation, thus customers satisfaction. Lean tries to remove everything which is of no value to the customer, simplifying procedures as much as possible[7, 6]. The continuous delivery movement, which also participates greatly in DevOps and Agile respectively, is responsible for frequent and quick delivery of software and process automation in this aspect, which is what both of them have in common [8].

3 2. Infrastructure as a Code

DevOps itself has a broader scope, as it is more of a cultural change, bringing together different teams as QA, operations, development, and others [8]. It is apparent that DevOps is not simple in any way, it is complex software engineering culture and practice. All elements of DevOps draws from, while only some of them where described, align at some points with their ideas and complement each other thus creating DevOps. In the next section method called Infrastructure as a Code will be described in greater detail as it draws from DevOps principles.

2.2 Infrastructure as a Code as a part of DevOps

Enterprise of today has a fast-growing tendency when performing every process manually is no longer possible. Not only due to a number of operations to perform but also due to their complexities. In such occasions, automation gradually took over business procedures not only in IT but across the whole spectrum of sectors. Automation is what makes execution of different operations possible even when there is negligible or no human interaction involved. A process which is automated should perform operations faster, at lower costs, with fewer errors than a person responsible for a given task [9, 3]. The engineer can focus on business, rather than focusing on different complexities, thus creating space for innovation and higher productivity. Gathering of information, monitoring, deployments and other processes are possible and maintainable thanks to automation. Today, performing effective IT without automation is unthinkable as it contributes to stability, less manual work and time to market in a great deal [10]. Infrastructure as a Code makes great use of automation and participates in rapid deployments not limited to virtual environments. It allows system engineers to treat infrastructure as if it was software, which makes things like fast changes or version control possible while maintaining reliability and security. Before IaC extensive usage of scripting was utilized and it still is, but while similar, they have one great difference [10, 9].

4 2. Infrastructure as a Code

It is much easier to make the code behave more dynamically and operational system agnostic, even though it depends on the toolset. Code form is what makes IaC so popular, because it also serves as documentation of infrastructure, making it understandable to both development and operations – reflecting DevOps principles of bringing different departments of IT together. Utilization of IaC might be achieved via different automation tools frequently also called configuration management tools, based on principles of configuration management [9, 11, 3, 10].

2.3 Configuration management

Regarding IaC, configuration management means that system administrator or DevOps engineer shouldn’t configure new server instances manually, else configuration will become inconsistent over time, and those servers will become so-called "snowflakes". When practicing IaC specifically CM one should use only configuration management tools to make changes to infrastructure. CM didn’t come initially from IT, but today is frequently used and referred to as configuration management of servers. The integrity of documentation and product data and also change control are achieved by utilization of CM practices. The success of a project is said to be highly dependant on CM, be it software product or IaC as its necessity grows with the growth of a project itself. Unfortunately, companies often see CM as a costly liabil- ity and tend to ignore it. Avoiding CM during project development might result in chaos and lower productivity among the other issues. One of the advantages of CM is that, if practicing CM correctly, it should minimize the need for rework, which is both time-consuming and costly [12]. Configuration management has several key points:

Configuration identification: CM is responsible for identification of items to be controlled, the establishment of schemes to be able to identify items, selection of techniques and tools to be used during item management [12].

5 2. Infrastructure as a Code

Configuration control: CM concerns itself with the project, e.g., software life-cycle management. Configuration control is responsible for the selection of changes to be made, approval of such changes and supports the implementation of these changes [12].

Configuration Status Accounting: Configuration Status Accounting is a CM process that handles reporting and recording of information without which configuration management would not be possible [12].

Configuration Auditing: A repetitive process which ensures that products content, baseline integrity, and release integrity corresponds to its specification. Thorough planning is required for each audit, not only each task but also tools to be used during reviews. The audit is responsible for the evaluation of the degree to which the project satisfies desired specifications [12].

"The fundamental purpose of Configuration Management is to establish and maintain the integrity and control of software products throughout a project’s life cycle[12, p. 3]."

6 3 Comparison of configuration management tools

In the following chapters, I will be evaluating four modern configuration management tools, namely Ansible, Chef, Puppet, and SaltStack. In the scope of this thesis, it is not possible to cover every detail, but I’ve made a great effort in presenting the most critical mechanics and parts of CM tools to make the thesis as comprehensive as possible. Before the evaluation itself, in the following paragraphs, I provide essential information, which will not go into greater detail, but I believe they are needed even in smaller scope. The selection process of configuration management tool to be used in the real-world enterprise is not trivial, as many characteristics have to be taken into account. This chapter will present tool agnostic information, as well as tool overview. There are a lot of comparisons available on the internet, but data validity is low. Usually, reviewers only compare the essential general features, which is insufficient, without any real-world data or hands-on experience. Selection of right tools is critical as initial implementation of DevOps, or individually, IaC is time-consuming and often costly. The company has to hire DevOps professional or has to dedicate enough human resources to do the job. The realization, that company has chosen the wrong tool in the middle of the implementation process, might result in management dissatisfaction, lost time, delayed projects. With the help of configuration management system administrators can operate and configure machines on the network. CM tools users in general write specification of how to configure the system and howit should behave at the end of the configuration process [13]. Respective configuration management tools are then responsible for carrying out the configuration. As noted earlier configuration management is tightly coupled with the philosophy of DevOps, therefore is part of DevOps methodology. Even today it’s possible to automate different tasks with scripts, which is a method still widely used. There might be some pitfalls, scripts might sometimes fail, even if they are well written [13]. Frequently, in this cases, administrators fall back to ad-hoc fixing. This, over time, might lead to inconsistency among the same type of

7 3. Comparison of configuration management tools systems. The configuration can not be always reliably reproduced by manual configurations. The strength of CM tools lies in idempotence, which means, that the system always ends up in the same state. Configuration management gives administrator ability to describe the desired state as a code, so even developers are capable of reading, reviewing and rewriting. Tracking of any change or modification is possible thanks to version control system just as in case of any other source code. Engineer would not define what changes are supposed to happen, but what should be the outcome of those changes. As an example imagine software package installation. The user won’t define the task as "install this package", but rather "I want this package to be present at the end" – it is completely different point of view. Usually, configuration management tools work from a single central node, which is capable of managing subordinate nodes delivering the desired configuration. Again, the system which is controlled by configuration management should not be modified manually, asit falls back to the ad-hoc way administration [13]. Configuration management has two different ways of handling configurations, agent-less, and agent-based configuration management. Agent-less IT automation means that there is no need for any client running on the subordinate system. The idea of one (or several master server due to maintaining high availability and potential scalability) and multiple client-server doesn’t apply. Usually, they utilize some of the standard protocols where one of the most well known and widely used is SSH. Agent-based utilizes server-client paradigm, which means that it manages the configuration from the central point [14]. Selection of these specific tools is partially based on personal experience, as I was integrating IaC to company infrastructure. Another reason is the popularity of evaluated tools, as forum searches and google searches show. With keyword such as "popular CM tools" or "comparison of CM tools" one can find precisely these evaluated tools in almost all of the cases. I was unsatisfied with information these comparisons provided as they were too shallow and did not give enough information to be able to select the right software.

8 3. Comparison of configuration management tools

This chapter aims to bring an in-depth comparison of tools previously mentioned in several sections:

Communication and security How nodes communicate with each other and what means are used to secure the communication.

Specific working mode of evaluated tools What and how the CM tool uses its working mode (agent-less or agent-based).

Mechanics and terminology of evaluated tools Description of concepts and structure of evaluated CM tools.

DSL and templating engine of evaluated tools What DSL and the templating engine is used for these tools – presented with description and evaluation.

3.1 Ansible

Ansible is software found in 2012 and at the moment owned by Red Hat corporation [15, 16].

3.1.1 Communication and security Ansible relies purely on standardized, secure shell protocol also called SSH – most common implementation being OpenSSH. By default the whole network communication is encrypted. SSH utilizes several encryption methods such as hashing, symmetric cryptography, and asymmetric cryptography. Symmetric encryption is used to encrypt the entire connection, as it allows secure communication even with password only, without preshared keys, while also utilizing hashing during the transmission. Asymmetrical keys are used for authentication between nodes [17, 14].

3.1.2 Ansible specific working mode As previously mentioned Ansible commonly works with SSH, in the majority of cases by passwordless SSH key based communication. It is the only tool from all evaluated CM tools, which is strictly working only in agent-less mode. Ansible pushes the configuration to the server

9 3. Comparison of configuration management tools instead of pulling. There are more possible options off connection such as Paramiko SSH or Windows Remote Management. Targeting of multiple servers is possible by forking SSH connections, and if one wants to target a larger number of servers at once, it’s possible to override default values of forks depending on available system resources [15, 18, 19].

3.1.3 Ansibles mechanics and terminology Modules Ansible modules are the core of Ansible, as based on their purpose they can perform desired actions, they are later grouped into playbooks. They usually take advantage of Ansible setup module precisely client-specific information described later in the thesis [15]. To beable to operate on a subordinate node Python is required, as developers usually write Ansible modules in Python programming language. In case of Windows clients, Ansible uses PowerShell [15, 18, 20, 21]. Ansible performs the module execution in several steps. After the invocation of the module, the module is read into the memory, followed by arguments parsing. Variable and other arguments are parsed and passed to the module and saved to memory as well. Ansible then creates a connection to the target system creating a temporary directory, after that it closes the connection. As a next step, Ansible establishes other connection, and the object which was saved only in memory so far is copied to this temporary directory. It performs the execution of the module on the target system via the third SSH connection. As speed is always a concern, there are mechanisms overriding this default behavior of Ansible, which makes sure, that there is no speed loss in case of a large number of servers to manage. It is possible to perform these steps without the need to create separate SSH connections [22].

Playbooks Playbooks are text files written in YAML, which describe how the client node is supposed to look at the end of a configuration process. Ansible modules are used in the backend to perform certain operations,

10 3. Comparison of configuration management tools

allowed to perform them both synchronously and asynchronously. Playbook might be composed of multiple "plays", which means that there might be certain steps targeting database servers and a different set of steps to perform on an application server. Simple example is available in Appendix B.1 [23, 24].

Roles Roles are in its core prewritten playbooks, which load specific variable files, tasks, etc. and perform certain operations. One of Ansibles strongest points is Ansible galaxy, a repository of prewritten roles, which contributes greatly to speed and ease of use. Great positive of Ansible roles is that they can specify dependencies on other roles, making the work with Ansible even easier, by ’chaining’ different roles [14, 25].

Inventory files Opposed to other CM tools Ansible has information about hosts usually stored in one main file called inventory file, which stores information such as hostname, IP addresses, etc. Node-specific information about nodes might also be present. Ansible can target nodes individually, but can also target groups of hosts, differentiated by their purpose – this is not limiting client not to be present in more than one group. Inventory file is supposed to store only the necessary information, but also data such as variables (node specific and group-specific alike) might be available. Ansible is not limited to single inventory file, which might be both static or dynamic – the best practice is to split them per environments such as development, testing or production [23, 22].

11 3. Comparison of configuration management tools

Setup Name of the subsection might be misleading, but "setup" regarding Ansible is a module able to retrieve information or facts about the remote system such as IP address, amount of RAM, etc. The setup module is called by Ansible playbooks on each run to retrieve data, which, depending on the playbook, might be used during the run to determine what and how to run certain actions. Ansible is not limited only to its implementation of information gathering but is also able to communicate with Puppets software Facter described later in the thesis [26].

Vault Ansible Vault is a way to store sensitive information, as it gives one ability to store encrypted data within source code. Then either in an interactive or non-interactive mode, it encrypts and decrypts the data on behalf of a user [23].

3.1.4 Ansible DSL and templating engine Ansible DSL is not the perfect term in case of Ansible as it uses Jinja2 as it is templating system widely used in Python web application frameworks such as Django or Flask [27, 28]. Jinja makes it possible to use variables, loops, conditionals or filters (usually a form of Python methods) in playbooks or file templates. As noted earlier in the chapter, Ansible uses YAML as it’s data format which is then enhanced by Jinja 2 templating engine [22]. A short example of Ansible YAML and Jinja template is available in Appendix B.5.

Jinja2 Variables: One might define variables by wrapping the variable name in double curly braces: {{ variable }}. If used in a template, the variable name gets substituted by actual value. If the user wants to access variables nested in a dictionary, he can access them: {{top_dir.variable }}. Braces symbolize a print statement an are not part of the variable [29, 30].

12 3. Comparison of configuration management tools

filters: Filters are functions/methods in a way, which might be used to manipulate variables. Ansible has some unique filters, which are not part of plain Jinja. Statement such as {{ int_value | float }} presents one of many filters. One uses the regular print statement as in caseof plain variables, but with the pipe symbol "|", followed by the filter name, which pipes the variable to the filter. In this example, float converts the input into floating point number. Capabilities of filters are far beyond the simplicity of this example [29, 30].

Loops: As in any programming language, loops are used to perform specific steps repeatedly. {% for user in users %}

{% endfor %}

Tests: Purpose of tests is decision making as in programming to test the variable against another expression [29, 30]. {% if loop.index is divisibleby 3 %}

3.2 Chef

Chef is the most complex CM tool among all four evaluated software, at least from software number point of view as Chef in the most usual setup consists of three components. Apart from the standard structure where master and client are distinguished there is also a node called workstation. It is a software actively developed from the year 2009 [31, 32].

3.2.1 Communication and security

Several services and protocols are running on Chef server but the main communication between nodes, server and workstation are based on HTTP/HTTPS protocol. Chef secures the communication between nodes by SSL certificates, where server and nodes have to have matching certificates to authorize the communication [33, 34].

13 3. Comparison of configuration management tools

3.2.2 Chefs specific working mode Chefs workflow is a little different as opposed to other evaluated CM tools as it has three components workstation, server, and client. Chef itself is agent-based software where node pulls the configuration from the server. There comes the workstation, where the user prepares the configuration files [10, 35].

Workstation As a name suggests, the workstation is a node where DevOps engineer develops and tests their Infrastructure as a Code. The workstation is used to deploy Chef code to Chef server. Development of recipes and cookbooks is performed on this type of node – this includes maintaining of Chef repository and version control [10].

Server Similar to other CM tools, server software is a central node from which other subordinate nodes are managed and configured. IaC code is uploaded from workstation to the server and used to provision clients [10].

3.2.3 Chefs mechanics an terminology Knife A tool providing a way of communication between workstation and server, giving an engineer ability to upload their work to the Chef server. Also, it gives one the ability to download cookbook from the Supermarket – I describe both of these terms later [31].

Ohai Ohai provides functionality similar to Ansibles setup module, which means it gathers facts about clients. Ohai serves the same purpose. It collects information like operating system type, CPUs, RAM, etc. while extendable by plugins. Chef cookbooks later use this data collected from Ohai, so it is possible for a client to be brought to the desired state as described by DevOps engineer on a workstation [31].

14 3. Comparison of configuration management tools

Resources The resource is the smallest building block of Chef as resources describe operations such as "install package" or "start and enable service". Grouping of resources results in the creation of recipes [36].

Recipes Recipes are snippets of ruby code built from Chef resources defining what command are supposed to run on subordinate nodes – they define every step needed to configure part of the client system. A recipe might be included in another recipe and has to be stored in a cookbook [31, 37]. Recipes are organized and versioned as any other type of software. Basic recipe example is available in Appendix B.2.

Cookbooks "Cookbook is the most important and reusable component of a Chef server, and it is widely used by developers and infrastructure administrators because cookbooks the basic unit used to define rules and regulations for configuration purposes[10, p. 36]." Cookbooks are once again building upon smaller primitives, in this case, recipes, files, templates, etc. They are scenarios defining steps needed to perform complete configuration of a part of the system such as web server for example – the application itself, configuration files, etc. Every version of cookbook provides unique functionality and possibly bugfixes or new features [38, 39].

Run lists "Run list" is a node-specific set of steps necessary to configure the entire node to its desired final state. Run list notes every step in correct order consisting of recipes or/and roles [31, 40].

15 3. Comparison of configuration management tools

Roles A role is a grouping of cookbooks, recipes, and templates which results in provisioned client ready to serve its purpose – DB server, application server, etc. They are reusable and applicable independently always bringing the same result to a secondary system. They are usually created from run-lists and smaller primitives respectively [31, 41].

Data bags Data bag is a collection of data in JSON format specific to the deployment process. Chef server handles this data bag as a global variable. It is made to store sensitive data such as password, account information, etc [10].

Supermarket The supermarket is a community repository consisting of Chef cookbooks available for the public to speed up workflow. As Chef itself, it is possible to install on-premise private version [42, 43].

3.2.4 Chef DSL and templating engine Declarations of resources in recipes are written in Ruby DSL, precisely called Recipe DSL. If the specified property is present in a client- server, action is taken according to this property – this functionality is what majority of methods available in Recipe DSL provide. Any operation available in Ruby is also possible in Recipe DSL as in its core it is Ruby DSL, which means that engineer can use conditionals, loops, variables, etc [44]. I provide only the basics of DSL, the short example of Chef file template is available in Appendix B.6.

Erubis Chef DSL uses Erubis as it templating language, which gives Chef DSL ability to use and generate files from templates. This way Chef can provide not only configuration files for software [42].

16 3. Comparison of configuration management tools

Variables Variables are evaluated by wrapping variable name in <%= %> which is a print statement. This way one can print the value into the generated file [42].

Loops If one wants to use Ruby and Ruby logic in their files not limited to loops, it is defined with <%- %> [42]. Thus example of loop can look like this: <%- 4.times do %> <%= @var %> <%- end %>

Conditionals Below example describes how to use conditionals in templates. If the condition is met in this example, the template file is filled with list of users servers defined in "users" array– note the "<=" and "<-" [42]. <%- if @enabled %> <%- <%= @users.each end do |user| %> <%= user %> <%- end %> <%- else %> No users to choose! <%- end %>

3.3 Puppet

Puppet corporation develops Puppet from the year of 2005 [45].

3.3.1 Communication and security One of the software qualities demanded by engineers is security. Puppet operates and encrypts the communication with SSL certificates, while HTTP/S protocol handles the communication itself. The master node has HTTP endpoints handling ingoing and outgoing requests. Puppet agent communicates over this endpoint over HTTPS. Both communicating sides are supposed to have the matching SSL certificate to avoid any possible security attack or fraud [46].

17 3. Comparison of configuration management tools

Puppet master server has built-in certification authority which, upon request from Puppet agent might generate and sign this node specific certificate [46].

3.3.2 Puppet specific working mode As mentioned in previous chapters, there are two types of communication used in the scope of CM – agent-based and agent-less modes. Agent-less mode is called "masterless" mode in the terminology of Puppet, which means that every subordinate operating system has all of the configuration available on the host (agent node) itself [47, 48]. In case of masterless mode subordinate systems also handle the catalogs (to be described later) as well as their compilation. When using standalone-mode benefits such as centralization, usage of fewer system resources are rendered useless, but there is no overhead when communicating over HTTPS, etc[47, 48].

3.3.3 Puppets mechanics an terminology

Manifests Manifest are main configuration files of Puppet, which utilize the Puppet DSL. The main components of manifests are called resources, primitives on which manifests build upon. Similarly to other CM tools, resources defined in manifest files might be enforced onthe subordinate system, holding on the principle of idempotence. Separate steps are defined in groups of resources. These resource groups are defined by title, which has to be unique, uniqueness has to bealso applied to resources type per title [49]. Short manifest example is available in Appendix B.3. Puppet evaluates the manifest files in several steps:

∙ Puppet compiles Manifest assigned to a node into "Puppet catalog"

∙ order and dependencies defined are considered to run the catalog in the correct sequence

18 3. Comparison of configuration management tools

∙ state of the subordinate system is checked to decide if defined resources should be applied

∙ Puppet performs changes on the subordinate operating system

∙ Puppet provides an engineer with verbose output and results of steps performed [49]

Classes Puppet groups resources into classes, which might define a purpose of a system. This way of implementation helps to keep the code clean and reusable. As in programming, it is possible to extend classes with parameters, which are based on user input. If no default value is present, an engineer should always provide one [49].

Modules The code is kept clean by grouping Puppet classes into modules, which might be considered similar to Ansibles roles. The user should differentiate them in module-specific paths. Not only they store manifests (classes respectively), but they include data, templates, tests, etc. The structure is as follows:

∙ class can be created from resources

∙ manifest might consist of classes

∙ module can be created from the manifest [50]

Hiera Hiera is a Puppet tool collecting data for manifests, but different from Facts described below. Hiera gathers node specific user-defined information. The user can write Hiera data file in in YAML or JSON. Hiera is extensible by "eyaml" (encrypted yaml) backend giving it an ability to store sensitive data securely [49, 51].

19 3. Comparison of configuration management tools

Facts Facts are a node-specific information, built-in or programmed bya user, set by the administrator in various manners. They are used to perform host specific operations, making the CM dynamic and adjustable. Facts are gathered by ’Facter’ command on a puppet subordinate node also called an agent before Puppet master compiles a catalog. Puppet server compiles the catalog with collected data and sends them back to Puppet agent, which then triggers the configuration process [49, 48].

Puppet Forge Forge is a repository of user-written Puppet modules freely available to the public. It might speed up the overall workflow. It is similar to Ansible Galaxy and other repositories of similar CM tools [49, 48].

3.3.4 Puppet DSL and templating engine Puppet used Ruby DSL (which is also used as a base DSL for Chef) in the past, but Puppet developers removed it in newer versions of Puppet. Puppet DSL was created as a replacement with as stated by developers, with sysadmins in mind while being inspired by Nagios1 configuration file format. The main purpose of Puppet DSL isdo define classes and resources [52, 53].

Embended Ruby language ERB is a templating system used by Puppet, while similar to Erubis, it is not identical – Erubis is rendered faster than ERB. It is useful to generate and operate usually file templates, such as configuration files [54, 52].

Variables Identical to Chefs print statement "<%=" and "%>" symbols are used to evaluate the variable or generally speaking expression [55]. <%= @var %>

1. Nagios is an infrastructure monitoring software.

20 3. Comparison of configuration management tools

Loops Simple generic code snippet describing iteration over an array of values. Functionality is the same as in Erubis used by Chef [55]. <% @variables.each do |var| -%> To do operations <% end -%>

Conditionals Syntax is simillar to Erubis syntax mentioned in Chef chapter. If variable "positive" is set to true, logic inside the block is executed [55]. <% if @positive == true -%> logic here <% end -%>

3.4 SaltStack

SaltStack is an automation software created and maintained by SaltStack company, first released in 2011 and written in Python [56].

3.4.1 Communication and security Salt is CM tool which utilizes master-client architecture. It secures its communication with the public key, where every client key has to be allowed to communicate with the master server. On top of public key authentication lies AES encryption, which encrypts the messages between nodes, later handled by ZeroMQ, high performance asyn- chronous messaging library. Salt is also able to communicate via SSH in the agent-less mode described later – this way of communication is slower than ZeroMQ [56, 57].

3.4.2 Salts specific working mode Salt can operate in both agent-based and agent-less mode, while the agent-based is preferred. It can use both push and pull modes with Salt as a user can push the configuration to clients, but also clients are allowed to check for updates and pull them accordingly [58, 59].

21 3. Comparison of configuration management tools

Thus Salt is considered self-healing. In agent-less modes it operates similarly to Ansible, as it pushes the configuration to a temporary location, runs the commands and cleans up after [58, 59].

3.4.3 Salts mechanics an terminology Grains Grains are a Salt specific system to retrieve information about client nodes. The term comes from the fact that Salt presents grains of information to a user. Grains are to be usually used in state files to achieve dynamic and adaptable state, depending on different system properties. Grains consist of information about RAM, CPU, network interface and IP addresses, HDD, underlying operating system and wast number of other options. It is possible for one to write custom grains, as the default ones might not satisfy the needs in specific cases. There is a constraint as the grain should be written in Python and is supposed to return key: value-based dictionary [60].

Pillars Pillars are tree structures similar to grains in a way. The difference is that grains are stored on the client, as opposed to the pillar which is stored on the master server. One of the main purposes of pillars is to serve confidential data to clients, allowing to target only specific nodes which should be able to access these data. Another ability of pillar data is that they might serve as input for Salt states, which are then executed based on information from pillar. To be more general pillars can serve data to minions in the secure and node-specific manner [61].

Mine Mine is used to store data from minions on the master server, so one can retrieve this data when demanded. Mine data are more up to date compared to grains as their main purpose is to serve data when one minion needs information about another minion. Instead of connecting to each node separately to retrieve data, it is enough for a client to connect to master [56, 62].

22 3. Comparison of configuration management tools

Modules Execution modules are functions, which can handle different tasks like creating a user, managing databases, installing software, etc. They can be used in Salt states and formulas. Execution modules themselves are stateless, which means that they are run each time they are executed, not concerning themselves if it is necessary for them to run, or not. The second group are "state modules", which enforce a particular state of a client as they are state aware. [56, 62]

States State files are a definition of how the client node in caseofSalt called "minion" should look at the end of a configuration process. Salt state files are distinguishable by SLS extension and YAML language. Salt groups Salt states in so-called top SLS file, where all state files are noted. One can assign states to client nodes based on different properties like the purpose of the system, hostname, and many other properties. State file itself consists of Salt modules, which, together are creating a state of a system [62, 63]. Example of basic Salt state is available in Appendix B.4.

Top file The main state file is called "top file", as even its name in file structure should be "top.sls". This file contains the name of all state files to be used and made available to clients. One can target client and apply state respectively by grains, pillar or environment filtering, e.g., system purpose and other properties. Top file is not limited only to states, as the same principle isalso applied to pillars, which has the top file of its own. Pillars structure is defined in this file. Similar to state top file, it is possible tofilterwhich nodes will get what set of data [64].

Formulas Formulas are prewritten salt states usually by the community, making it sometimes more comfortable to perform specific tasks without the need to write state files from scratch. Unlike some other evaluated

23 3. Comparison of configuration management tools

CM tools, Salt doesn’t have software available to pull formulas straight from repositories. They are capable of bringing the desired state on their own, but a lot of Formulas is extensible by pillar data. Formulas are available in the official Salt GitHub repository [65, 66].

3.4.4 Salt DSL and templating engine Salt states, pillars and even configuration files of Salt are all written in YAML possibly combined with Jinja templating engine. It appeared to me that Salt utilizes Jinja in its State files more than Ansible in its playbook. Loops, variables, conditionals, and filters can be used the same way in state files and templates. Even tough Anible also hassuch capabilities. I’ve described Jinja templating engine in Ansible chapter, and I won’t be doing the re-evaluation, as the basics are identical [62, 56].

3.5 Intermediate result of evaluation

The third chapter has shown that evaluated CM tools share the same concepts and mechanics. Terminology, programming language, templating engine, etc. are different, but the core principles are identical. There are several key points:

∙ Every tool has similar means of gathering client information as Facter for Puppet, Ohai for Chef, etc.

∙ There are differences in particular DSL of evaluated software as might be seen in the appendix, but one can not avoid seeing similarities.

∙ Evaluated tools utilize only two templating engines – Jinja2 and ERB (Erubis) while integrating them in their way.

∙ All CM tools share the idea of configuration files, smaller building blocks and larger logical units.

To recapitulate on the findings from the previous chapter, I provide the short reference table.

24 3. Comparison of configuration management tools

Ansible Chef Puppet SaltStack Communication SSH HTTPS HTTPS ZeroMQ

Security AES, RSA SSL SSL AES etc.

Configuration file Playbook Recipe Manifest State file

Puppet DSL YAML Chef DSL YAML DSL

Templating engine Jinja2 Erubis ERB Jinja2

Table 3.1: Overview of practical CM tools comparison

Evaluated CM tools share the same concept, and for a user with no prior experience with CM, they might look overwhelming and without any significant differences. Information provided in the previous chapter is not enough to decide which CM tools are suitable for Enterprise automation. For this reason, next chapter evaluates this issue further a concerns itself with questions such as "installability", "configurability" and others. I believe that the next chapter dramati- cally enhances the understanding and pictures quality of evaluated tools.

4 Public survey of configuration management tools

This chapter builds on the result of the previous chapter, as it expands the evaluation process providing answers directly from users of evaluated tools with day to day experience. Section aims to bring insight on how users perceive these tools and their qualities. The non-random purposing sampling is used, as a specific group of the population has been targeted, because the configuration management tools are still not so well known and spread [67]. The questionnaire, available in Appendix A, is based on the study of Md Jahidur Rahman from the University of Oslo, where he evaluated different CM tools, even though two of them were Chefand Puppet [68]. Ansible and SaltStack were in their early days of development at a time of his study and were not evaluated by him. That is why it is not possible to do the direct comparison and possible changes in user perception of these CM tools, as it would not be meaningful to have only the partial comparison. Every section describes specific software properties and qualities and provides results with graphical representation. Part of questions is "satisfactory level" based, some questions are "value to the user" based to give insight on how users perceive different software properties and qualities and what characteristics might lead them not to choose specific software. Properties present in the evaluation are:

∙ Installability

∙ Configurability

∙ Stability

∙ Scalability

∙ Usability/ learning curve

∙ ACL capabilities

∙ Documentation

27 4. Public survey of configuration management tools

∙ Support

∙ Training

∙ Supported operation systems

∙ Advantages

∙ Disadvantages

∙ Satisfaction

∙ Deployment properties

∙ Specification management properties

I have aimed the survey at System engineers, DevOps engineers, Developers and other professionals with day to day experience with evaluated CM tools. Respondents come from all around the world, entirely from 12 countries. I have submitted the survey on Facebook in 3 different Facebook groups, where approximately 29 800 DevOps professional are gathered. The questionnaire was also given to members of mailing lists of evaluated CM tools, but due to rules of the respective mailing lists, members were disallowed to view the posts, and the posts themselves got deleted. Another large target group was on Reddit, precisely DevOps "subreddit" with approximately 44 000 users. From all these groups 40 answers were gathered. 20 for Ansible, 9 for Puppet, 6 for Chef and 5 for SaltStack. On the next page I present table, which summarizes occupation of respondents, as it reflects the findings in several cases.

28 4. Public survey of configuration management tools

Tool Occupation Security Engineer (DevSecOps) Senior DevOps coach Network Engineer (Operations) HPC Systems Engineer DevOps Engineer DevOps Manager Ansible Site Reliability Engineer Automation Engineer System Administrator System Engineer Software Engineer Software Developer Infrastructure engineer IT division manager System Engineer Senior Director of IT infrastructure Chef Cloud Engineer Consultant DevOps Engineer Site Reliability Engineer System Administrator PhD student Puppet IT Analyst DevOps Engineer Software Engineer Manager of Technical Training Consultant DevOps Engineer Software Developer SaltStack Software Engineer System Administrator

Table 4.1: Overview of respondents occupation

29 4. Public survey of configuration management tools

The low response rate (0.054 %) is surprising, as the sampling number was quite large, which might be due to several reasons:

∙ Respondents were not interested in such a survey.

∙ Number of users of such groups and forums active, so the questionnaire did not reach them at all.

∙ Postal types of research usually have low response rates in general [67].

4.1 Installability

I have asked the respondents how difficult they consider the installation process of configuration management tools they use. Properties such as some packages to be installed, number of steps needed to prepare the CM tools stock environment, as well as time consumption, resource consumption all taken into account – contributing to the overall difficulty level.

Figure 4.1: Installability graph

SaltStack among all of the tool seems to be the easiest to install as all 100% of users considers this tools installability to be very easy.

30 4. Public survey of configuration management tools

Second easiest with 55% of respondents seem to be Ansible. As the third when it comes to easiness came Puppet with 22.2% and last but not least Chef with its 16.7%. 66.7% of Chef considers its installation to be easy, Ansible is deemed to be easy to install by only 35% and Puppet by 22.2% of users. Puppets installation is considered normally difficult by 33.3% of its users, while Chef is considered this wayonly by 16.7% and Ansible by 5%. Only two tools are deemed difficult to install, Ansible by 5% of asked and Puppet by 11.1%. Puppet is considered to be the most difficult by 11.1% of its users. During the evaluation, I’ve installed every tool on Linux workstation. Results in regards of SaltStack corresponds with the installation procedure, as it was unproblematic and straightforward on both server and client. Installation of Ansible was even more comfortable. The reason for respondents to answer anything below normal might be due to the overall inexperience of a user, or in the past, it may have been more challenging, which is most likely also the case for Puppet, as the installation progresses without any issues. Chefs installation, on the other hand, was not as straightforward as the results show, because the installation failed and extra steps were needed, but the overall level of difficulty was not hight.

4.2 Configurability

In the survey, I have asked respondents how difficult it is to configure evaluated configuration management tools, both master and subordinate nodes taken into account – depending on the evaluated tool. The configuration process I had in mind was in a scope large enough so that one can use CM tool to its full potential. Connfigura- bility is one of the most important properties of software, as it greatly contributes to learning process during the integration of CM.

31 4. Public survey of configuration management tools

Figure 4.2: Configurability graph

From the total of 20 Ansible respondent, 35% of them considers Ansible configuration to be very easy, which is probably thanks to the overall simplicity of the configuration process. The majority of Ansible respondents considers configuration to be easy, precisely 55%. Only 10% of all respondents consider Ansible configuration to be of normal difficulty. Neither of respondents thinks of Ansible as difficult to configure. Chef consists of 3 separate software instances, which might contribute to overall configuration difficulty, even though the result shows otherwise. 33.3% percent of Chef users considers configurability of Chef to be very easy, rest of respondents making the total of 66.7% consider the configuration to be easy. Puppet configuration is considered very easy by 11.1% of users, the same percentage of respondents thinks that setup of Puppet is easy. 33.3% of asked looks at Puppet as normally difficult to configure. The rest of respondents making the total of 44.4% considers the configuration of Puppet to be difficult. Majority of SaltStack users looks at Salts configuration as very easy totaling 60% of users. The rest of respondents, both making it to 20%, considers Salts configuration to be easy or normal. When it comes to levels of difficulty from very easy to normal, it most likely depends on experience with configuring software in general, as some users might be more comfortable with such tasks

32 4. Public survey of configuration management tools

than others. Puppet seems to be the most difficult to configure, but as data shows, among respondents who considered Puppet to be difficult were people, which do not have to necessarily have in-depth experience in "Ops" e.g., system administration, as their occupation shows (Ph.D. Student, Consultant, and Software engineer). During the evaluation, I have only done the necessary configuration as it is not possible to simulate larger-scale deployment in a LAB environment – setup to this extent can not be considered demanding. Structure of configuration files1 is similar among the tools - with a background in system administration I believe them to be straightforward, as they are "variable: value" based. One has only to invest enough time to study the options to make the configuration reflect the needs of an organization.

4.3 Stability

I have asked the users how stable evaluated CM tools are based on their experience. How long CM tools can operate without a breakdown in a production environment – overall reliability is taken into account. As an example consider an SLA between two companies with 99.9% uptime guaranty. There is an outage because of software instability – in case of CM, it might be a bug not carrying out the configuration process as expected. Stability is one of the most important properties as it is always necessary for software integrated into production to be reliable [69].

1. Not configuration files as for deployment and configuration of nodes, butfor configuration of CM software.

33 4. Public survey of configuration management tools

Figure 4.3: Stability graph

When it comes to stability Puppet is considered to be the most stable among evaluated CM tools, making it to 66.7% of all users. This might be due to the fact, that Puppet is the oldest thus the most mature software among evaluated tools. Ansible is believed to be very stable by 40% of users, which is the same percentage as for SaltStack – 40%. Precisely 50% of Chef users consider Chef to be very stable. The most of SaltStack users, precisely 60% considers Salt to be stable. The number is different for Puppet, where this particular CM tool is deemed to be stable by 33.3% of users, which still contributes to Puppets high stability. Ansible is considered to be stable by less than half – 40% and Chef is believed to be stable by 50% of users. From all four evaluated CM tools, only Ansible is considered to be very unstable by its users – entirely by 5%. The same amount of users consider Ansible to be unstable, no other CM tool is regarded as this way by its users, which might be due to inadequate personal experience as the number is low. Ansibles stability is considered to be normal by 10% of respondents.

4.4 Scalability

Characteristic of the system, software, process, etc. which is capable of potential adaptation to higher load is called scalability. From

34 4. Public survey of configuration management tools personal experience, I might say that scalability capabilities are essential, as the number of systems to manage is continuously growing. If a tool has a bottleneck in regards to scalability, it can bring different issues in the future which might result in "hacks" and workaround, even change of software used. The question asked it this case was how many nodes is it possible to operate and manage with particular CM tool. Secondly, how the system performance is influenced by adding/removing resources or load. Increasing of system resources usually accomplish scalability – most often hardware, which contributes to the overall performance of the system. There are two types of scalability – vertical and horizontal scaling. Vertical scaling consists of increasing the resources like RAM, CPU or storage on a single node so that this node can keep up with the growth of demand. Horizontal scaling, on the other hand, utilizes increase of nodes to handle this load by adding another machine or VM for example. This particular question brings insight into how large systems are in practice managed by evaluated CM tools and also how large infrastructures they can handle overall [70].

Figure 4.4: Scalability graph

35 4. Public survey of configuration management tools

Ansible can be scaled up to more than 100 000 hosts, which is number suitable even for the largest of the enterprise according to 10% of respondents. 20% of respondents use Ansible to manage between 10 000 to 100 000 hosts. 35% of respondents consider Ansible capable of handling between 1000 to 10000 hosts. The rest of the group making the total of 35% use Ansible to manage between 100 to 1000 hosts – which is a scale large enough for medium sized. The majority of Chef users considers Chef to be able to manage more than 100 000 nodes, making it total of 50%. 33.3% of Chef respondents are running Chef in setup managing between 10 000 and 100 000 nodes. Only 16.7% of respondents manage between 1000-10000 nodes. Puppet is considered for large-scale enterprise where more than 100000 of nodes are managed by 62.5%. Exactly one-third of Puppet users think Puppet is suitable to manage between 1000 and 10 000 nodes. The remaining 12.5% uses Puppet to manage between 10 000 and 100 000 nodes. SaltStack percentage is quite even where exactly 20% of Salt users consider it suitable to manage between 100 and 1000 nodes, 1000 and 10 000 nodes and 10 000 - 100 000 nodes. Remaining 40% believes SaltStack to be suitable for 100 000 and more nodes.

4.5 Usability/learning curve

When integrating such methodology as IaC, it is important to consider the time needed to get to know such tool to a suitable level, as often it is demanded as fast as possible. Based on personal experience, when the period given to integrating the IaC is short, it might be the deciding factor. In this question, I have asked the respondents how difficult it is to get to know particular CM tool, e.g., how steepthe learning curve is and how difficult is to work with this tool. The extent to which product can be used by the user to achieve specified goals with effectiveness, efficiency, and satisfaction.

36 4. Public survey of configuration management tools

Figure 4.5: Usability graph

Both Ansible and SaltStack are considered to be very easy to learn and use by the similar percentage of users – 35% of Ansible users and 40% of SaltStack users. No other evaluated CM tools are believed to be very easy to learn. Tree of four of evaluated CM tools is considered easy to learn by a percentage of users - 40% of Ansible users, 33.3% of Chef users and 20% of SaltStack users. 25% of Ansible users, 50% of Chef users, 33.3% of Puppet users and 20% of SaltStack users believe that their respective tool is averagely difficult to learn. Respondents consider Chef, Puppet, and SaltStack to be difficult to study and use. Ansible was not mentioned in a group above average difficulty, which is because Ansible was built from scratch with simplicity and ease-of- use in mind. Only 16.7% of Chef respondents consider it to be difficult, SaltStack is believed to be difficult just by 20%. Puppet seems tobe viewed as the most difficult of all by 67.7%, which is more than ahalf. Reason for that is probably that many respondents do not operate Puppet as their primary job duty, with occupations like IT analyst, Manager of technical training, etc. On the contrary, based on the results Ansible seems to be the easiest to learn. During the evaluation I found Chef to be the most difficult, as it has three different components, which was overwhelming and seemed unnecessary – it depends on the personal workflow and preference.

37 4. Public survey of configuration management tools

As the second most difficult I consider Puppet. It is because Icome from Python background and I have had no prior knowledge of Ruby language. For somebody with a similar issue, this might be limiting. As Ansible and SaltStack have the very similar structure, they seemed rather easy to use [71].

4.6 ACL capabilities

I consider Access control list (ACL) to be one of the most critical aspects of software, more so in case, that several teams with differently skilled employees are working on the same project – usually in an enterprise. ACL is a data structure which consists of users and user groups. These groups, or also individual users might have to access different parts of the software, or their right is limited to the scopeof their competencies. ACL can be used to limit access to files, folders or particular parts of the software, etc. As an example – the contrast between administrator account and user account on the webpage [72].

Figure 4.6: ACL graph All SaltStack and Chef are utilizing ACL capabilities – both making it to 100%. These tools are suitable for large-scale deployments as a result has shown before, present usually in enterprise-level companies. Based on how work positions in corporations are typically laid

38 4. Public survey of configuration management tools

out, it is certain that ACL is utilized. Only 55.6% of Puppet users are aware of Puppets ACL capabilities, 22.2% of asked believes Puppet not to have ACL capabilities. 65% of Ansible users think that it has ACL capabilities, on the other hand, 15% believes it has not. This is most probably because there are two products Ansible and Ansible Tower, where Ansible Tower is enterprise level software with ACL [73]. 22.2% of Puppet users and 20% of Ansible users do not know if their respective tools are ACL capable, probably due to lack of need for such functionality.

4.7 Documentation

I have asked the users how well documented are steps such as installation, configuration, usage for particular CM tools. Quality of documentation is an important aspect when it comes to any technical or non-technical field. It is usually available in different formats such as online documents, booklets, etc. providing insight on how particular software works and how to use it. Importance of documentation is apparent not only for newcomers but even for developers refactor- ing their code, managers, etc [74]. "It is always beneficial to have detailed documentation about an application and its environments. Having this information readily available is invaluable when setting up new environments for an application and/ or maintaining existing ones for development, testing, and production[75, online]." This rule might be applied, but not limited to development, business, database engineering, troubleshoot- ing or application installation.

39 4. Public survey of configuration management tools

Figure 4.7: Documentation graph

More than a half of Ansible users consider its documentation to be very good, making it total of 55%. 20% of Ansible users find it’s documentation to be good and the same number of respondents thinks of Ansible documentation as normal. Only 5% of respondents consider Ansible documentation to be inadequate. Chef documentation is considered to be very good by exactly half of the respondents - 50%. Part of users making it up to 33.3% think of Chef documentation to be good, 16.7% considers Chef documentation to be normal. During evaluation of Chef documentation, it appeared to be quite confusing and not ideally structured, but the overall impression was good. Puppets documentation is considered good in average by 44.4% respondents. Some of the Puppet users making the total of 33.3% consider the documentation to be excellent. 22.2% of respondents think of documentation as normal. 40% of SaltStack users believe about Salts documentation to be very good. The rest of answers is divided between good, normal and very bad all of them making the total of 20%. Neither of evaluated tools had documentation which could be considered unsatisfying. In general, it was sometimes quicker to gather the information elsewhere, when I have searched for a specific piece of information. Among all four tools, I found Ansible documentation

40 4. Public survey of configuration management tools

to be the best as it was well structured and had all information needed. Salts documentation is lacking and average from my point of view.

4.8 Support

Support quality is essential not only for enterprise-level software but also for open source community support, especially when trou- bleshooting issues. In the survey I have asked the respondents how satisfied they are with overall support, be it enterprise technical support or community support and forums [76].

Figure 4.8: Support graph

35% percent of Ansible users considers it’s support to be very good, even though it’s the largest group, the percentage is quite average. 30% percents of Ansible users think of its support to be of good quality. Another 30% of respondents evaluated Ansibles support to be average, e.g., normal. Only 5% of Ansible users think of Ansible support to be poor, which might be due to an unsatisfactory experience. As for Chef, only 20% of asked people consider Chefs support to be very good. Majority of respondents making the total of 60% thinks of Chef support as good, the last 20% thinks of the Chefs support to be average – normal. 44% of Puppet users consider its support to be very good making it

41 4. Public survey of configuration management tools the majority of asked. 11.1% of asked considers Puppets support to be good. The rest of respondents thinks of Puppet support as normal or poor, both of them making it to 22.2% each. SaltStack support is above average, 40% percent of user thinks of it a very good. Majority of Saltack users making the total of 60% considers its documentation to be good.

4.9 Training

I have asked the users how satisfied they are with training in evaluated CM tools – official by CM tool vendor or training provided by the third party company. Unavailability or unawareness are both taken into account.

Figure 4.9: Training graph

As Ansible is developed by RedHat, which is also known for their certification programs, they provided official certified training [77]. We might assume that the majority of respondents have taken official RedHat training, as RedHat corporation is all around the world. Third party courses of Ansible training are quite widespread as well. 25% percent of respondents were satisfied with training provided. A large number of respondents, more precisely 50% do not know about any

42 4. Public survey of configuration management tools practice in general, as they might not need to take any courses. 5% percent of respondents think that Ansible hasn’t any training available, which might be due to the location of the respondent or overall unawareness. 10% percent of asked considers Ansible training courses to be average and another 10% are dissatisfied. Chef provides official training, but based on information on official website, they are running only in America, which limits Chef users when selecting a course. Apart from that, there are online courses available. 16.7% of asked considers Chef training courses to be satisfactory. 66.7%, which is majority thinks the classes to be average, e.g., neutral. 16.7% of Chef users do not know any courses [78, 79]. Puppet has both official and third party training available –both in-person training and online courses. In-person training is available in larger cities around the world [80, 81]. Exactly half of the Puppet users are satisfied with training provided in Puppet CM – 50%.25% percent of respondents consider Puppets training to be average. 12.5% of asked feel dissatisfaction and precisely the same percentage do not have any knowledge about available Puppet training. SaltStack provides training just like its competition, both official, and third-party. Apart from training courses, SaltStack also offers certified training in Salt automation. 40% of SaltStack users are satisfied with training provided. Majority of respondents – 60%, did not consider SaltStack training at all [80, 81].

4.10 Supported Operating Systems

In the questionnaire, I have asked the respondents what operating systems are supported, master and client respectively and what operating systems users use in practice. It is essential to know which operating systems are supported and utilized because also based on this information one can decide what CM tools is the most fitting, depending on the site’s infrastructure.

43 4. Public survey of configuration management tools

Figure 4.10: OS Support graph

4.10.1 OS Support of Ansible Ansible itself is based on Python, so it may be run anywhere, where an engineer can install Python backend. In case of Linux workstations, Ansible is usually available in repositories of several Linux distributions and might be installed by executing command which is generally used to install software on particular distribution type or installing directly from the vendor. The installation process is not tricky either in case of other Unix like system such as OSX or BSD, where the process is quite similar to one on Linux workstations. As for Windows machine serving as a workstation, Windows doesn’t come with python installed out of the box, nor it does support Ansible directly, but it is possible to install Ansible on Windows with underlying tools such as Cygwin2 or WSL3, which might make the installation not so pleasing [23, 84]. The only requirement of Windows subordinate node is to have remote execution of Powershell enabled because PowerShell manages the whole configuration of Windows machines [85]. Ansible is used as a workstation on Windows by 15%, which is most likely because it is not supported natively. 35% of Ansible users considers Ansible workstation to be Linux capable. 15% of asked use

2. Cygwin is a collection of GNU software compiled for Windows, providing Linux like environment [82]. 3. Windows Subsystem for Linux is a layer of compatibility to run Linux applica- tions [83].

44 4. Public survey of configuration management tools

Ansible as a workstation on BSD system and the same number on Solaris. Little higher percentage of users use OSX as their Ansible workstation, precisely 22.5%. 47.5% of asked use Ansible to manage Windows nodes, 45% manage Linux nodes. Solaris and BSD are both managed by 35% of users each. 40% of Ansible users control OSX machines. Percentage of Ansible users did not answer this question at all.

4.10.2 OS Support of Chef Installation of Chef Consist of three different primary steps, as Chef itself consists of three separate tools, which are to be installed to make the most of Chefs capabilities – ChefDK, Chef Server and Chef workstation From an engineers point of view, this might seem a lot of steps to perform to be able to use the Chef. Across the different platforms such as Linux, Windows, BSD, Chef is not available out of the box anywhere. Installation steps vary across the different host system [86]. Installation packages are officially supported on the major Linux distributions as well as OSX, Windows systems making the installation process quite easy and straightforward [87]. For other operating systems such as FreeBSD, it is still possible to install Chef, even tough official packages are not available. The Chef is programmed inRuby language, which has the capability of installing additional libraries, software, etc. with RubyGems. This particular software makes the building of ChefDK on FreeBSD possible [88]. Chef server can run only on Linux operating system, while official packages are available just for major Linux Distributions. That doesn’t mean that it is not possible to install Chef server to other distributions [89]. The most extensive variety of supported systems is by Chef client with officially supported packages. 66.7% of Chef users claim they are using Windows as the master server, which based on documentation should not be possible. Thus the result might be inaccurate, or the users might run Linux VM with Chef on Windows as one of the scenarios. 50% of Chef users are using Linux as their master server. Again, based on documentation only Linux is supported as master, but 16.7% of BSD and Solaris users, even 50% of OSX users claim they use their respective systems. All Chef users claim that Chef client software is available for Windows

45 4. Public survey of configuration management tools machines, 83.3 % of users are using Chef to manage Linux nodes. The rest of systems such as Solaris, BSD, and OSX are managed by 66.7% of users each.

4.10.3 OS Support of Puppet

Puppet consists of two separate software suites – Puppet master software and Puppet client software. Master server of Puppet runs only on Linux machines and is so stated in the documentation, even though the source code is available, and there has been an effort to bring Puppet master to other platforms such as free BSD. On Linux platform, an engineer can add Puppet software repositories making the installation easier with updates available as soon as they appear in repositories. As for Puppet client, it is available for all major platforms such as Linux, Windows, and OSX – officially. One can install Puppet on Windows and OSX workstation from official self-contained Puppet packages, which means that there is no other software necessary. Linux machines rely on Puppet software repositories. During evaluation installation of both client and master was carried out on Linux based machine, without any issues [90, 91, 92, 93, 94, 95]. As in case of Chef, a percentage of users claims they use Puppet master software on Windows and OSX, while not officially supported, users might have found a way to install Puppet server on these operating systems. Puppet server is used by 11.1% each on Windows, Solaris, and OSX. BSD is used as Puppet server by 22.2% and Linux by 88.9%. 100% of Puppet users manage Linux machines. 77% of asked use Puppet client on Windows machines, 77.8% on Solaris machines, 66.7% on BSD and only 55.6% on OSX.

4.10.4 OS Support of SaltStack

According to documentation, it is possible to install Salt minion on a wide variety of systems. SaltStack from all of the evaluated tools officially supports the most Operating Systems and Distribution – according to list available in the documentation. Installation is made easy by using official released packages and repositories, as wellas packages and installers for Windows and OSX workstations. In case

46 4. Public survey of configuration management tools

of OSX running Salt master on this kind of system is not officially supported, but according to following results it is possible to do so. Based on official documentation from listed operating systems SaltStack can run as master on Linux, BSD which is also supported by answers from SaltStack users by 20% each. 20% of users also claim that Salt master can run on Windows machine, which is not supported officially. When it comes to subordinate nodes, the support isbroad. 100% of users claim that Salt can manage Windows, Linux, and OSX. 80% of users consider SaltStack to be capable of managing BSD and Solaris [96].

4.11 Advantages

I have asked the respondents what software characteristic they consider to be an advantage of CM tool they use while giving value to each feature.

Figure 4.11: Advantages graph

Most of the users consider the flexibility of a tool to be the most important among all listed advantages, marked by 90% of users. 52.5% of all users consider good quality support to be an advantage of their respective CM tool. 77.5% of CM users appreciate the ease of use.

47 4. Public survey of configuration management tools

The same numbers of respondents consider documentation to be a significant advantage, precisely 77.5%. 75% of asked acknowledges the lower cost of CM tool. Security brought a surprising result, where only 67.5% of users consider it as an advantage. What is also surprising only 32.5% of respondents uses and considers monitoring as an advantage, which might be due to the fact, that they utilize dedicated monitoring software. Inventory tracking is deemed to be an advantage by precisely 50% of all users.

4.12 Disadvantages

I have the respondents what software characteristics they consider to be a disadvantage. If disadvantages outweigh the qualities, one might consider switching to different software. As a result below depictures, not a high percentage of users where dissatisfied in general.

Figure 4.12: Disadvantages graph

Disadvantages were not considered by users in general and the percentage of users considering provided option as disadvantages were low. Poor documentation was considered to be a disadvantage by only 7.5% of users. 17.5% of users consider CM tools to be difficult to use. High costs are considered to be a disadvantage by only 5% of users.

48 4. Public survey of configuration management tools

Only negligible 2.5% of users think that policy is to be considered a weakness. Poor support is believed to be a disadvantage by 10% of users. The same number of respondents considers lack of functionality to be a disadvantage among CM tools.

4.13 Satisfaction

I have given the CM tools users the question of how satisfied they are with tools they use while considering all previous questions and properties. "The degree of satisfaction provided by the goods or services of a company as measured by the number of repeat customers[97, online]." Customer satisfaction is important as it decides whether the customer will use the provided tool or not.

Figure 4.13: Satisfaction graph

Every tool is very satisfying to the users SaltStack with 80%, An- sible with totally 55% percent, Chef with 50% and Puppet with only 33.3%. 40% of Ansible users believed it o be satisfying, a similar number of Puppet users, precisely 44.4% are also satisfied. SaltStack is satisfying for 20% of users, Chef users are satisfied by 50%. 5% of An- sible users considers Ansible to be averagely satisfying hence normal, this level of satisfaction also applies to Puppet with 22.2% of asking.

49 4. Public survey of configuration management tools

For the results, Puppet seems to be the least satisfying, on the other hand, SaltStack appears to be the most satisfying.

4.14 Deployment properties

Question about deployment properties aimed to bring insight on what deployment properties users consider as valuable or invaluable to particular respondents. I have taken the values from the sum of all evaluated CM tools.

Figure 4.14: Deployment properties graph

4.14.1 Portability 2.5% of all respondents consider portability of CM tool to be not valuable, which might be due to a limited scope of OS used by respondents. 15% of asked thinks of portability as marginally valuable. Almost one quarter - 22.5% - of CM tools users think that portability is valuable. 35% of all respondents consider portability to ve very valuable property. Only 15% of CM users evaluate portability as extremely valuable. The rest, more precisely 10% of all users did not answer at all.

50 4. Public survey of configuration management tools

4.14.2 Installability 7.5% of all users did not answer about installability at all. 15% of survey respondents consider installability to be marginally valuable, a little more users - 22.5% - thinks about installability as valuable property. 35% of respondents values installability as very valuable and only 20% considers installability to be extremely valuable.

4.14.3 Adaptability Adaptability is considered to be extremely valuable by 22.5% of respondents, a larger number, precisely 42.5% of user thinks that it is very valuable. 22.5% of asked thinks that adaptability is valuable. The rest of respondents – 12.5% - did not answer at all.

4.14.4 Configurability The majority of asked considers configurability to be very valuable making the total of 52.5%. 35% of CM tools users think about configurability to be extremely valuable, and only 2.5% considers it to be valuable. The rest of 10% did not answer at all.

4.14.5 Distributability Distributabiliity is considered to be extremely valuable property by 17.5% of respondents. The most of user making precisely half – 50% believes distributability to be very valuable property. Distributability is considered to be valuable by 12.5% and marginally valuable by 10%. The rest of 10% did not answer at all.

4.14.6 Scalability Scalability is believed to be marginally valuable by 7.5% of CM tools users, the same number of respondents did not answer at all. Almost quarter, precisely 22.5% thinks that scalability is valuable CM tool property. Almost half – 47.5% of respondents consider scalability to be very valuable. The rest of user, totally 15% believes scalability to be extremely valuable property.

51 4. Public survey of configuration management tools

4.14.7 Stability Stability is believed to be very valuable by 52.5% of respondents which is more than a half. Quite a large number – 30% of users considers stability to be extremely valuable. Only 7.5% percent of respondents think that stability is valuable property. The rest did not consider stability at all, making a total of 7.5%.

4.15 Specification management properties

In the survey I have asked how valuable users find specific specification management properties, namely language, monitoring, versioning, integration and access control.

Figure 4.15: Specification Management properties graph

4.15.1 Language Available CM tools language is considered as not valuable by 2.5% of all respondents. 17.5% of CM tools users believe language to be marginally valuable. 30% considers CM tool language to be valuable. Something less than quarter considers language to be very valuable, precisely 22.5%. Language is extremely valuable to12.5% of users. The rest of users did not answer at all.

52 4. Public survey of configuration management tools

4.15.2 Monitoring

Value of monitoring was not considered at all by one-fifth of respondents. 5% of CM user considers monitoring to be not valuable. Monitoring is believed to be marginally valuable by 17.5% of respondents. 22.5% considers monitoring to be valuable and 27.5% considers it to be very valuable. Only 7.5% considers monitoring to be extremely valuable.

4.15.3 Versioning

Versioning is believed to be extremely valuable and very valuable by the same amount of asked, both making total of 25%. A larger number of respondents – 27.5% thinks versioning is valuable property. A small number of asked, precisely 5% considers versioning to be marginally valuable, and only 2.5% considers versioning not to be valuable at all.

4.15.4 Integration

Almost majority of users believes integration to be very valuable, precisely 45% of users. Almost quarter - 22.5% of asked - believes integration to be valuable property. 12.5% of users think that integration is extremely valuable. Only 5% considers integration not to be valuable. The rest of 15% did not consider integration at all.

4.15.5 Access Control

When it comes to access control, quite a large number o users did not consider it at all, precisely 17.5%. Only small percentage – 2.5% - believes that access control is not valuable property. 15% od asked considers ACL to be only marginally valuable. ACL is considered valuable by 22.5% of uses. The largest group consisted of user believing that ACL is very valuable, making the total of 32.5%. The rest making 10% thinks that ACL is extremely valuable.

53 4. Public survey of configuration management tools 4.16 Result of the public survey of CM tools

With the total number of 40 respondents - 20 for Ansible, 9 for Puppet, 6 for Chef and 5 for SaltStack, it is clear that Ansible is the most popular among the tools, while SaltStack is the least popular. Regarding installability, SaltStack seems to be the easiest to install based on data gathered followed by Ansible which is more comfortable compared to Chef. Puppet is the most difficult on the other hand which is apparent from the graph. The most accessible tool to configure is again SaltStack, while the second easiest is Chef according to data gathered. Third when it comes to configurability seems to be Ansible. Puppet is still the most difficult in its area. Readings are similar in regards to stability. Puppet is considered to be the most stable among the tools which I find understandable, as Puppet is the oldest of all evaluated tools, so it probably had the most space for improvement. Majority of users believe their tool to be stable or very stable, only Ansible is considered to be unstable by the small percentage of users. The most scalable among the evaluated tools were found to be Puppet while being comparable to Chef as both these tools seem to be made with scalability in mind. As the third most scalable appears to be SaltStack, the least scalable among the tools is considered Ansible. Among all of the tools, Ansible is considered to be the easiest to learn as neither of respondents considers Ansible to be difficult or very difficult to use. SaltStack is deemed to be a little more difficult when compared to Ansible, but easier compared to Chef. The most difficult tool to use is considered to be Puppet. Chef and SaltStack users utilize the ACL capabilities to their full extent. A large number of Puppet and Ansible users, on the other hand, either do not use ACL capabilities or are unaware of them at all. Puppet and Chef documentation is of the highest quality. Ansibles documentation is considered to be very good by the most significant percentage of user, but on the other hand percentage of users considers Ansible documentation to be inadequate, that’s why it can not be considered the best. SaltStacks documentation is the worst among the tools, as no other respondents consider their tool to have documentation of very bad quality. The more significant percentage of

54 4. Public survey of configuration management tools

users considers Salts documentation to be very good compared to Puppet on the other hand. I think that this area is highly dependant on which modules etc. one utilize, as documentation is often written by developers, as these projects are usually divided into open source version and enterprise version. SaltStack users are the most satisfied with their support. Chef and Puppet support provide similar readings, even though the number of Puppet users which consider their support to be of very good quality is greater than in case of Chef, quite large number of Chef users considers their support to be of good quality. On the contrary, neither of Chef users consider Chef support to be bad, which is precisely the case for some Puppet users, who find Puppets support to be of bad quality. Ansible support is considered to be bad by a smaller percentage of users than in case of Puppet, and also the higher number of Ansible user consider its support to be good and normal. Based on these readings I would say that Ansible and Puppet are on par when it comes to quality of support. A large number of users are not aware of any training or thinks that their respective tool doesn’t have any training available, which might be due to the area where respondents live, or they do not find it necessary to participate in any training. Thus they are unaware. It is not readily distinguishable to decide which tool has the most satisfying training, but according to reading Chef and SaltStack training might be considered the most satisfying as neither of the users was dissatisfied. Support of operating systems is broad among the tools, by official means of installation of user made workarounds. In this area, one would have to consider what kind of infrastructure he is going to manage and decide depending on available functions of the respective tool. Users from all evaluated tools consider flexibility, ease of use and documentation to be the most valuable advantages, on the other hand when the tool is difficult to use, has poor support or lacks functionality it is considered to be the most significant disadvantage. Considering the question of overall satisfaction were every previous aspect taken into account, most satisfied users are the ones of SaltStack. Ansible and Chef users seem to be similarly satisfied. The least satisfied seems to be users of Puppet, but they are not dissatisfied.

55 4. Public survey of configuration management tools

Deployment properties which users find the most valuable configurability, distributability, and stability which are the deciding factors for most of the users. Integration and Access control are the most valued specification management properties, as they are necessary for the enterprise environment.

Property Findings Installability According to users perception, Salt- Stack is the easiest to install. I find the basic installation of all four tools to be straightforward and easy while testing on Linux workstation. Configurability I couldn’t simulate enterprise level of infrastructure, but according to respondents, the most difficult to configure is Puppet. The rest of the tools are similarly demanding, while Salt seems to be the easiest. Stability In regards to stability, Puppet is considered as the most stable and based on the user experience Ansible seems to be the most unstable. Scalability According to respondents answers Chef and Puppet are the most scalable, thus the most suitable for the enterprise when considering scalability as deciding factor. Usability Ansible seems to be the easiest to grasp among all of the tools; Puppet is the most difficult. ACL Every CM tool is ACL capable according to users, which is property un- doubtedly necessary for enterprise.

56 4. Public survey of configuration management tools

Documentation I found Chefs and SaltStacks documentation to be the most confusing or lacking in case of Salt. According to users Ansible and Chef have the most satisfying documentation. Support Users of answers vary across different CM tools, but the support of Salt- Stack is the most satisfying according to the user’s experience. Training Training provided across CM tools vary without the precise result, sur- prisingly large number of users is not aware of any training. OS support According to findings, all tools have broad OS support with good distributability. One should decide pri- marily on the infrastructure which he is going to manage. Advantages The most valuable advantage to users is flexibility according to the results. What is considered to be a slight advantage of CM tools is monitoring. Disadvantages Users did not usually answer about disadvantages at all, small, but still, the most significant group of users founds difficulty to use the worst downside. Satisfaction The highest overall satisfaction was by SaltStack users, neither of the tools is dissatisfactory. Deployment properties Two of the most valued properties seem to be configurability and stability, closely followed by distributability and scalability, which is all critical software characteristics.

57 4. Public survey of configuration management tools

Specification management The most valued properties of the evaluated tools are integration and access control. I find access control the most critical specification management property in the enterprise.

Table 4.2: Overview of public survey results

58 5 Conclusion

This chapter reflects on the findings of the evaluation process – both comparison of CM tools and the public survey as well as my standpoint based on these findings.

5.1 Results in regards to the comparison of CM tools

The practical comparison of CM tools has shown that all CM tools share similar concepts as they have a similar structure. The notable differences were security and communication, as every tool handles this characteristic in their way. Tools usually use different ways of implementation of symmetric or/and asymmetric ciphers in the backend and are developed with security in mind. I find programming language the user is proficient in asasub- stantial factor to consider – tools utilize Python and Ruby. I found Ansible and SaltStack easier to understand, as they are Python based. Chef and Puppet seemed to be more robust and overwhelming. The perception of CM tool is strongly dependant on the user’s background and occupation as results reflect in some cases. CM tools utilize the same idea when it comes to the writing of configuration files and file templates, distinguished by DSL language and templating engine.

5.2 Results in regards to the public survey of CM tools

If ease of installation is deciding factor SaltStack and Ansible are the right software to choose, as they are the easiest among the tools. SaltStack appears to be the easiest to configure, followed by Chef, which depending on the value of this property to the user might be the deciding factor. The most stable among CM tools is considered Puppet, the least stable is considered Ansible - if stability is one of the critical properties to the users, Puppet is the right tool to choose according to users. In case that one is going to manage vast infrastructure right from the

59 5. Conclusion start, he should consider using Chef or Puppet, as they are the most scalable based on the readings. Be it learning curve and ease of use to be critical for the user he should avoid Puppet as it is the most difficult tool based on the survey results. The easiest and similarly difficult tools are SaltStack and Ansible, while Ansible seems to be slightly easier. While part of the respondents was unaware of ACL capabilities of CM tool they use, every tool has ACL capabilities. I find documentation of Puppet, Chef, and Ansible to be comparable in quality, while the SaltStack documentation is not as clear. In case that quality of support is critical for the user, he should consider SaltStack as his preferred CM tool. SaltStack and Chef have the most satisfying training among the tools so if a user has no previous experience with CM tools and considers a training course he should include SaltStack and Chef into the selection. In overall greatest satisfaction is felt by SaltStack users, the least satisfying is considered Puppet.

5.3 In retrospect

The thesis provided an extensive evaluation of configuration management tools considering the size of the thesis. It consisted of two evaluation parts, one of the being comparison of different aspects of CM tools and survey of CM tools followed by analysis. As findings show, the selection of the right software is not simple, as there are many factors to consider for the potential user. It is not possible to readily choose which tool is "the best", as each CM tool has its pros and cons, but it is possible to decide based on specific properties and value of such property to the user. I wouldrec- ommend to a potential user of such tool to test each of evaluated tools briefly, potentially excluding those with clearly unsuitable properties for the user. From my point of view and based on the readings I would suggest selecting CM tool depending on valuable properties, as they appear to the user. From the personal standpoint my recommendation is to use either Ansible or SaltStack as CM tool for enterprise automation.

60 Bibliography

[1] Ernest Mueller. What is DevOps. url: https://theagileadmin. com/what-is-devops. (accessed: 04.24.2018). [2] Trevor A. Roberts et al. DevOps for VMware Administrators. 1st ed. Upper Saddle River, NJ: VMWare Press, 2015. 384 pp. isbn: 978- 0-13-384647-8. [3] Andy Patrizio. What is ’infrastructure as code’ and why should you embrace it? url: https : / / www . cio . com / article / 3017722 / infrastructure / what - is - infrastructure - as - code - and - why-should-you-embrace-it.html. (accessed: 04.24.2018). [4] Kim Gene et al. The DevOps handbook. how to create world-class agility, reliability, & security in technology organizations. 1st ed. Port- land, OR: IT Revolution Press, 2016. 480 pp. isbn: 978-1-942788- 07-2. [5] Joakim Verona. Practical DevOps. 1st ed. Birmingham: Packt Pub- lishing Ltd., 2016. 240 pp. isbn: 978-1-78588-287-6. [6] Jennifer Davis and Ryn Daniels. Effective DevOps. building a culture of collaboration, affinity, and tooling at. scale 1st ed. Boston: O’Reilly, 2016. 410 pp. isbn: 978-1-491-92630-7. [7] Lean manufacturing. url: https://en.wikipedia.org/wiki/ Lean_manufacturing. (accessed: 04.24.2018). [8] Continuous delivery. url: https://en.wikipedia.org/wiki/ Continuous_delivery. (accessed: 04.24.2018). [9] Erika Heidi. An Introduction to Configuration Management. url: https : / / www . digitalocean . com / community / tutorials / an-introduction-to-configuration-management. (accessed: 04.24.2018). [10] Rishabh Sharma and Mitesh Soni. Learning Chef. 1st ed. Birming- ham: Packt Publishing Ltd, 2015. 316 pp. isbn: 978-1-78328-521-1. [11] Infrastructure as Code. url: https://en.wikipedia.org/wiki/ Infrastructure_as_Code. (accessed: 04.24.2018). [12] Norm Brown. Little book of Configuration Management. 1st ed. Ar- lington, VA: Arlie, 1998. 50 pp. [13] Evi Nemeth et al. UNIX and Linux System Administration Hand- book. 5th ed. Boston, MA: Addison-Wesley Professional, 2017. 1500 pp. isbn: 978-0134277554.

61 BIBLIOGRAPHY

[14] Fabio A. Locati. Learning Ansible. 2nd ed. Birmingham: Packt Publishing Ltd, 2016. 266 pp. isbn: 978-1-78646-423-1. [15] Lorin Hochstein. Ansible Up & Running. 1st ed. Sebastopol, CA: O’Reilly, 2015. 312 pp. isbn: 978-1-491-91532-5. [16] Alessandro Perilli. Why Red Hat Acquired Ansible. url: https: //www.redhat.com/en/blog/why-red-hat-acquired-ansible. (accessed: 04.24.2018). [17] Daniel J. Barret, Richard E. Silverman, and Robert G. Byrnes. SSH, the secure shell: the definitive guide. 2nd ed. Sebastopol, CA: O’Reilly, 2005. 672 pp. isbn: 978-0-596-00895-6. [18] How Ansible works. url: https://www.ansible.com/overview/ how-ansible-works. (accessed: 04.24.2018). [19] Connection plugins. url: https://docs.ansible.com/ansible/ latest/plugins/connection.html. (accessed: 04.24.2018). [20] Working with modules. url: http://docs.ansible.com/ansible/ latest/user_guide/modules.html. (accessed: 04.24.2018). [21] Connecting to Windows hosts. url: https://www.ansible.com/ blog/connecting-to-a-windows-host. (accessed: 04.24.2018). [22] Jesse Keating. Mastering Ansible. 1st ed. Birmingham: Packt Pub- lishing Ltd, 2015. 236 pp. isbn: 978-1-78439-548-3. [23] Jeff Geerling. Ansible for DevOps. Server and configuration management for users. 1st ed. Lean Publishing, 2018. 390 pp. isbn: 978-0-9863934-0-2. [24] Intro to Playbooks. url: docs.ansible.com/ansible/latest/ user_guide/playbooks_intro.html. (accessed: 04.24.2018). [25] Role dependencies. url: docs.ansihttp://docs.ansible.com/ ansible/latest/user_guide/playbooks_reuse_roles.html# role-dependencies. (accessed: 04.24.2018). [26] Facter module. url: http://docs.ansible.com/ansible/latest/ modules/facter_module.html. (accessed: 04.24.2018). [27] Templates. url: https://docs.djangoproject.com/en/2.0/ topics/templates/. (accessed: 04.24.2018). [28] Templates. url: http://flask.pocoo.org/docs/1.0/templating/. (accessed: 04.24.2018). [29] Playbook fitlers. url: http : / / docs . ansible . com / ansible / latest / user _ guide / playbooks _ filters . html. (accessed: 04.24.2018).

62 BIBLIOGRAPHY

[30] Jinja Templates. url: http://jinja.pocoo.org/docs/2.10/ templates/. (accessed: 04.24.2018). [31] Earl Waud. Mastering Chef Provisioning. Learn Chef Provisioning like a boss and finally own your infrastructure. 1st ed. Birmingham: Packt Publishing Ltd, 2016. 235 pp. isbn: 978-1-78588-891-5. [32] Jesse Robins. Announcing Chef. url: https://blog.chef.io/ 2009/01/15/announcing-chef. (accessed: 04.24.2018). [33] Server security. url: https://docs.chef.io/server_security. html. (accessed: 04.24.2018). [34] Server firewalls and ports. url: https://docs.chef.io/server_ firewalls_and_ports.html. (accessed: 04.24.2018). [35] Chef client overview. url: https://docs.chef.io/chef_client_ overview.html. (accessed: 04.24.2018). [36] Resource. url: https : / / docs . chef . io / resource . html. (accessed: 04.24.2018). [37] Recipes. url: https://docs.chef.io/recipes.html. (accessed: 04.24.2018). [38] John Ewart. Chef Essentials. Discover how to deploy software, manage hosts, and scale your infrastructure with Chef. 1st ed. Birming- ham: Packt Publishing Ltd, 2014. 201 pp. isbn: 978-1-78398-304-9. [39] Cookbooks. url: https://docs.chef.io/cookbooks.html. (accessed: 04.24.2018). [40] Run lists. url: https://docs.chef.io/run_lists.html. (accessed: 04.24.2018). [41] Roles. url: https : / / docs . chef . io / roles . html. (accessed: 04.24.2018). [42] Matthias Marschall. Chef Infrastructure Automation Cookbook. 2nd ed. Birmingham: Packt Publishing Ltd, 2015. 256 pp. isbn: 978-1- 78528-794-7. [43] Supermarket. url: https://docs.chef.io/supermarket.html. (accessed: 04.24.2018). [44] DSL recipe. url: https://docs.chef.io/dsl_recipe.html. (accessed: 04.24.2018). [45] Puppet (software). url: https : / / en . wikipedia . org / wiki / Puppet_(software). (accessed: 04.24.2018). [46] Catalogs. url: https://puppet.com/docs/puppet/5.0/architecture. html#catalogs. (accessed: 04.24.2018).

63 BIBLIOGRAPHY

[47] Subsystem - agent/master communication. url: https://puppet. com/docs/puppet/5.0/subsystem_agent_master_comm.html. (accessed: 04.24.2018). [48] Thomas Uphill. Mastering Puppet. Master Puppet for configuration management of your systems in an enterprise deployment. 2nd ed. Birmingham: Packt Publishing Ltd, 2016. 255 pp. isbn: 978-1- 78588-810-6. [49] Jo Rhet. Learning Puppet 4. A Guide to Configuration Management and Automation. 1st ed. Sebastopol, CA: O’Reilly, 2016. 547 pp. isbn: 978-1-491-90766-5. [50] Starting Puppet basics from a puppet labs employee. url: https:// puppet.com/blog/starting-puppet-basics-from-a-puppet- labs-employee. (accessed: 04.24.2018). [51] Configuring Hiera. url: https://puppet.com/docs/puppet/5. 3/hiera_config_yaml_5.html. (accessed: 04.24.2018). [52] Language: Using templates. url: https : / / puppet . com / docs / puppet/5.5/lang_template.html. (accessed: 04.24.2018). [53] Ruby DSL. url: https : / / puppet . com / blog / ruby - dsl. (accessed: 04.24.2018). [54] Erubis. url: https://github.com/kwatch/erubis. (accessed: 04.24.2018). [55] Language: Embedded Ruby (ERB) template syntax. url: https:// puppet.com/docs/puppet/5.3/lang_template_erb.html. (accessed: 04.24.2018). [56] Anirban Saha. Salt Cookbook. 1st ed. Birmingham: Packt Publish- ing Ltd, 2015. 329 pp. isbn: 978-1-78439-974-0. [57] ZeroMQ. url: https://en.wikipedia.org/wiki/ZeroMQ. (accessed: 04.24.2018). [58] Margaret Rouse. SaltStack. url: https://searchitoperations. techtarget.com/definition/SaltStack. (accessed: 04.24.2018). [59] Agentless Salt. url: https://docs.saltstack.com/en/getstarted/ ssh/index.html. (accessed: 04.24.2018). [60] Grains. url: https://docs.saltstack.com/en/latest/topics/ grains/. (accessed: 04.24.2018). [61] Pillar Walktrough. url: https : / / docs . saltstack . com / en / latest/topics/tutorials/pillar.html. (accessed: 04.24.2018). [62] Joseph Hall. Mastering SaltStack. 2nd ed. Birmingham: Packt Pub- lishing Ltd, 2016. 378 pp. isbn: N 978-1-78646-739-3.

64 BIBLIOGRAPHY

[63] How do I use Salt states. url: https://docs.saltstack.com/en/ latest/topics/tutorials/starting_states.html. (accessed: 04.24.2018). [64] The Top file. url: https://docs.saltstack.com/en/latest/ ref/states/top.html. (accessed: 04.24.2018). [65] Salt Formulas. url: https://docs.saltstack.com/en/latest/ topics/development/conventions/formulas.html. (accessed: 04.24.2018). [66] SaltStack Formulas. url: https://github.com/saltstack-formulas. (accessed: 04.24.2018). [67] Kate Kelley et al. “Good practice in the conduct and reporting of survey research”. In: International Journal for Quality in Health Care (2003). [68] Jahidur Rahman. “Investigating Configuration Management Tools Usage in Large Infrastructure”. PhD thesis. University of Oslo, May 2012. [69] What is a Service Level Agreement (SLA)? url: https : / / www . paloaltonetworks . com / cyberpedia / what - is - a - service - level-agreement-sla. (accessed: 04.24.2018). [70] Scalability. url: https://en.wikipedia.org/wiki/Scalability. (accessed: 04.24.2018). [71] Arriving at simplicity. url: https://www.ansible.com/blog/ 2013/12/07/arriving-at-simplicity. (accessed: 04.24.2018). [72] Access Control List (ACL). url: https://www.techopedia.com/ definition / 24766 / access - control - list - acl. (accessed: 04.24.2018). [73] Security. url: http://docs.ansible.com/ansible-tower/3.2. 3/html/userguide/security.html. (accessed: 04.24.2018). [74] Documentation. url: https://en.wikipedia.org/wiki/Documentation. (accessed: 04.24.2018). [75] Irma Azarian. Why is Documentation Extremely Important for De- velopers? url: https://www.seguetech.com/why-is-documentation- extremely-important-for-developers/. (accessed: 04.24.2018). [76] Technical support. url: https : / / en . wikipedia . org / wiki / Technical_support. (accessed: 04.24.2018). [77] Ansible Courses. url: https://www.ansible.com/products/ training-certification. (accessed: 04.24.2018).

65 BIBLIOGRAPHY

[78] Chef Fundamentals. url: https : / / www . cbtnuggets . com / it - training/chef-fundamentals. (accessed: 04.24.2018). [79] Chef Essentials Workshop - Official Chef Training. url: http : / / techtowntraining.com/courses/chef-configuration-management/. (accessed: 04.24.2018). [80] SaltStack Training Services. url: https://everythingshouldbevirtual. com/automation/ansible-using-ansible-on-windows-via- cygwin/. (accessed: 04.24.2018). [81] Training. url: https://puppet.com/support-services/training. (accessed: 04.24.2018). [82] Cygwin. url: http://www.cygwin.com. (accessed: 04.24.2018). [83] Windows Subsystem for Linux. url: https://en.wikipedia.org/ wiki/Windows_Subsystem_for_Linux. (accessed: 04.24.2018). [84] Larry Jr. Smith. Ansible - Using Ansible on Windows via Cygwin. url: https://everythingshouldbevirtual.com/automation/ ansible-using-ansible-on-windows-via-cygwin/. (accessed: 04.24.2018). [85] Windows Guides. url: http : / / docs . ansible . com / ansible / latest/user_guide/windows.html. (accessed: 04.24.2018). [86] An Overview of Chef¶. url: https : / / docs . chef . io / chef _ overview.html. (accessed: 04.24.2018). [87] Chef Development Kit. url: https://downloads.chef.io/chefdk. (accessed: 04.24.2018). [88] Building ChefDK on FreeBSD. url: https://shawnwilsher.com/ 2017/01/building- chef- dk- on- freebsd- 11- 0. (accessed: 04.24.2018). [89] Chef Server. url: https://downloads.chef.io/chef-server. (accessed: 04.24.2018). [90] Installing Puppet agent: Linux. url: https://puppet.com/docs/ puppet/5.0/install_linux.html. (accessed: 04.24.2018). [91] Puppet master server. url: https://puppet.com/docs/puppet/ 5.0/install_windows.html#puppetmasterserver. (accessed: 04.24.2018). [92] Installing Puppet agent: macOS. url: https://puppet.com/docs/ puppet/5.0/install_osx.html. (accessed: 04.24.2018). [93] Installing a Puppet Master. url: https://wiki.freebsd.org/ Puppet/GettingStarted#Installing_a_Puppet_Master. (accessed: 04.24.2018).

66 BIBLIOGRAPHY

[94] Getting Started with Puppet on Oracle Solaris 11. url: http : / / www.oracle.com/technetwork/articles/servers-storage- admin/howto-automate-config-datacenter-2212734.html#3. (accessed: 04.24.2018). [95] WIP FreeBSD Puppet 5 ports. url: https://github.com/smortex/ puppet5. (accessed: 04.24.2018). [96] Supported Operating Systems. url: http://saltstack.com/wp- content/uploads/2016/08/SaltStack-Supported-Operating- Systems.pdf. (accessed: 04.24.2018). [97] Customer satisfaction. url: http : / / www . businessdictionary . com / definition / customer - satisfaction . html. (accessed: 04.24.2018).

A Online survey

Below pages show the questionnaire of SaltStack, every other questionnaire had the same structure and questions.

69 A. Online survey

70 A. Online survey

71 A. Online survey

72 A. Online survey

73 A. Online survey

74 A. Online survey

B CM tools code examples

B.1 Ansible

B.1.1 Playbook example

- name: Install and configure httpd hosts: 192.168.1.56 become: true

tasks: - name: Install Apache yum: name: httpd state: present

- name: Install custom Apache configuration file template: src: /srv/ansible/production/httpd/configs/httpd.conf dest: /etc/httpd/conf/httpd.conf owner: httpd group: httpd

- name: Start and enable HTTPD service: name: httpd enabled: yes state: started

The shown basic and simple Playbook example targets the node on a private network with "hosts" directive. The "become" statement runs the Playbook with elevated rights on the target system. Ansible has of course much more possibilities, then shown in this example. Steps performed in this Playbook are respectively:

∙ Installation of httpd (Apache) software package

77 B. CM tools code examples

∙ Copy of custom httpd.conf is copied over to the targeted node and Ansible makes sure that the target file is identical to the source file, each time the playbook is running.

∙ Playbook starts the httpd service and enables it on system startup.

B.2 Chef

B.2.1 Recipe example yum_package "httpd" do vaction :install end service "httpd" do action [ :enable, :start ] end cookbook_file "/etc/httpd/conf/httpd.conf" do source "/srv/ansible/production/httpd/configs/httpd.conf" owner "apache" group "apache" action :create end service "httpd" do action [ :enable, :start ] end

The example shows the same set of steps, as in case of Ansible, but as you might see, the directives are different.

78 B. CM tools code examples B.3 Puppet

B.3.1 Manifest example package { "httpd": ensure => installed, }

file { "/etc/httpd/conf/httpd.conf": ensure => "present", source => "/srv/puppet/production/httpd/configs/httpd.conf", owner => "apache", group => "apache", }

service { "httpd": ensure => running, enable => true, } This example depictures once again the same set op steps showing syntax differences.

79 B. CM tools code examples B.4 SaltStack

B.4.1 State file example httpd: pkg.installed

/etc/httpd/conf/http.conf: file.managed: - source: /srv/puppet/production/httpd/configs/httpd.conf - user: apache - group: apache httpd: service.running: - enable: True

Shown Salt state file example has the same result as the previous examples of other CM tools.

80 B. CM tools code examples B.5 Jinja template example

ServerName {{ virthost.name }} {% if virthost.serveralias is defined %} ServerAlias {{ virthost.alias }} {% endif %} {% if virthost.docroot is defined %} docroot "{{ virthost.docroot }}" {% endif %}

{% if virthost.docroot is defined %} AllowOverride {{ virthost.allow_override \ | default(apache_allow_override) }} Options {{ virthost.options | default(apache_options) }} {% if apache_virthosts_version == "2.2" %} Order allow,deny Allow from all {% else %} Require all granted {% endif %} {% endif %}

I use the \ symbol in the template to symbolize a line break not to have the line too long; it would not be present in the real template. This Example of Apache virtual host shows basic Jinja mechanisms. Note the conditionals symbolized by {% if %} and {% endif %}. Variables are encapsulated in curly braces {{ }} and are printed into the file upon evaluation. The "is defined" directive tells the conditional to perform steps defined in the template if the respective variable is defined. Also look at the | default() filter. In case of this example, it passes the default values to the virthost file, if the variable preceding is not defined.

81 B. CM tools code examples B.6 ERB template example

:<%= @listen_port %> ServerName <%= @virthost.name %> <% if defined? @virthost.alias %> ServerAlias <%= @virthost.alias => <% end -%>

<% if defined?virthost.docroot %> "> AllowOverride <%= @virthost.allow_override \ || @apache_allow_override %> Options <%= @virthost.options || @apache_options => <% if @apache_virthosts_version == "2.2" %> Order allow,deny Allow from all <% elseif %> Require all granted <% end %> <% end %>

This basic example outputs the same file as the previous template written in Jinja. Note the differences. Istead of is defined Ansible functionality if defined?variable is used. For variable evaluation <%= %> syntax is used. Instead of default() filter used with Ansible, ERB can use || operator.