Puppet & in the DevOps Toolchain

1 A discussion regarding tools and DevOps should Introduction therefore begin by considering the individuals who will be utilizing the tools. The rise of so- DevOps Depends on People called “polyglot ” and systems In a few short years DevOps has gone from administrators with coding proficiency reflects a fringe movement to a must-have for any a general trend in IT towards despecialization. IT leader. There’s a lot of buzz around it, but Developers these days are adept in a number there’s a lot of practical knowledge in there of languages and approaches, applying each as well. Provisioning environments, deploying accordingly based on the problem at hand. applications, maintaining infrastructures--these Similarly, most systems administrators are all critical yet delicate tasks traditionally possess competent programming abilities for done by hand. What if we could get a machine to traversing the stack-- on top of the requisite do all that stuff for us, not just saving hours of skills for managing IT operations. The industry work, but also removing the element of human has been quick in attaching new labels to these error? emerging hybrid roles: DevOps Engineer and DevOps Specialist being the most common. So began the boom in Notwithstanding, the key takeaway is that no (CM) tools. Some regard CM solutions as single IT skill is more important or valuable than “DevOps in a Box,” but that’s not right. DevOps another; subsequently, many different tools are is about collaboration between people, while required to do the job effectively. So as DevOps CM tools are just that: tools for automating the is comprised of a group of concepts clustered application of configuration states. Like any around the premise of continuous other tools, they are designed to solve certain delivery, these concepts in turn encompass a problems in certain ways. How effectively they range of associated tools for fulfilling particular do so depends on the knowledge and ability of functions. the person wielding them. or Chef? So when it comes to CM automation solutions, should you choose Puppet or Chef? This age old question of configuration automation &

2 Understanding the DevOps Toolchain

Project Infrastructure changes are tracked as tickets in UpGuard Examples: Management and sent to your project management tool of choice. Jira, Asana, Pivotal

Requirements The requirements of current applications are avaliable in Examples: Gathering UpGuard’s system state documentation. Word, Wikis, Spreadsheets

Artifacts Code and UpGuard policies are versioned and Examples: Versioning checked in to be used in build and deployment process Git, SVN

Continuous integration and deployment tools use Examples: Continuous UpGuard policies to validate environments before Jenkins, Team City, Travis, Integration and after deployment. CircleCI, Drone.io

UpGuard generates manifests for configuration Configuration Examples: management tools like Puppert, Chef, Powershell DSC, Management Puppet, Chef, Ansible, and more.

Configuration state is continuously checked for deviation Examples: Monitoring from baseline, much like you would with Shell Scripts performance monitoring.

Complete configuration state is documented and Examples: Discovery accessible for anomaly analysis and troubleshooting. CMDBs

After confirming the system state’s health, Examples: New Baseline UpGuard documents the new baseline None for development.

UpGuard provides the feedback mechanism from the Examples: Standardize end of one development cycle to the beginning of the next. None

3 Versioning and Source Control Configuration Management (CM) Tracking code level changes is a common CM tools allow one to define the desired state and necessary activity of today’s software of a system and/or environment in regards to developers. Doing so enables concurrent configuration files, software installed, users, development, merging, and rollback capabilities groups and many other resource types. They for applications/software. Source Control also provide functionality to automatically push Management (SCM) tools are popular options for changes onto specific machines, also known as keeping track of software code; many DevOps automation and orchestration. This is where practitioners also track versions of their systems Chef and Puppet live in the toolchain. Let’s take configuration with these tools, essentially a closer look now at CM tools to understand how managing their infrastructure “as code.” For Chef and Puppet compare in this role. example, it’s a common practice for systems administrators to store and manage their Puppet Configuration Manifests or Chef Cookbooks in GitHub.

Continuing Integration Management and Orchestration (CI) and orchestration Tools tools enable the integration of development code Aside from Puppet and Chef, which we will look into the overall software product frequently at in detail, there are other CM tools available on and early in order to mitigate potential conflicts the market. down the line. Typically, these tools are employed to automate software builds and testing, and are CFEngine crucial for applying quality control on a continual CFEngine runs on C, as opposed to Puppet’s basis (as opposed to after the software has been use of Ruby. C is the more low level of the two developed and released). These tools can also languages, and one of the main complaints be used to track and manage changes for CM-- regarding CFEngine is that the learning curve is for example, CI tools can be used to test Chef very steep. It does mean though that CFEngine Cookbooks and Puppet Manifests for bugs and has a dramatically smaller memory footprint, it errors, and be configured to do so automatically runs faster and has far fewer dependencies. every time infrastructure changes are committed and merged. SaltStack Salt, like Ansible, is developed in Python. It was Testing and Validation also developed in response to dissatisfaction Tools and frameworks for testing and validation with the Puppet/ Chef hegemony, especially their are important for ensuring quality at all phases slow speed of deployment and restricting users of development. In many cases, unique solutions to Ruby. Salt is sort of halfway between Puppet are applied to a specific aspect of testing-- for and Ansible – it supports Python, but also forces example, one tool may be used for unit testing users to write all CLI commands in either Python, while another is used for integration testing. or the custom DSL called PyDSL. It uses a master

4 and deployed agents called minions to Now that we’ve seen some of the various control and communicate with the target servers, offerings of Configuration Management tools but this is implemented using the ZeroMq for DevOps, let’s dive into the two most popular messaging at the transport layer, which solutions for configuration automation, Chef and makes it a few orders of magnitude faster than Puppet. Puppet/ Chef.

Anisble Ansible consists of two offerings: Ansible and Ansible Tower, the latter featuring the platform’s UI and dashboard. Despite being a relatively new player in the arena when compared to competitors like Chef or Puppet, it’s gained quite a favorable reputation amongst DevOps professionals for its straightforward operations and simple management capabilities.

Docker To think about CM in a different way, consider Docker. Docker deploys software applications with all the necessary parts in a container, thereby ensuring it will run on any server, regardless of configuration and/or settings. Containers can be created, configured, and saved as templates for use on other hosts running the Docker engine. These templates can then be used to create more containers with the same OS, configuration, and binaries.

Powershell DSC For shops, DSC is a management platform in Windows PowerShell that enables deploying and managing configuration data for software services and managing the environment in which these services run. DSC provides a set of Windows PowerShell language extensions, new Windows PowerShell cmdlets, and resources that you can use to declaratively specify how you want your software environment to be configured. It also provides a means to maintain and manage existing configurations.

5 Versioning and Source Control Confirguation Management (CM) Tracking code level changes is a common CM tools allow one to define the desired state and necessary activity of today’s software of a system and/or environment in regards to developers. Doing so enables concurrent configuration files, software installed, users, development, merging, and rollback capabilities groups and many other resource types. They for applications/software. Source Control also provide functionality to automatically push Management (SCM) tools are popular options for changes onto specific machines, also known as keeping track of software code; many DevOps automation and orchestration. This is where practitioners also track versions of their systems Chef and Puppet live in the toolchain. Let’s take configuration with these tools, essentially a closer look now at CM tools to understand how managing their infrastructure “as code.” For Chef and Puppet compare in this role. example, it’s a common practice for systems administrators to store and manage their Puppet Manifests or Chef Cookbooks in GitHub. Chef and Puppet Chef Continuing Integration Merely labeling a tool as a DevOps solution does and Orchestration not make it so. It must address contemporary IT Continuous integration (CI) and orchestration challenges in building/managing high-velocity tools enable the integration of development code organizations while facilitating constant into the overall software product frequently improvement and collaboration between and early in order to mitigate potential conflicts groups. Tools—as critical agents of change—are down the line. Typically, these tools are employed instrumental in both managing technology as well to automate software builds and testing, and are as shaping culture: crucial for applying quality control on a continual basis (as opposed to after the software has been “The tools we use reinforce the behavior; the developed and released). These tools can also behavior reinforces the tool. Thus, if you want to be used to track and manage changes for CM-- change your behavior, change your tools.” for example, CI tools can be used to test Chef – Adam Jacob, CTO, Chef Cookbooks and Puppet Manifests for bugs and errors, and be configured to do so automatically every time infrastructure changes are committed Chef Features and Highlights and merged. At the basic level, Chef is a tool for automation, provisioning and configuration management. The Testing and Validation platform is made up of the following components: Tools and frameworks for testing and validation are important for ensuring quality at all phases Chef Server - the main hub where Chef of development. In many cases, unique solutions propagates and stores system configuration are applied to a specific aspect of testing-- for information and policies (i.e., recipes and example, one tool may be used for unit testing cookbooks). The Chef management console is the while another is used for integration testing. web user interface for Chef Server.

6 Chef Client - installed on every node being Vault, a project started by Nordstrom to managed, the Chef Client performs configuration improve upon the platform’s inherent security tasks on the local machine. mechanisms. Chef can natively store sensitive data (e.g., SSL certificate keys, database Workstation - allows designated workstations to passwords) in encrypted “data bags”— author/test/maintain cookbooks and upload them repositories of key/value pairs—for secure and to Chef Server. Workstations are also used when easy access. Management of these data bags, utilizing the Chef development kit package. however, is a tedious and error-prone process. Chef Vault provides an additional layer of Chef Analytics - a platform that provides security that enables easier management of these actions and run history, real-time reporting, and encrypted data bags. notifications around Chef automation activities.

Chef Supermarket - an open source directory of community-contributed cookbooks.

Chef Delivery - Chef Delivery is a set of tools that add yet more developer-friendly features like comprehensive codebase change histories, metrics, and permissions management to the platform.

Chef Delivery’s automated testing and continuous integration/delivery tools augment the platform with new features such as a shared workflow pipeline, collaboration capabilities, and enhanced analytics—as well as new ecosystem integrations with AWS, Azure, and Docker, to name a few. Though these enhancements are no doubt a boon to Chef’s developer community, Chef’s aspirations arguably have little to do with becoming a developer-centric automation tool and more with building a comprehensive platform for DevOps pipeline management.

Improve Security with Chef Vault Customer and/or community customizations quite often become so widespread and integral that they find their way into bonafide product releases. This is certainly the case with Chef

7 Puppet and synchronization with the Puppetmaster Puppet is considered a more operations and sysadmin-oriented solution when compared to Puppet Enterprise Console - a web GUI for Chef, though again—these role-based distinctions analyzing reports and controlling infrastructure are becoming less relevant with each release. resources DevOps practitioners—both developers and operations staff alike—strive to achieve optimal PuppetDB - data storage service for Puppet the conditions for continuous integration/delivery. data produced by Puppet Tooling is therefore increasingly evaluated based on its ability to achieve these ends effectively Other key components worth mentioning over and efficiently in the context of an enterprise’s others include MCollective, a framework for unique needs. Notwithstanding, Puppet has supporting server orchestration or parallel enjoyed significant first-mover advantages over job execution, and Hiera—a hierarchical key/ the years, and though both Chef and Puppet have value lookup utility for providing node-specific been neck-to-neck market leaders since the early configuration data to Puppet (for keeping days of IT automation, the latter boasts a longer site-specific data out of manifests). Puppet has commercial track record and larger install base. integrated MCollective, Hiera, and a myriad of other open source projects into its platform Currently on version 4.9, Puppet is commonly to provide comprehensive automation and deployed in a client/server configuration with management of mission-critical enterprise managed nodes periodically synchronizing their infrastructures. Many community-contributed configurations with the server. Reporting (e.g., add-ons are also available on Puppet Forge— results from automation runs, errors/exceptions) an expansive library of open source modules and other information is sent by the clients for extending the platform’s features and back to the server for aggregate analysis and capabilities. processing. Puppet Node Manager Puppet Features and Highlights Puppet’s Node Manager enables the creation of Puppet automation works by enforcing the rules around node attributes, which allows for desired state of an environment as defined in easier more efficient node management. With Puppet Manifests—files containing pre-defined Node Manager, nodes can be managed based on information (i.e., resources) describing the state their job rather than name, eliminating the need of a system. The core components that comprise to manually classify each node. New updates Puppet are as follows: include powerful provisioning capabilities for Docker containers, AWS infrastructure and bare- Puppet Server - the central server that manages metal machines. Puppet nodes (agents) Puppet Code Manager Puppet Agent - client software installed on Puppet has been a mainstay of the DevOps managed nodes that enables communication movement since its inception and continues to address the enterprise’s continuous

8 integration/delivery requirements. The as code.” As such, the company has made concept of “infrastructure-as-code” entails significant strategic initiatives and partnerships using software development best practices in support of SDN. For example, Puppet to manage infrastructure configurations and Labs recently announced a partnership with provisioning details—including code review, Arista Networks—a leading developer of SDN version control, and collaborative development, switches—to provide automation support to the among others. And like Chef, Puppet’s platform vendor’s SDN equipment line. This and other has evolved in response to the growing needs similar partnerships (e.g, Cumulus Networks, for a comprehensive mechanism to manage the Dell, Cisco) will position Puppet favorably over pipeline. competing vendors once SDN technologies gain widespread adoption. Introduced in Puppet Enterprise 3.8, Puppet Code Manager provides a consistent, automated Chef or Puppet? way to change, review, test and promote Puppet Chef and Puppet continue to expand their code in a continuous delivery framework. automation platforms in response to the needs Based around R10K—a general purpose toolset of the DevOps-enabled enterprise. Both vendors for deploying Puppet environments and are forging partnerships that may ultimately modules by interfacing with a version control define—as Chef would put it—what school of system—Puppet Code Manager accelerates the DevOps a particular organization belongs to. deployment of infrastructure by rendering it Recently, the two have partnered with Microsoft a testable and programmatic process. And by to integrate their platforms with Azure, and enabling easy integration with Git for version Puppet—no stranger to being a first mover—has control, this latest addition to the Puppet made key alliances with leading SDN vendors to platform further blurs the line between software position it favorably once the technology takes and infrastructure. hold. So if your organization plans on adopting SDN, Puppet might be a stronger candidate in Software Defined Network (SDN) this respect. SDN is a new paradigm for networking that decouples network control and forwarding Security is an enterprise-wide concern these from physical infrastructure, enabling agile days and should be taken into account when management of network resources in rapidly evaluating technologies. Chef has made changing environments. Just as significant strides in improving its platform’s enables IT to quickly spin up compute and security with Chef Vault. It’s interesting that storage instances on-demand, SDN replaces rigid Puppet Labs rebroadcasts CVEs for vendored (and sometimes manual) network operations with software such as ruby while Chef does not, dynamically provisioned network services and despite both products including Ruby as a core resources. component. It’s also difficult to believe Chef hardly ever finds vulnerabilities in any of their This new model for networking is right in line software, whereas Puppet published CVEs on a with Puppet’s advocacy of “infrastructure regular basis.

9 In short, both IT automation platforms have quicker and with less errors; this is accomplished matured greatly as enterprise solutions. We’ve by breaking down silos between development and highlighted some of Chef and Puppet’s key operations and creating a smoother path towards attributes and benefits—selecting the right software delivery. DevOps and its underlying option comes down to identifying each platform’s concepts provide undisputed benefits to any core competencies and determining which of forward-thinking organization, and the DevOps these fall in line with your organization’s unique toolchain provides mechanisms to realize these needs and requirements. Regardless of which benefits. automation platform you choose, UpGuard can complement either solution to round out the DevOps toolchain with advanced vulnerability assessment and monitoring, ensuring that security—as a function of quality—is baked in at every step of the continuous delivery process. Conclusion A typical DevOps toolchain might consist of the following: UpGuard to discover and track what you have and to determine what your environment should look like. The platform then outputs to a tool like Chef or Puppet for provisioning and automation-- or directly to Docker for creating containers or for creating development and test environments. Once systems changes and applications have been deployed to production, UpGuard can validate that the changes have indeed been rolled out successfully, and provide further validation that any deployed applications and systems are free of vulnerabilities through comprehensive vulnerability scanning.

In the context of DevOps, the whole is truly greater than the sum of its parts. One must be equipped with the proper range of tools to address the unique, ongoing challenges of continuous integration and software delivery, and no one tool can do the job alone. DevOps is about delivering higher quality applications

10 Businesses depend on trust, but breaches and outages erode that trust. UpGuard is the world’s first cyber resilience platform, designed to proactively assess and manage the business risks posed by technology.

UpGuard gathers complete information across every digital surface, stores it in a single, searchable repository, and provides continuous validation and insightful visualizations so companies can make informed decisions.

© 2017 UpGuard, Inc. All rights reserved. UpGuard and the 909 San Rafael Ave. UpGuard logo are registered trademarks of UpGuard, Inc. All other Mountain View, CA 94043 products or services mentioned herein are trademarks of their +1 888 882 3223 respective companies. Information subject to change without notice. www.UpGuard.com 11

UNKNOWN