FACILITATING THE SPREAD OF KNOWLEDGE AND INNOVATION IN PROFESSIONAL SOFTWARE DEVELOPMENT Advanced DevOps Toolchain eMag Issue 28 - May 2015 ARTICLE VIRTUAL PANEL ARTICLE Docker: Present Immutable Service and Future Infrastructure Discovery Advanced DevOps Toolchain // eMag Issue 28 - May 2015 1 Virtual Panel on Immutable Infrastructure Docker: Present and Future InfoQ reached out to experienced ops engineers to ask about Chris Swan presents an overview of the Docker journey so their definition and borders of immutable infrastructure as far and where it is headed along with its growing ecosystem well as its benefits and drawbacks, in particular when com- of tools for orchestration, composition and scaling. This ar- pared to “desired state” configuration management solutions. ticle provides both a business and a technical point of view Is it a step forward or backwards in effective infrastructure on Docker and separates the hype from the reality. management. How I Built a Self-Service Automation Platform The current buzz in the industry is how DevOps is chang- ing the world and how the traditional role of operations is changing. Part of these discussions lead to building self-service automation tools which enable developers to actually deploy and support their own code. Brian Carpio describes his journey towards an infrastructure that development teams could deploy with the click of a mouse or an API call. Managing Build Jobs for Continuous Delivery The number of jobs in a continuous inte- gration tool can range from a few to sev- eral thousand, all performing various functions. There is an approach to manage these jobs in a more efficient manner. Service Discovery Building microservices using immutable infrastructure means that your services need to be reconfigured with the location of the other services they need to connect to. This process is called Service Dis- covery. This article compares the approaches by several key con- tenders, from Zookeeper to Consul, Etcd, Eureka and “rolling your own”. FOLLOW US CONTACT US GENERAL FEEDBACK [email protected] ADVERTISING [email protected] EDITORIAL [email protected] facebook.com @InfoQ google.com linkedin.com /InfoQ /+InfoQ company/infoq is InfoQ’s DevOps Lead Editor and an enthusiast MANUEL of Continuous Delivery and Agile practices. PAIS Manuel Pais tweets @manupaisable A LETTER FROM and weaknesses of containers when compared to THE EDITOR the more established virtual machines. He also high- lights the expanding ecosystem around Docker, in- cluding tools and libraries for networking, composi- tion and orchestration of containers. Many companies haven taken the route of au- tomating delivery of traditional ops services such as provisioning infrastructure, deploying to production environments, adding metrics to monitor or creat- The edition follows in the footsteps of the previous ing test environments as similar as possible to pro- “DevOps Toolchain for Beginners” eMag. Once an duction. This decision often comes from reaching a organization has acquired the essential skill set to bottleneck in operations workload as development provision and manage infrastructure, set up delivery teams are empowered to deploy and own their ap- pipelines and analyze logs and metrics, it will then plications through their entire lifecycle in a DevOps gradually face more complex decisions. Decisions mindset. Brian Carpio describes his technical jour- that depend much more on the type of systems ar- ney into creating such a self-service platform. chitecture and organizational structure than on indi- When teams reap the benefits of continuous vidual tool choices. delivery, they want more of it. On every project. Ev- In this eMag we provide both implementation ery component. The number of jobs in the contin- examples and comparisons of different possible ap- uous integration/delivery server explodes and a proaches on a range of topics from immutable infra- maintenance nightmare ensues if care is not taken. structure to self-service ops platforms and service Automated job creation and update becomes cru- discovery. In addition, we talk about the Docker eco- cial to keep pace and avoid development teams get- system and the different aspects to consider when ting bogged down by build and deployment issues. moving to this increasingly popular system for ship- Martin Peston provides an overview and practical ping and running applications. examples on this topic. In the immutable infrastructure virtual panel we When adopting immutable infrastructure the learn from both advocates and skeptics on the pros need to (re)connect services dynamically becomes and cons of this “disposable” approach to infrastruc- a priority. This process is called Service Discovery. ture changes. Instead of trying to converge machines Microservice architectures further exacerbate the to a desired state, outdated machine are simply de- importance of reliably discovering and connecting stroyed and replaced by a brand new machine with an increasing number of standalone services. The the new changes applied. Simplicity Itself team analyzed the main solutions Chris Swan gives us a rundown of the increas- available, as well as the pros and cons of (trying to) ingly popular Docker tool, outlining the strengths do this on your own. 4 Advanced DevOps Toolchain // eMag Issue 28 - May 2015 READ ONLINE ON InfoQ Virtual Panel on Immutable Infrastructure By Manuel Pais THE PANELISTS Chad Fowler is an internationally known software developer, trainer, manager, speaker, and musician. Over the past decade, he has worked with some of the world’s largest companies and most admired software developers. Chad is CTO of 6 Wunderkinder. He is the author or co-author of a number of popular software books, including Rails Recipes and The Passionate Programmer: Creating a Remarkable Career in Software Development. Mark Burgess is the CTO and founder of CFEngine, formerly professor of network and system administration at Oslo University College, and the principal author of the CFEngine software. He’s the author of numerous books and papers on topics from physics to network and system administration to fiction. Mitchell Hashimoto is best known as the creator of Vagrant and founder of HashiCorp. He is also an O’Reilly author and professional speaker. He is one of the top GitHub users by followers, activity, and contributions. “Automation obsessed”, Mitchell strives to build elegant, powerful DevOps tools at HashiCorp that automate anything and everything. Mitchell is probably the only person in the world with deep knowledge of most virtualization hypervisors. Advanced DevOps Toolchain // eMag Issue 28 - May 2015 5 Servers accumulate change for better or worse over time, including: new applications, upgrades, configuration changes, scheduled tasks, and in the moment fixes for issues. One thing is certain, the longer a server has been provisioned and running the more likely it is in an unknown state. Immutable Servers solve the problem of being certain about server state by creating them anew for each change described previously. Immutable servers make infrastructure scalable and reliable in spite of change. However, this can require fundamentally new views of systems, patterns, deployments, application code and team structure, says panel member Chad Fowler. Mitchell Hashimoto, another in functional programming have. one: it would be frozen and noth- panelist, sees immutable infra- Infrastructure components, like ing would happen). What the structure as an enabler for re- in-memory data structures, are term tries to capture is the idea peatable environments (through running components that can that one should try to predeter- configuration management be accessed concurrently. So the mine as many of the details as tools) and extremely fast deploy- same problems of shared state possible of a server configuration ments times (through pre-built exist. at the disk-image stage, so that static images) and service or- In my definition of im- «no configuration is needed». chestration times (through de- mutable infrastructure, servers Then the idea supposes that this centralized orchestration tools). (or whatever) are deployed once will make it fast and reliable to This idea has been deemed and not changed. If they are spin up machines. When some- utopian by Mark Burgess, our changed for some reason, they thing goes wrong, you just dis- third panelist, due to the large are marked for garbage collec- pose of the machine and rebuild number of external dependen- tion. Software is never upgraded a new one from scratch. That is cies in any system today. Burgess on an existing server. Instead, my understanding of what peo- criticizes the “abuse of the notion the server is replaced with a new, ple are trying to say with this of immutability”. functionally equivalent server. phrase. InfoQ reached out to them Mitchell: Immutable infrastruc- I believe there are a number to ask about their definition and ture is treating the various com- of things wrong with this argu- borders of immutable infrastruc- ponents of your infrastructure ment though. The idea of what it ture as well as its benefits and as immutable, or unchangeable. means to fix the configuration is drawbacks, in particular when Rather than changing any com- left very unclear and this makes compared to “desired state” con- ponent, you create a new one the proposal unnecessarily con- figuration management solu- with the changes and remove tentious. First of all, predeter- tions. Is it a step forward or back- the old one.