DZONE RESEARCH PRESENTS

2014 GUIDE TO

BROUGHT TO YOU IN PARTNERSHIP WITH dzone.com/research/continuousdelivery © DZONE, INC. 2014

WELCOME TABLE OF CONTENTS

Dear Reader, Summary & Highlights 3

In recent years, we have grown accustomed to a KEY RESEARCH FINDINGS 4 constant stream of information that has fueled our news feeds, flooded our inboxes, and triggered Introducing: Continuous Delivery constant notifications... and in all of the noise, by Steve Smith 6 something has been lost. Clarity, concision, and relevance have become innocent bystanders in an Continuous Delivery Pitfalls age of information overload. While it is true the by J. Paul Reed 10 task of gathering information has become easier, finding the right information, when and where you The Continuous Delivery need it, has become increasingly complex and, at Toolchain times, overwhelming. by Matthew Skelton 14

The guide you hold in your hands (or on your Continuous Delivery Visualized 16 device) is a vital part of DZone’s fight against this growing trend. We recognize that as a technology Infrastructure as Code: When professional, having actionable knowledge at automation isn’t enough 22 your fingertips is a necessity. Your business by Mitch pronschinske challenges will not be put on hold while you try to identify the right questions to ask and sort Continuous Delivery Maturity Checklist 24 through endless information. In publishing our 2014 Guide to Continuous Delivery, we have worked Solution DIRECTORY 25 hard to curate and aggregate a wealth of useful topic knowledge and insight into a concise and informative publication.

While this is only our second publication of this CREDITS nature, we have come a long way so far and we have big plans for the future. I would like to thank you for DZone Research DZone CorporatE your support along the way and want you to know Kellet Atkinson Rick Ross that as we continue to publish more research, we Director of Research CEO are eager for your feedback. Our goal is to make the Jayashree Gopal Matt Schmidt most informative and practical publications that you Project Manager President & CTO can find for free on the web and your input will go a Mitch Pronschinske Brandon Nokes long way in helping us to achieve that goal. Senior Research Analyst VP, Operations

Matt Werner Alex Crafts On that note, I hope you find that this guide meets Market Researcher Senior Account Manager our goal to deliver to you the relevant and useful Hernâni Cerqueira insight you deserve. Thanks again for taking the time Lead Engineer

to check it out and, as always, thanks for being a part Special thanks to our topic experts Ashley Slate Graphic Designer of our great community. Steve Smith, J. Paul Reed, Matthew Skelton, Paul Duvall, Eric Minick, Chris Smith Marketing Associate Christopher Little, Benjamin Kellet Atkinson Wootton, Matt Jaynes, Willie Alec Noller Wheeler, Chris Shayan and Andrew Content Curator Director of Research Phillips as well as our trusted DZone Benjamin Ball [email protected] Most Valuable Bloggers for all their Content Curator

help and feedback in making this Sarah Ervin report a great success. Content Curator

PAGE 2 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

Summary AND KEY TAKEAWAYS

There’s nothing new about most of today’s research on IT’s business goals: get applications to market faster, under budget, with less people, and with pristine quality. The best companies will always be driving to achieve these goals. We don’t need more research reports to tell us that this is ultimately what business leaders want. What we do need are new ideas and industry data that tell businesses how other organizations are accomplishing these process improvement goals. The DZone 2014 Guide to Continuous Delivery services this need by providing data, ideas, and solutions that your organization can use to drastically improve its software production process. Explore the content in this guide and use the technology comparison features on its companion site dzone.com/research/continuousdelivery to understand:

• Continuous Delivery, DevOps practices, and their benefits • Tools and technologies that organizations use to implement Continuous Delivery • The visual structure of Continuous Delivery pipelines • Common pitfalls for organizations adopting Continuous Delivery • Software production metrics and trends from surveying 500+ IT professionals • Practical strategies regarding software industry best practices for deployment • Comparison metrics for 30+ solutions to aid Continuous Delivery implementation

KEY TAKEAWAYS

Continuous Delivery: It’s Not What You Think it is Close to half (41%) of the 500+ IT professionals from our survey believe that they have implemented Continuous Delivery for some or all of their projects. However, based on the questions they answered, only 26% overall are actually performing the basic Continuous Delivery practices. When we filtered respondents by stricter criteria based on three key traits in the definition of Continuous Delivery [1], the number dropped to 8% overall. You can read more about our methodology and breakdown in the Key Research Findings section of this guide.

Developers Shouldn’t be the Only Ones Doing it 60% of all respondents report that they have Continuous Delivery implemented in their application build management environment. Significantly less have extended these practices to their database management environments and production environments. Industry consultants often observe Continuous Delivery transformations that tend to stop at the development environments, but true Continuous Delivery can’t cover just one section of the software production process. It has to be present at every stage.

The Strategies of Development can be Extended to Operations Operations teams can significantly improve their release processes if they employ the tools and techniques of developers. By treating infrastructure configurations in the same way that developers treat source code—employing systems, automated testing, and deployment tools—sysadmins can further organize and unify the process. It’s heartening to see that nearly half (48%) of those surveyed use version control for system definitions and environment changes.

Many Organizations Haven’t Even Mastered Yet Continuous Delivery is an extension of Continuous Integration practices into the infrastructure management and the production environment. Many organizations incorrectly assume that they can implement Continuous Delivery even if their CI practices are flawed or incomplete. Supporters of the CD transition must understand that it is an incremental process that starts with CI. A good percentage (38%) of survey respondents are taking the right initial steps toward CD by extending their CI practices to production deployments.

Continuous Delivery is not a Fantasy Implementing Continuous Delivery can be more difficult for some organizations, but never impossible. Continuous Delivery isn’t just for startups and major web companies like Etsy or Netflix. Any organization can obtain the benefits of Continuous Delivery with consistent effort and the right resources. The content of this guide will show you how.

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 3 dzone.com/research/continuousdelivery

KEY RESEARCH FINDINGS

DZone surveyed over 500 IT professionals for the 2014 Guide to Continuous Delivery, revealing key insights into DevOps practices, Continuous Delivery adoption, and deployment performance. Respondents came from a fairly even distribribution of large and small companies; 44% came from organizations with over 500 employees and 56% were from organizations with under 500. A large portion of the respondents were developers (68%) or team managers (14%) with smaller portions of operations, QA, and executive management represented. The majority of respondents were headquartered in the US (36%) or Europe (43%).

The DevOps Way is Catching On

Continuous Delivery is a practice that fits squarely within the larger philosophy of DevOps, encouraging better technical collaboration and understanding between all areas of an organization, especially development and operations. Some companies have even opted to have a team that is solely focused on this collaboration with cross-compatible skills for multiple disciplines. 30% of the survey respondents have an officially designated DevOps team. Cross referencing this question with company size data shows that these teams are more prevalent in organizations over 100 employees. This is an encouraging sign for the proponents of DevOps and so is the 60% of respondents that said both development and operations were responsible for production support.

More People Are Treating Infrastructure as Code

Nearly half of those surveyed are using time-tested development techniques in operational settings. 49% are using using an infrastructure configuration management tool such as Chef or Puppet, and 48% are using a version control system for infrastructure changes and system definitions. However, many organizations aren’t automating those infrastructure changes. 73% of respondents still have to use manual scripts for at least half of their infrastructure changes.

Achieving Textbook Continuous Delivery is Hard

There are three key traits of Continuous Delivery defined by the authors of the methodology [1] that determine when an organization has fully implemented its practices. The table to the right shows how many survey respondents have these traits in their systems:

[1] http://martinfowler.com/bliki/ContinuousDelivery.html

For many, Continuous Delivery is still a work in progress. 27% of those surveyed believe they have achieved CD for some of their projects, and 14% believe they are doing it in all of their projects (41% combined). Respondents were considered to be using basic CD practices if they used version control, issue tracking, resource monitoring, automated at least 30% of their infrastructure changes, and if they frequently confirm their software updates as shippable. Only 26% of all respondents met these criteria. When respondents were filtered by the three key traits of Continuous Delivery from the section above, only 8% said “yes” or “always” for all of the questions.

PAGE 4 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

Deployment Times Leave Room For Improvement

Among the 8% of respondents who have all three Continuous Delivery traits, 63% have an official DevOps team and 73% needed to roll back fewer than 10% of their deployments. Compared to the process performance metrics of the entire survey pool, the respondents who exhibit the three key CD traits had faster average deployment times, less rollbacks, and shorter outages. Below are the process performance stats for all respondents:

Culture and Lack of Time: The Major CD Barriers

The two biggest barriers to Continuous Delivery adoption, both mentioned by 66% of respondents, are lack of time and problems with company culture, such as insufficient collaboration and DevOps practices. Other significant barriers include engineers and sysadmins not having the right skillset (46%), not enough budget (42%), and no support from management (45%). In the survey’s free response section, management’s support was the most common answer to the question, “what is the one thing that could help you implement Continuous Delivery faster?” The least significant barriers to CD adoption included lack of appropriate technology (25%) and regulation compliance (16%). Surprisingly, 37% of respondents claim that compliance requirements are hindering their ability to automate more of their release process.

Continuous Delivery Still Primarily in Development

Continuous Delivery isn’t a hard sell for developers. 60% say that they have already implemented it for their application build environments, and only 6% have no desire to implement in development. Database and infrastructure environments are still lagging behind though, with only 21% of respondents implementing it for the database environment and 18% implementing it for infrastructure. However, over half of all respondents hope to implement it for those environments. The survey also showed some headway toward extending CI practices to production deployments, with 38% saying they’ve done this.

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 5 dzone.com/research/continuousdelivery

Introducing: Continuous Delivery BY STEVE SMITH

“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” - The Agile Manifesto, First Principle

Since the publication of the seminal book Continuous Delivery by Dave Farley and Jez Humble in 2010, Continuous Delivery has become a widely discussed topic within the IT industry and an essential competitive advantage for technology companies such as Etsy, Facebook, and Netflix. But where did Continuous Delivery come from, what does it offer, and how does it work?

Beyond Continuous Integration Continuous DeliverY When Kent Beck published the inaugural eXtreme Inspired by the Agile Manifesto which states “our highest Programming paper in 1998, his proposal that developers priority is to satisfy the customer through early and should “integrate and test several times a day” was continuous delivery of valuable software,” Continuous revolutionary. Enshrined in XP as the Integrate Often Delivery is a method that advocates the creation of an rule, the frequent integration of mainline code allows automated deployment pipeline to release software rapidly developers to rapidly discover integration problems and and reliably into production. The goal of Continuous reduce development costs. Frequent integration has Delivery is to adopt a holistic end-to-end delivery proven so successful over time that it is now a mainstream perspective and optimize cycle time – the average time development practice known as Continuous Integration. between production releases – so that development However, since Continuous Integration is focused on costs are lowered, the risk of release failure is minimized, development, it can only benefit a fraction of the end-to-end and customer feedback loops are faster. The result is an release process, which remains a high-risk, labor-intensive automated release workflow similar to the illustration below: affair in a majority of IT organizations akin to the following:

Such a release process will likely involve extensive In order to guide Continuous Delivery adoption within an documentation, overnight scheduling, unversioned organization, Dave Farley and Jez Humble defined the configuration management, ad hoc server management, following principles: and large numbers of participants. In this situation, • Repeatable, Reliable Process: Use the same software releases inevitably become high cost, high risk deterministic release mechanism in all environments. events susceptible to human error, and given prominent failures such as the Knight’s Capital $440 million glitch • Automate Almost Everything: Automate acceptance [1], there can be an understandable reluctance to testing, deployment tasks, configuration management, release software frequently. However, there is always an etc. opportunity cost associated with not delivering software. • Keep Everything In Version Control: Store all code, This was recently highlighted by reports of Microsoft’s configuration, schemas, etc. in source control. decade of e-book/smartphone opportunity costs [2]. This poses a seemingly intractable business problem for many • Bring Pain Forward: Shrink feedback loops for time- organizations – how can the risk of delivering software be consuming, error-prone operations. reduced, while simultaneously delivering new features to • Build Quality In: Fix defects in development as soon as customers faster? they occur.

[1] http://money.cnn.com/2012/08/09/technology/knight-expensive- • Done Means Released: Do not consider features computer-bug complete until released to production.

[2] http://www.vanityfair.com/business/2012/08/microsoft-lost-mojo-steve- • Everybody Is Responsible: Align teams and individuals ballmer with the release process.

PAGE 6 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

• Continuous Improvement: Continuously improve the of organizations, a deployment pipeline will be used by people and technology involved. non-aligned siloed teams, meaning that lead times will be substantially inflated by handover delays between teams These principles are promoted by the deployment pipeline regardless of pipeline execution time. pattern, which has been described by Dave Farley and Jez Humble as “Continuous Integration taken to its logical Dave Farley and Jez Humble have repeatedly warned that conclusion” [3] and lies at the heart of Continuous Delivery. “where the delivery process is divided between different A deployment pipeline is an automated implementation of groups… the cost of coordination between these silos can the build/deploy/test/release cycle that enables self-serviced be enormous,” [3] and this is reflected in the Everybody releases of any application version into any Is Responsible and Done Means Released environment. A diagram of a typical deployment principles listed above. Testing and operational pipeline can be found in a free online chapter of The deployment tasks must become intrinsic development Continuous Delivery [4]. A real world pipeline, activities rather than discrete work phases, and however, will be specialized to an organization’s pipeline pattern the people involved must be integrated into requirements in the software delivery process. the product development team in what is often [3] http://www.amazon.com/dp/0321601912 is Continuous a slow and painstaking process of change. It is

[4] http://ptgmedia.pearsoncmg.com/images/ for this reason that the parallel growth of the chap5_9780321601919/elementLinks/fig5_4.jpg Integration taken DevOps philosophy has been welcomed by the Continuous Delivery community. In the basic deployment pipeline pattern, the to its logical commit stage is triggered by a source code or conclusion. DevOps has been defined by Damon Edwards configuration change. This stage includes code as “aligning development and operations compilation, unit tests, static analysis checks, and roles and processes in the context of shared assembly of the application binaries for the binary repository. business objectives,” [6] and aims to increase communication A successful run automatically triggers the acceptance and collaboration between IT divisions such as development stage, which runs the automated acceptance tests against and operations. Continuous Delivery and DevOps have that application version. If the acceptance tests pass, that evolved independently, but are interdependent upon one application version can progress to manual exploratory another – a deployment pipeline can act as a focal point testing, automated capacity testing, and production usage. for DevOps collaboration, and the DevOps integration of This closely follows the deployment pipeline best practices: Agile principles with operations practices can eliminate the handover delays between development and operations • Build Your Binaries Only Once: Create immutable binaries teams. to eliminate recompilation errors. • Deploy The Same Way: Use the same automated release Conclusion mechanism in each stage. While Continuous Integration has become a mainstream • Smoke Test Deployments: Assert deployment success development practice, Continuous Delivery goes much prior to usage. further and is poised to become a mainstream IT practice. • Deploy Into Production Copy: Create a production-like test Through creating an automated deployment pipeline, a environment for testing. repeatable and reliable delivery mechanism enables an organization to increase product revenues by releasing • Instant Propagation: Make an application version new features to customers more frequently without fear of automatically available to the next stage upon success in failure. However, Continuous Delivery is far more reliant upon the previous stage. organizational change than technology change in order to • Stop The Line: When an application version fails on a truly be successful. certain stage, automatically stop its progress through the pipeline. [5] http://www.melconway.com/Home/Conways_Law.html

[6] http://dev2ops.org/2010/02/what-is-devops/ The creation of a deployment pipeline establishes a pull- based release mechanism that reduces development costs, minimizes the risk of release failure, and allows the production release process to be rehearsed thousands of times. It provides visibility into the production readiness of different application versions at any point in time, and drives continuous improvement of the release process by identifying bottlenecks in the system. WRITTEN BY Steve Smith Steve Smith is an Agile consultant and Continuous Organizational Change and DevOps Delivery specialist at Always Agile Consulting While a deployment pipeline reduces costs and risk, the Ltd. Steve is a regular speaker at Skills Matter for reality is that Continuous Delivery is vulnerable to Conway’s the London Continuous Delivery group, and has spoken at conferences such as Agile Horizons and Law [5] and dependent upon organizational change to QCon New York about Continuous Delivery. achieve a significant reduction in cycle time. In the majority

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 7

2014 GUIDE TO CONTINUOUS DELIVERY

Extending Continuous Integration to Continuous Delivery with

Jenkins has been used widely among all the way from source code review to or whether you are doing it on premise, in developers for several years now, and production deployment. This process the cloud, or somewhere in-between. various polls suggest that it is by far the is known as Continuous Delivery (CD). most adopted continuous integration (CI) The days when Jenkins was used by I invite you to take a look at how you server on the market today. Deployment developers who only had to automate can push your automation further with automation is widely accepted and builds and hand binaries off to the CloudBees’ Jenkins-powered Continuous practiced to the point where Jenkins operation guys are over. Instead, an Delivery Platform. Whether you have has become mission-critical for many increasing number of teams are just started using Jenkins or have been companies, which is amazing. responsible for the entire lifecycle from using it for 5 years, there’s always an coding to operation, and this is how opportunity to take it one step further While a growing number of people Jenkins is used by sophisticated users and get even more out of it. Assisting embrace automation, the depth of today. code reviews, running tests in parallel, automation among existing users has tracking deployments, you name it—you gotten deeper too. We are now at the The CloudBees Continuous Delivery can orchestrate all sorts of complex point where it is completely practical to Platform is powered by Jenkins to application delivery processes. build software-managed automation help people get there. Thanks to the incredible plugin ecosystem of Jenkins, it can be deployed in Continuous Delivery is a never-ending practice just about any kind of WRITTEN BY environment, regardless Kohsuke Kawaguchi of pushing the envelope of automation little by of whether you use Chief Technology Officer little, empowering people to focus and attain Java or PHP, whether CloudBees higher productivity and visibility into the process. you are writing web apps or mobile apps,

CloudBees Continuous Delivery Platform COMPANY: CloudBees CI

PHONE: (323) 843-4483 LOCATION: Woburn, MA, USA LAUNCH DATE: January ‘11 @CloudBees

SUPPORTED TARGET SYSTEMS FULL PROFILE LINK dzone.com/r/aVAt  Windows  Solaris   AIX DESCRIPTION  Mac OS X  Other The CloudBees Continuous Delivery Platform provides a range of Jenkins- (See companion site) based Continuous Integration and Continuous Delivery solutions on premise, in the cloud or in a hybrid configuration. The CloudBees Platform offers STRENGTHS solutions for both development and DevOps teams to utilize Jenkins and • Jenkins-based • Hosted iOS and Platform as a Service technology in a configuration that best meets their Continuous Delivery Android builds for lifecycle mobile Continuous needs. The CloudBees CD Platform can also be extended to work with Delivery technologies across the application lifecycle. • On premise, public cloud, or hybrid • Manage cloud configuration multiple Jenkins CUSTOMER SUCCESS STORY and hosting options configurations Viridity Energy produces the Java-based VPower software, which allows Viridity’s across the enterprise customers to manage their electricity usage and transform energy profiles into financial returns. For VPower, Viridity Energy needed to streamline and accelerate NOTABLE CUSTOMERS the deployment process, establish an integrated build and runtime environment, and support continuous integration along with the Scala programming language. • ChooseDigital • Viridity Energy CloudBees’ PaaS helped VPower accelerate reactive application development by • Netflix • Alcatel-Lucent providing Jenkins-based continuous integration, reducing deployment windows by • Altares 85% and costs by 66%, and scaling with Viridity’s needs.

Hosting Options FREE TRIAL 30-day free trial • On premise, public cloud, or hybrid cloud

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 9 dzone.com/research/continuousdelivery

Continuous Delivery Pitfalls BY J. PAUL REED

There is little question that Continuous • Inconsistencies among CI agent configuration (despite Delivery provides a compelling business story. how much it’s emphasized as an issue by the DevOps movement, it remains a disturbingly common problem). Everyone is looking for ways to increase their own business agility with cloud-based builds, built-in unit and • CI master servers or agent hosts running on individual integration testing, and fully automated deployments, all employees’ computers or in unofficial/personal cloud instrumented with monitoring utilities. If there weren’t accounts. This is often observed in organizations that several high-profile companies discussing both their need to support Mac OS X. Continuous Delivery successes and failures so publicly, it • CI masters with no access control, with job might all seem like a fairy tale. The incredible complexity configurations that can be modified without notification of software production makes it extremely difficult or an audit trail. to create a clean path for software to travel from a developer’s keyboard all the way to the customer’s hands. • Unactionable CI, which can be caused by insufficient communication mechanisms from the CI infrastructure For every one of the “Continuous Delivery unicorns,” (email, which everyone promptly filters for instance), or like Etsy and Netflix, there are at least ten organizations a cultural barrier, where CI errors or build/test failures who struggle significantly to implement CD within their are not considered by everyone in the organization to be organization. Many of those will ultimately fail entirely to worthy of a timely response. deliver on CD’s promises. In the worst cases, organizations get excited about CD and Like any new development methodology, implementing try to skip over CI. Continuous Delivery has a number of pitfalls that can trip up even the most mature organizations. Successful The bad news: the solutions to these problems are varied. Continuous Delivery processes are predicated on a Some are simple, while others (especially those involving number of technical and cultural assumptions, and in many company culture) may be tougher to overcome. cases organizations don’t have the foundation necessary for CD. Organizations often try to copy the practices of a The good news: once the organization has achieved company like Netflix wholesale after reading some of their stable, operationalized, actionable continuous integration, resources, ignoring the differences in their own products you’ve already done 50% of the work to build a legitimate or market. These examples are just a few of the reasons Continuous Delivery environment. why companies find themselves in trouble when adopting Continuous Delivery. Below are four of the most common pitfalls to avoid when implementing Continuous Delivery. 2. Confusing Continuous Delivery with Continuous Deployment 1. Attempting to Build Continuous Delivery Since Continuous Delivery and continuous deployment on Top of an Unstable (or Non-Existent) have very similar themes, it’s easy to understand the origin Continuous Integration Foundation of this confusion. Continuous Delivery is the discipline and infrastructure around the ability to deploy changes Continuous integration is a foundational requirement whenever it makes sense for the business to do so. It does for Continuous Delivery. The constantly referenced not require that every single developer commit directly Continuous Delivery deployment pipeline is really out to production in real-time. just the practices of continuous integration extended to infrastructure management and the production Getting every single improvement out to customers the environment. Thus, any successful CD implementation moment that it is checked in is so tantalizing that business begins with a CI system that is stable, operationalized, and stakeholders often believe that this is the ultimate goal of able to provide actionable data. The organization should Continuous Delivery. The metric of achieving commit-to- also foster a company culture that knows how to react to release for all situations starts to be overly emphasized, that data. leading to the Hawthorne Effect [1], where people improve performance on the metrics they know they’re being Many environments, however, have no continuous measured by. Unfortunately, the effect is often temporary. integration currently deployed or a poor CI foundation for CD. Common issues with CI infrastructures include: To emphasize the difference between Continuous Delivery and continuous deployment, one just has to understand • configuration, libraries, and tool that Continuous Delivery is possible even in low-change inconsistencies between developer and build/test/ tolerance, high risk environments, like a nuclear power production environments. plant. In such situations, the focus isn’t on continuously

PAGE 10 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

deploying updates to plant software and infrastructure, A great example of this is Mozilla Corporation’s move from but rather on integrating and deploying it to staging 18+ month release cycles for its Firefox web browser to six infrastructure so that even infrequent deployments will have week release trains. The move required a massive investment a higher probability of success. in QA process and infrastructure, including a cultural change so that every component of the browser and every bug fixed To be clear, continuous deployment is not a fantasy: had a comprehensive suite of unit tests, ensuring that any organizations who have long focused on their Continuous mistakes that would “break the web” would be easy to spot Delivery pipelines and processes, like Etsy and Facebook, as the rate of change to the code-base increased. It required do deploy changes at an attractively high rate. But for an investment in build/release infrastructure to support organizations starting with Continuous Delivery, focusing running all of these tests on the supported platforms and on the path commits take through the delivery pipeline, as mobile devices at a opposed to the rate at which they’re deployed, will produce scale previously not better initial results and a higher chance of permanent In the worst cases, encountered. adoption. organizations get In short, it required [1] http://en.wikipedia.org/wiki/Hawthorne_effect that the business excited about CD and invest heavily in 3. Complicated or Inconsistent Source infrastructure which, try to skip over CI. to put it frankly, isn’t Code Workflows sexy and usually doesn’t provide Continuous Delivery father Jez Humble often conducts an direct business value. In many software shops, infrastructure informal survey during talks. He’ll ask the audience to raise has been chronically underinvested in already, so they their hand if they do continuous integration. Most hands go are already starting their CD initiative at a disadvantage. up. His second question: “Put them down if all the devs on Successful Continuous Delivery stories commonly emphasize your team don’t check into trunk/mainline/master at least the organization’s commitment to investing in infrastructure once a day.” Many of the hands invariably go down. throughout the CD transformation and beyond.

This survey indicates that feature branches and complex branching/merging processes are an enemy of continuous 5. Difficult, but Worth It integration, and therefore of CD. In fact, in its pure form, Moving to Continuous Delivery as the release mechanism Humble argues there is no need for any branch other than for your software is not a trivial undertaking. Similar to (what Git calls) the master branch. implementing DevOps, it requires committed investment in new tooling and a re-examination of the organizational This is especially hard for users of Git to accept because of its culture from all technical and business teams before any ability to easily create and publish infinite feature branches. benefits can be gained. This can be very difficult if the Overuse and reliance on an unmerged codeline can make cultural foundation for Continuous Delivery is initially code flow through a delivery pipeline difficult for anyone unstable. It’s even harder if you’ve fallen into one of the to track. Creating release branches and other automation- pitfalls mentioned in this article and still believe that related branches can quickly add complexities that make it you’re going down the right path. It takes work, humility, almost impossible to answer simple questions like, “show perseverance, and a commitment to balancing feature me all the commits that are in this release, but not in the development with infrastructure development. If your integration branch.” organization is willing to put in the extra effort to get things right, you will put your business on the path to reliably ship This doesn’t mean that Git is a bad tool for CD, but users software any time. should know that complicated branching/merging and different individual workflows will make Continuous Delivery

very difficult, regardless of the tools used. 4. Attempting to Switch to Continuous Delivery Without any Supporting Infrastructure WRITTEN BY The ability to deploy code to production at a moment’s notice requires visibility into the state of your application. J. Paul Reed J. Paul Reed has over a decade’s experience Your application may require configuration management, in the trenches as a build/release and tools monitoring and alarms, and rollback conditions. Setting engineer, working for such companies as VMware, up this infrastructure is not very exciting and many times Mozilla, and Symantec. In 2012, he founded new features will get priority and investment instead of Approaches, a consultancy infrastructure support. This is a mistake that will make the incorporating a host of tools and techniques implementation of CD more difficult. focused on helping organizations “Simply Ship. Every time.” He also hosts a DevOps podcast at theshipshow.com.

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 11

2014 GUIDE TO CONTINUOUS DELIVERY

Continuous Delivery One Improvement at a Time

Getting your code and great new features perhaps the fundamental idea—of your current situation and with your current into the hands of your users is the most Continuous Delivery is the notion of steady, processes. Improve them incrementally to important part of your development incremental improvement. Getting CD build the delivery pipelines that you need, lifecycle. We all want to get away from right in a real-world situation, especially and grow your DevOps teams and culture, painful month-long release cycles, stressful in the enterprise, means having a clear day by day. troubleshooting sessions trying to figure and pragmatic vision of what your goals out why our system is not working in the are—initial and long-term—and introducing We have the vision and knowledge of next environment, and endless early- tools and practices to build the Continuous what fully-automated delivery pipelines morning status calls. Delivery tool chain and culture that is right and a strong DevOps culture can do—it’s for you. how we develop our own software. But Continuous Delivery we also have the hands-on, hard-nosed makes life so much You write great code. Our delivery experience of what Continuous Delivery easier for developers, We get it to your users. automation is implementations look like in the real allowing you to focus designed with world—the kind of experience that allows on the important stuff: It’s that simple. one goal in mind: us to build tools that will help you succeed. writing cool code and to allow you to implementing great features. It’s also easy get started on your road to Continuous to sell to your managers, allowing you to Delivery today. Whether it’s deployment speed up time to market while maintaining automation, on-demand environment WRITTEN BY service quality, and to close the feedback provisioning, Agile test set management, or Andrew Phillips loop between developers and users. the full coordination and orchestration of VP Product Management your release process and delivery pipelines, XebiaLabs One of the fundamental principles— our tooling allows you to get started in

XebiaLabs XL Deploy COMPANY: XebiaLabs ARA

PHONE: (323) 843-4483 LOCATION: Boston, MA, USA LAUNCH DATE: May ‘11 @xebialabs

SUPPORTED TARGET SYSTEMS FULL PROFILE LINK dzone.com/r/M3aA  Windows  Solaris  Linux  AIX DESCRIPTION  Mac OS X  Other XL Deploy is an agentless application release automation solution. XL (See companion site) Deploy combines intelligent deployment automation, the richest content STRENGTHS set for enterprise middleware, and seamless integration into the application • Agentless • Enterprise security delivery ecosystem. XL Deploy’s unique, agentless architecture means less architecture and centralized maintenance, lower costs, and support for much more diverse deployment auditing scenarios that will become common with the Internet of Things. • Declarative automation that • Visualize eliminates scripting Continuous Delivery and workflow pipelines with CUSTOMER SUCCESS STORY design template reports The Rabobank Group is a multinational banking and financial services company that • Fully automated maintains hundreds of in-house and customer-facing applications. Before using XL rollback and recovery Deploy, it took them four weeks to manually modify the infrastructure configuration for each application across every environment. XL Deploy allowed Rabobank to NOTABLE CUSTOMERS automate the entire deployment process and develop infrastructure configurations • 3M • Expedia along with the application code. By integrating Continuous Delivery into the deployment process with XL Deploy, Rabobank has accelerated their time-to-market • GE • Digital Globe while eliminating errors and reducing costs. • John Deere • Tribune Supported CI Engines FREE TRIAL • Jenkins/Hudson • Atlassian Bamboo • Microsoft TFS Fully functional, time-limited evaluation version. Trial license extensions available upon request.

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 13 dzone.com/research/continuousdelivery

The Continuous Delivery Toolchain BY MATTHEW SKELTON

Unless your software is very simple, no single Continuous Integration tool, automation product, or deployment CI tools can support orchestration and visualization, but their core functionality is to integrate new code with the pipeline implementation will provide you with stable main release line and alert stakeholders if any new Continuous Delivery. Effective Continuous code would cause issues with the final product. This makes Delivery requires an organizational it easy for teams to combine work on different features while keeping a master code branch ready for release. It understanding of the intent and purposes should feel natural to integrate many times a day. Teams of the activities you undertake, not merely should also connect a code metrics and profiling utility the automation of those activities. However, that can stop integrations if certain metrics reach an undesirable threshold. Continuous Delivery is impossible without some core capabilities that tools provide. CI: Jenkins, Travis CI, ThoughtWorks GO, CircleCI, Each section below will examine a link in JetBrains TeamCity, Atlassian Bamboo Code Metrics: SonarQube, SLOC (and variants), SciTools the core Continuous Delivery toolchain with Understand examples from each tool category in the boxed sections. Artifact Management Packaged artifacts, rather than the application’s raw source code, are the focus of deployment pipelines. Orchestration and Deployment Pipeline Artifacts are assembled pieces of an application that Visualization include packaged application code, application assets, Orchestration tools are the backbone of any CD system. infrastructure code, virtual machine images, and (typically) They allow teams to build an effective sequence of configuration data. Artifacts are identifiable (unique deployment pipeline steps by integrating with their entire name), versioned (preferably semantic versioning), and toolchain. These tools can also provide visualization immutable (we never edit them). Together, these artifacts utilities, which are important for enabling the full allow developers to build a bill of materials (BOM) involvement of stakeholders from all departments. For package describing the exact versions of all the artifacts pipeline orchestration and visualization, you can use a in a particular version of their software system. Package dedicated deployment pipeline tool or you can use an metadata identifies when and how the package was application release automation (ARA) solution. Whichever tested or deployed into a particular environment. direction you take for your orchestration tool, you should be sure that it helps your team detect and expose delays Artifact management at each stage of the pipeline, including wait times between Ensure that versioned, is most effective stages. Visualization and orchestration working in tandem traceable artifacts are the with an artifact allow teams to quickly identify the places they should key unit of currency for the repository manager. optimize first. Artifact repositories deployment pipeline. contain a complete Dedicated Deployment Pipeline Tools: Jenkins, Travis CI, artifact usage history, ThoughtWorks GO, CircleCI, JetBrains TeamCity, Atlassian Bamboo similar to the way version control systems track source ARA: ElectricCommander, CA LISA, IBM UrbanCode, code changes. They use dependency resolution between XebiaLabs XL the package versions and allow the system to build a Orchestration Engine: MaestroDev, CollabNet dependency graph from the hardware all the way up to the user interface of the application. The ability to verify Version Control dependencies through an entire system is powerful for Most software development teams use a version control tracking exactly what was (or will be) tested or deployed. system for the files they consider source code. However, many organizations forget to include configuration files, Version Control Systems: Git, Subversion, Mercurial such as the configuration that defines the build and Artifact Repository Managers: Archiva, Artifactory, Nexus release system. All text-based assets should be stored in (roll-your-own with zip files, metadata, shared storage, and a version control system that everyone can easily access. access controls) The code changes should be very easy to review, line-by- Language-specific Package Managers: Composer (PHP), line (ideally in a web browser), with a pull or merge request. Ruby Gems, npm (Node.js), Python PIP OS-level Package Managers: APT, Chocolatey, RPM Version Control: Git, Mercurial, Perforce, Subversion, TFS

PAGE 14 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

Test and Environment Automation A Final Look at Some Guiding Principles for The only manual testing in a deployment pipeline should Tools be for tests that are tough for a computer to handle, such It’s important to understand all the capabilities your team as exploratory testing, inspection of user interface designs, needs before selecting the tools to build your Continuous and user acceptance tests. The rest should be automated. Delivery system. The following list details the key areas you Tools for automated testing should operate in a completely should always keep in mind when selecting tools: headless (non-interactive) manner and be lightweight • Visibility: Look for tools that have clear, comprehensive enough to run across many test servers simultaneously. visualizations for everything that your organization needs Teams also need to create testing environments on-demand to track. by using environment automation tools that can provision a VM and configure an environment template. • Traceability: Select tools that will allow you to easily track important metadata from your source code, infrastructure Test Automation: JMeter, Selenium/WebDriver, Cucumber code, binary artifacts, application and infrastructure (BDD), RSpec (BDD), SpecFlow (BDD), LoadUI (Performance), configuration, VM images, and the deployment pipeline PageSpeed (Performance), Netem(Network Emulation), configuration. SoapUI (Web Services), Test Kitchen (Infrastructure) • Full Coverage: Tools must cover all of the applicable Environment Automation: Vagrant, Docker, Packer environments in order for delivery to be consistent. If a tool is too expensive to have in every environment, you shouldn’t choose it. Server Configuration and Deployment Current deployment tools support three models: In addition to these tool selection suggestions, you should • Push model: Manages the distribution and installation of consider using separate tools for CI and visualization/ packages to multiple remote machines. It’s a good choice orchestration, because these capabilities have different for smaller systems because it’s usually simple and quick. requirements. You should also ensure that versioned, • Pull model: Requires an infrastructure configuration traceable artifacts are the key unit of currency for the tool such as Chef or Puppet. Supporters say it scales deployment pipeline. Avoid simplistic linear sequential better than push and like that it treats the deployment pipeline stages where parallel flows would better meet of application code as another step in configuring the business needs. Finally, insist on a system that tracks, infrastructure. measures, and visualizes the flow of artifacts toward production so that all stakeholders can effectively engage • Hybrid model: Uses a push tool to trigger a pull client on with your software production process. target servers. For any of the three models, your team must ensure that the process is fully automated, provides detailed information with standard output and error messages, and allows easy and Insist on a system that tracks, rapid rollback to a stable state. measures, and visualizes the flow Push deployment: Capistrano, Fabric, ThoughtWorks Go, of artifacts toward production so MSdeploy, Octopus, RunDeck, various CI and build tools, various ARA tools that all stakeholders can effectively Pull deployment: Ansible, Chef, CFEngine, Puppet, Salt. engage with your software production process. Monitoring and Reporting Monitoring your system logs is essential for spotting problems and halting the deployment pipeline. Rather than manually collecting logs from each machine in an WRITTEN BY environment, logs should be shipped to a central store that Matthew Skelton indexes them and makes them available for searching via Matthew Skelton is an independent software a web browser. This is a crucial capability for a Continuous consultant who helps organizations to design and Delivery environment. The log store should be connected to operate effective software delivery practices, with all environments (including the developer’s system) to speed a focus on the re-design and evolution of software up problem diagnosis and resolution. Most monitoring tools in order to support Continuous Delivery. He initiated and helped to run PIPELINE 2014, the first should work in a dynamic infrastructure and integrate with conference in Europe dedicated to Continuous yours through scripted configuration. Delivery, having co-founded and organized the London Continuous Delivery meetup group since 2012. He is writing two books: one on package Log Aggregation & Search: Fluentd, Graylog2, LogStash, management in a Windows environment, and one nxlog, Splunk on software operability. Metrics, Monitoring, Audit: Collectd, Ganglia, Graphite, Icinga, Sensu, ScriptRock @matthewpskelton | matthewskelton.net

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 15 Dazzling new products go to market faster with CA at the center. From planning to delivery to management and security, CA software simplifies IT so business can focus on wowing the market. It’s a beautiful thing. ca.com/atthecenter ©2014 CA, Inc. All rights reserved. 2014 GUIDE TO CONTINUOUS DELIVERY

The Need for Speed and Quality in Release Cycles

In today’s fast-moving, ultra-competitive slow release cycle times and ultimately and collaboration to improve innovation business landscape, the days of delay time to revenue. and time to revenue. enterprises spending months—or even years—building, testing, and releasing An enterprise-class Continuous CA Technologies has a solution designed an application, service, or new set of Delivery solution, fully automating the to help you along your DevOps journey. capabilities to the market are over. deployment and promotion of changes CA Release Automation enables Today’s customers, technology users, and from development through production, customers to automate the delivery the market as a whole are more nimble significantly helps enterprises overcome process and reduce the complexity of than ever before, and the demand for these challenges. You can think of promoting new applications through the innovative new functionality has grown in release automation as the central nervous development lifecycle and into production. concert. system of DevOps, constantly integrating It also provides unique capabilities information and coordinating activities specifically engineered to enable reuse, In response to this change in the for all parts of the deployment process. segregation of duties, collaboration, and marketplace, many enterprises have begun Organizations automating the release- deployment processes. Ultimately these focusing on ways they can deployment features will help you to achieve your accelerate their application process have the DevOps goals. releases without sacrificing You can think of opportunity to quality and cost. The develop a true problem is, IT development DevOps culture, release automation WRITTEN BY and operations teams are tearing down often operating in silos as the central barriers between Ruston Vickers and relying on inefficient development SVP, Product Management & and error-prone manual nervous system of and operations, Strategy Application Delivery deployment methods. The and speeding CA Technologies traditional manual methods DevOps. communication

CA Release Automation COMPANY: CA Technologies ARA

PHONE: (800) 225-5224 LOCATION: Islandia, NY, USA LAUNCH DATE: January ‘07 @cainc

SUPPORTED TARGET SYSTEMS FULL PROFILE LINK  Windows  Solaris dzone.com/r/wkdP   Linux AIX DESCRIPTION  Mac OS X  Other CA Release Automation is an enterprise-class, continuous delivery (See companion site) solution that automates complex, multi-tier release deployments through STRENGTHS orchestration and promotion of applications from development through • Action packs infrastructures production. CA Release Automation helps companies speed up application and plug-ins for • Graphical workflow release cycles, achieve higher quality and reduce the cost of application automating and for creating deployments, while promoting collaboration and alignment between integrating third- deployment Development and Operations. party solutions processes without scripting • Defines and manages CUSTOMER SUCCESS STORY • Designed to handle release activities complex, multi- A consumer electronics company that manufactures home entertainment devices across application tier distributed Dazzling new products go to components requires customers to connect to a website through their devices in order to solutions on various configure them. However, during the busy holiday season, many customers reported that they were unable to use their new devices due to performance issues. After NOTABLE CUSTOMERS the incident, they learned that their problems were related to their deployment market faster with CA at the center. • Atkia Bank • Logicalis SMC B.V. processes, and introduced CA Release Automation in order to standardize deployments across releases. As a result, the website and software are now • BUPA • British United consistently reliable, earning customer satisfaction and retention. From planning to delivery to management and security, CA software simplifies • Tesco Provident Association LTD. IT so business can focus on wowing the market. It’s a beautiful thing. • Liquidnet Supported CI Engines

FREE TRIAL • Jenkins/Hudson • JetBrains TeamCity • Microsoft TFS No free trial

ca.com/atthecenter dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 17 ©2014 CA, Inc. All rights reserved. Continuous Delivery advocates the creation of • Any long-running step, such as UAT, Pre-Production • If significant issues are found in any long-running step, deployment pipelines, which are illustrated below. testing, or Exploratory Testing, can happen even after the and the change has already been deployed to CONTINUOUS DELIVERY: The first is a long, linear diagram with all of the possible change has already been deployed to Production. Production, the team should rollback Production to the steps, and below that are two common deployment last working release. pipelines with parallel-running steps. The parallel (wide) • If significant issues are found in any long-running step, Diagrams are based on Jez Humble’s diagrams from the Continuous Delivery blog pipelines allow teams to deploy to production sooner or and the change has not been deployed to Production, the (http://continuousdelivery.com/2010/09/deployment-pipeline-anti-patterns) team should manually halt the pipeline. visualized to wait for test results before deploying. Special thanks to Matthew Skelton for helping build these diagrams.

VISUAL (UI) TESTS NETWORK TESTS SYNTAX CHECK POST-DEPLOYMENT CAPACITY TESTS TESTS CODE METRICS

USABILITY TESTS EXPLORATORY PERFORMANCE STAGING & PRE- COMPILE EXPLORATORY TESTS STAGING & PRE- TESTINGTESTING PRODUCTIONPRODUCTION START COMMITCOMMIT FOR COMPILED LANGUAGES

STUBBED AND MOCKED ENDPOINTS AND DATA

SMOKE TESTS REFERENCE KEY INTEGRATION BDD FRAMEWORK UNIT TESTS TESTS TESTS

AUTOMATED EXECUTION AUTOMATEDAUTOMATED USERUSER AUTO-MERGE FOR ACCEPTANCEACCEPTANCE ACCEPTANCEACCEPTANCE DEVELOPER BRANCHES MANUAL EXECUTION TESTINGTESTING (AAT)(AAT) TESTINGTESTING (UAT)(UAT) WHEN USING GIT FLOW, ETC. POST-DEPLOYMENT ROLLBACK & AUTOMATED TESTS RE-DEPLOY TRIGGER COMPONENT FEATURE-LEVEL SHOWCASES TESTS TESTS PRODUCTIONPRODUCTION MANUAL TRIGGER ONGOING LIVE OPTIONAL FEATURE-LEVEL TESTING TRANSACTION TESTS SMOKE TESTS (any dashed STEP STORY-LEVEL BY THE CLIENT directional line) TESTS

CSS CHANGE IN THE UI CHANGE IN TRANSACTION PROCESSING ENGINE A change in the core processing engine with a low ABOUT DZONE A minor CSS change that could adversely impact a user's chance of aecting performance ability to use a low-tra area of the site

Continuous Delivery and DevOps are just a couple of the dozens of technology topics EXPLORATORY EXPLORATORY covered by DZone. At DZone.com you will find TESTING TESTING a wide variety of resources including more guides, code cheat sheets (RefCardz), hundreds of articles written by technology COMMITCOMMIT COMMITCOMMIT STAGING/PRE-STAGING/PRE- experts, and useful links shared by other UAT PRODUCTION technology professionals. If you are passionate about technology, then DZone has something A AT A AT to oer you. Come explore more at: DZONE.COM PRODUCTION PRODUCTION Electric Cloud End-to-End Continuous Delivery

Deliver... Enterprise Apps

Mobile Apps

Web Apps

Embedded Devices

Bacon

Cancel Faster

www.electric-cloud.com/cd

Used by leading companies to: Accelerate builds • Increase developer productivity • Improve product quality

Accelerate Software Delivery. www.electric-cloud.com 00.@electricCloud 2014 GUIDE TO CONTINUOUS DELIVERY

From Commit to Consumer: 3 “Must-Haves” for Speedy Delivery

Electric Cloud So, you’ve just checked-in another killer A solution that automatically detects fail because of differences between QA feature. What happens next? Well, that dependencies to eliminate broken builds is and production. A CD platform should depends. Some of you may even better. automate and normalize the build, End-to-End be able to build, test, and test, and deploy process across ANY It’s not enough 2. Faster feedback deploy code to production environment (public, private, or hybrid) You shouldn’t have Continuous Delivery within minutes. Most of you to simply automate with one-click simplicity. to wait until builds probably aren’t so lucky—it the software delivery or tests are 100% may be hours, days or even CD should enable developers to spend process. To remain complete before weeks before it sees the their time developing code, not worrying receiving feedback. light of day. competitive, it also about infrastructure. Look for a CD Deliver... A CD platform that needs to go faster. platform that can take it one step further allows real-time drill- Enterprise Apps Here are three ways the to accelerate the end-to-end process. Your down into warnings right Continuous Delivery code is worth it! and errors as they occur helps eliminate (CD) platform keeps you and your team Mobile Apps wasted time and CPU cycles. productive and happy:

3. Bulletproof and painless 1. Accelerated, error-free builds Web Apps processes Agile development creates more frequent WRITTEN BY CD isn’t continuous if jobs in flight are builds. A CD platform that optimizes and Rohit Jainendra lost when a CI server goes down, or Embedded Devices parallelizes large builds across hundreds, Chief Product Officer when it takes days for QA machines to Electric Cloud or even thousands of cores is good. be provisioned, or when deployments Bacon

ElectricCommander/ElectricAccelerator COMPANY: Electric Cloud CI, ARA Cancel Faster PHONE: (408) 419-4300 LOCATION: Sunnyvale, CA, USA LAUNCH DATE: September ‘06 @electriccloud

SUPPORTED TARGET SYSTEMS FULL PROFILE LINK dzone.com/r/Tdv6  Windows  Solaris  Linux  AIX DESCRIPTION  Mac OS X  Other Electric Cloud helps organizations accelerate the delivery of high-quality (See companion site) software. They automate and parallelize manual build, test, and deploy processes across pools of cloud infrastructure to significantly boost DevOps STRENGTHS productivity and throughput. Software-driven companies like Intel, Cisco, • Visual authoring of • Dynamic creation of Qualcomm, Siemens, HP, GE, SpaceX, GAP, E*TRADE, and GM use Electric www.electric-cloud.com/cd complex multi-path steps, procedures Cloud to reduce costs and accelerate innovation. workflows and workflows • Ability to create • Process-specific data-driven user impersonation Used by leading companies to: dynamic processes and fine-grained CUSTOMER SUCCESS STORY access control Urban Science is a global automotive retail sales analytics leader, with 20+ Accelerate builds • Increase developer productivity • Improve product quality customizable products that are deployed to 6 environments over 18 global data centers. Before working with Electric Cloud, Urban Science was using scripts NOTABLE CUSTOMERS and manual installation instructions customized for each environment, resulting • Qualcomm • HP in inconsistent environments, slow deployment times, and poor quality. Using • Cisco • SpaceX ElectricCommander, Urban Science was able to standardize deployment procedures, automate every step of their deployment workflow, optimize their use of IT resources, • GE • Intel and gain security and visibility which they previously did not have.

FREE TRIAL Hosting Options 30-day free trial • On-premise and cloud hosted Accelerate Software Delivery. www.electric-cloud.com

00.@electricCloud dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 21 dzone.com/research/continuousdelivery Infrastructure as Code: When Automation isn’t Enough

BY Mitch Pronschinske

Some of the best ideas in the tech industry, lofty goal for many of these cutting edge organizations many of which have revolutionized the way was to be able to completely rebuild a business’ software systems with nothing more than physical server resources, organizations build software, have come from a complete backup of their databases, and source code. the cross-pollination of strategies and best Today, that’s how many modern successful tech businesses practices that occurs when all teams from operate. They have the ability to do exactly that. the organization leave their silos and work Infrastructure Automation at Scale together on their most difficult problems. It’s Infrastructure automation is the process of automatically widely understood that progress often comes running a set of custom actions to install an operating system on a virtual or physical server and then configure from the sharing of ideas, so it’s no surprise it according to specifications. When virtualization and that the fundamental principles behind cloud infrastructure became widely available, the new IT DevOps are widely accepted as beneficial strategies that emerged were only achievable through some form of infrastructure automation. The ability to spin practices. up new virtual machines in minutes meant that a manual installation with step-by-step instructions for the sysadmin Agile workflow methodologies were embraced early on by wasn’t going to cut it anymore. Those previously manual the development community, and today they are widely steps would now need to be handled by software in shared with business and operations teams. Continuous conjunction with the creation of the VMs. Otherwise, the Delivery is an expansion of the concepts behind full value of cloud and virtualization strategies wouldn’t continuous integration. Continuous integration started be realized. as a development strategy and expanded to encompass production deployments, which meant bringing the However, infrastructure automation brings its own set of operations team into the mix. When the principles of challenges. Any form of automation can cause a host of DevOps came into play, the concept of Continuous new issues if it is not carefully managed. Server image Delivery emerged. templates and copies (clones) are the first step in handling the scale of cloud and virtualization management, but if One of the most powerful innovations spawned by those templates or copies have any instructions that will the DevOps movement was the idea that operations cause problems, bad configurations get copied again and departments could manage the configurations of servers again, propagating the issue to a large number of servers. and virtual machines the same way that developers Automating correct infrastructure settings on a large scale manage the configurations in their software’s source can be a powerful ability, but it can be equally devastating code. This doesn’t simply mean command line scripts, if automation instructions are not composed carefully. which sysadmins were already familiar with. It means using a description language or full-fledged programming There are also dangers that come with the ease and language to manipulate infrastructure in an automated, speed of growing an organization’s infrastructure portfolio repeatable way. when they have automation in place. It’s very difficult with the basic automation that many organizations use to keep up with an infrastructure that is constantly growing Infrastructure as CODE and changing. The biggest problem that infrastructure With the arrival of tools like CFengine, Puppet, and Chef, automation can cause is Configuration Drift. the concept of Infrastructure as Code was born. These tools enable developers and sysadmins to abstract their From the Continuous Delivery Report glossary: problems around maintaining modular, automatable infrastructure, and being able to define them using a high-level language. Once teams started using these tools Configuration Drift: A term for the general and techniques, building and maintaining their server environments began to closely resemble the way that tendency of software and hardware configurations to software developers build and maintain application source “drift,” or become inconsistent with the baseline or template code. version of the system due to manual ad hoc changes (like

All of the testability, repeatability, and transparency of a hotfixes) that are not introduced back into the template. modern software development process suddenly became available for people who were building infrastructure. The

PAGE 22 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

Many of today’s deployment problems and IT outages are • Refactor the infrastructure code to reduce the amount of caused by Configuration Drift. Configuration Drift isn’t maintenance needed. limited to infrastructure configuration, either. Middleware settings also experience a great deal of untracked changes in many organizations. This causes numerous Configuration Tooling ExpectationS Drift-related errors, especially in testing and acceptance Organizations don’t want to waste their time or money using environments. Disaster recovery and high availability failures tools that don’t provide the features needed to avoid the are also common results of Configuration Drift. Having a biggest pitfalls in Continuous Delivery. Beware of tools that feature in your deployment automation solution, whether let you easily create endless server templates but don’t allow it’s custom or provided by a product vendor, is highly proper traceability for template changes like the inevitable recommended. Detecting configuration drift is the first step quick fixes that are prevalent in software production. to overcoming it. The software should be able to externalize infrastructure code definitions that can be stored in common version It’s important to note that infrastructure automation is control systems. Organizations should be wary of tools not the same as Infrastructure as Code. Infrastructure that don’t allow the usage of community tools that benefit automation can take repeatable actions and enact them from large community across any number of servers, but Infrastructure as knowledgebases. It’s better Code includes tools and capabilities from the software to use tools that are flexible One of the most development world to build, maintain, and test those actions and allow you to produce in a more manageable way. future-proof systems that powerful innovations can take advantage of the spawned by the many innovations in the Developer Lessons for Infrastructure wider development tooling DevOps movement as CODE community. was the idea that The biggest danger when trying to integrate an Infrastructure Infrastructure as Code operations departments as Code-enabling configuration management tool like Chef, emerged from the SaltStack, or Ansible in your software production process is development community could manage the not applying the same lessons learned by the development through open source configurations of servers world in the last ten years. Infrastructure code needs to be configuration management version controlled, tested, maintained, quickly deployed, and tools that were shared because and virtual machines crafted for easy usability. so many organizations were having the same problems the same way that Development principles that date all the way back to when using basic, non- developers manage the the beginning of Extreme Programming will provide the programmatic infrastructure best guidance for making Infrastructure as Code into a automation. The best piece configurations in their blessing rather than a curse. These are the principles that of advice to internalize about software’s source code. organizations who are new to Infrastructure as Code should using Infrastructure as Code focus on: is to stay connected to the vast community of innovative • Include infrastructure code in the software development developers. If your systems remain open to the rapid changes lifecycle by putting it through build, testing/QA, and in that community, you’ll be able to share and benefit from production along with application code. the cutting-edge ideas that will make your organization • Start with and maintain a simple design for your successful. infrastructure code and any new features. Use design patterns.

• Have the infrastructure team agree on the design and direction of the system. • Institute code reviews with the help of team notifications for each commit. Pair programming is also helpful. • Maintain a shared set of coding standards for the infrastructure code. WRITTEN BY • Continuously test with as many of the same styles of development testing that could be applicable to Mitch Pronschinske Infrastructure as Code. This ensures that the code Mitch Pronschinske is the Head Analyst for definitions produce the correct environments and don’t DZone’s Research division. He has been writing, introduce problems. curating, and editing content for an audience of IT professionals for over four years. In that time • Employ infrastructure monitoring for testing and pre- he has learned the complexity that software production environments as well as the production producers deal with on a daily basis, and he strives environment. to make their world easier to understand and digest.

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 23 dzone.com/research/continuousdelivery

Continuous Delivery MATURITY CHECKLIST Check the boxes next to the practices you currently perform to see your maturity in each area of Continuous Delivery. Add up your score at the end based on the highest levels you checked.

Source CONTROL DepLOYMENT

Baseline Baseline FF Early branching FF Fully scripted deployments FF Branches tend to remain apart Novice Novice FF Push-button deployments to test environments FF Branches are used for isolating work Intermediate FF Merges are common FF Auto deploy to first test environment Intermediate FF Standard deployments across all environments FF Pre-tested commits FF Push-button deployments to production FF Integration branch is pristine Advanced Advanced FF Automated deployments after tests pass FF All commits are tied to tasks FF Database deployments FF History used to rewrite features before pushing to central FF Multi-tier deployments repository FF Version control DB schema changes expert FF Ability to implement continuous deployment Expert FF Traceability analysis and release notes auto-generated Visibility FF Commits are clean enough for the master branch/trunk Baseline FF Build status notification is sent to committer Build Process Novice Baseline FF Latest build status is available to all team members FF Official builds are not performed on developers’ machines FF Self-service build or nightly build Intermediate FF Trend reports are automatically generated from build server Novice events FF System polls source control and builds on commit FF People outside the team can subscribe to build statuses FF Build artifacts are managed, some manual scripts still used Advanced Intermediate FF Stakeholders have dashboards with real-time product and dependency stats FF Build artifacts are managed by purpose-built tools, no manual scripts Expert FF Dependencies are managed in a repository FF Cross-team data mining and analysis

Advanced FF Distributed builds on build cluster, can be done in sequence FF Source control tells system when to build, no polling Overall Maturity scorecard Expert For each check mark add the assigned number of points and FF Build environments based on VMs total them for each section: FF Streams are never “broken” FF Gated commits TALLY YOUR SCORES:

Testing & QA POINT KEY: Source Control Baseline • Baseline 0 points FF Automatic with every build • Novice 1 point Build Process F F Code coverage is measured • Intermediate 2 points Novice Testing & QA • Advanced 3 points FF Peer-reviews • Expert 4 points DEPLOYMENT FF Mockups & proxies used

Intermediate Visibility FF Periodic static code analysis FF Automated functional testing TOTAL Advanced FF Integrated management and maintenance of the test data 0-7 Adequate 8-11 Average 12-14 Skilled FF Automated performance & security tests in target environments 15-17 Adept 18-20 Master Expert FF Automated acceptance testing Inspired by Chris Shayan and Eric Minick

PAGE 24 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

SOLUTIONS DIRECTORY

In the following Continuous Delivery Solutions By visiting this URL, you will be taken to an expanded Directory, you will find a side-by-side comparison profile for the given solution where you can explore a of the many solutions available to build out your much more comprehensive set of features and details. continuous delivery toolchain. Each solution profile In addition to the ability to see expanded profiles, will give you a glimpse of the essential set of criteria the companion site also gives you the ability to run a for categorizing and assessing each tool. side-by-side comparison of selected solutions by your chosen features and criteria.

Along with a basic snapshot, each profile also includes a DZone companion site “shortcode” URL Alternatively, you can visit http://dzone.com/research/ that looks like this: continuousdelivery and enter any shortcode in the search bar to find your selected vendor.

Ansible Open Source CM Ansible Tower CM Product Launch: Feb-12 TWITTER: @ansible LOCATION: Santa Barbara, CA, USA Product Launch: Jul-2013 TWITTER: @ansible LOCATION: Santa Barbara, CA, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/9szr  Windows dzone.com/r/tRUJ  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Ansible is an open source IT  Solaris Ansible is an IT orchestration engine  AIX configuration management  AIX that models your infrastructure engine, designed to be minimal by looking at how all of your  Other (See companion site)  Other (See companion site) and lightweight. Ansible models systems inter-relate, rather than STRENGTHS infrastructure by looking at how all STRENGTHS just managing one system at a • Free and open source systems inter-relate, and manages • Self-documenting time. Ansible Tower is the browser- systems parallel to each other. To YAML-based automation based Ansible UI, available as a Agentless model • language reduces complexity of further its goal of minimal design commercial product. It features managing systems and requirements, Ansible does not • Automates deployment, push-button automation, LDAP config management, Self-documenting use remote agents. It also uses SSH integration, scheduling, and a full • provisioning, and YAML-based automation to ensure system security, removing REST API. Tower also features out- orchestration language complexities around upgrading of-the-box integration with AWS and • Pulls cloud inventories All user-submitted and software and security patches. Rackspace. • automatically and officially tested modules assists auto-scaling packaged with Ansible scenarios • Full-REST API browser shows off every resource Agent Model • Agentless FREE TRIAL Agent Model FREE TRIAL Open source • Agentless Free up to ten nodes

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 25 dzone.com/research/continuousdelivery

Atlassian Bamboo CI Attunity RepliWeb ARA Product Launch: Feb-07 TWITTER: @Atlassian LOCATION: San Francisco, CA, USA Product Launch: Jan-07 TWITTER: @attunity LOCATION: Burlington, MA, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/4Lks  Windows dzone.com/r/7Muh  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Bamboo has all the capabilities of a  Solaris RepliWeb is a deployment platform  AIX CI server by running builds and tests.  AIX for applications and large-file It also connects issues, commits, content. The platform addresses  Other (See companion site)  Other (See companion site) test results, and deploys using other a wide range of deployment, STRENGTHS pieces of the Atlassian tool suite, STRENGTHS replication and data transfer • Build results and such as JIRA. This gives the entire • Accelerated large-file scenarios for operations, developers, deployment status project team, including developers, deployments across content creators, and business inside JIRA issues project managers, testers, and WANs and LANs continuity teams. RepliWeb includes • Automatically applies sysadmins, wide visibility into the • Ad hoc and a template deployment processes, your CI scheme to entire project from commit to programmatic package customized UIs with role-based new branches in your deployments production. To support Continuous access, proprietary transfer engines, repository Delivery, Bamboo can automate the • Defines multi-tiered, comprehensive monitoring and • Deployment automation entire delivery pipeline. multi-stage deployment reporting, and full rollback across from master or dev topologies branches server and application states. • Provides lines-of- • Easy parallelization of business with job build steps execution and visibility into deployment

Hosting Options Supported CI Engines

• On-premise and cloud FREE TRIAL • Microsoft TFS FREE TRIAL hosted 30-day full-feature trial 15-day free trial with approved POC definitions

Automic Deployment Manager BMC Release Lifecycle ARA Management ARA Product Launch: Jan-10 TWITTER: @automic LOCATION: Wolfsgraben, Austria, EU Product Launch: Oct-11 TWITTER: @BMC_DevOps LOCATION: Houston, TX, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/RdL7  Windows dzone.com/r/YQWw  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Automic’s ONE Automation Platform  Solaris BMC Release Lifecycle Management  AIX provides a centralized platform for  AIX is a process flow and application developers, QA, and operations to deployment solution that includes  Other (See companion site)  Other (See companion site) coordinate and automate the entire technology from Varalogix, a STRENGTHS application release. Deployments STRENGTHS deployment automation firm that • Use the same workflow and releases can be coordinated • Includes planning, BMC acquired in 2012. The broad in every environment with other critical business processes coordination, and ARA solution now features a (Dev - QA - UAT - such as workload, batch, or approval, automation execution centralized management console Production) ensuring that releases do not have • Compatible with many and a release repository for archiving • Automatic rollback any negative impact on business as existing prominent open application components, along with source and commercial Deployments done with usual. a drag-and-drop web UI for building • products managed file transfer multi-tier application deployment • Trend-spotting features templates. for the release process in visualization interface

Supported CI Engines Supported CI Engines

• Jenkins/Hudson FREE TRIAL • Jenkins/Hudson FREE TRIAL • Atlassian Bamboo • Atlassian Bamboo 7-day free trial No free trial • JetBrains TeamCity • JetBrains TeamCity • Microsoft TFS • Microsoft TFS

PAGE 26 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

CA Release Automation ARA CFEngine Community CM Product Launch: Jan-07 TWITTER: @cainc LOCATION: Islandia, NY, USA Product Launch: Jan-94 TWITTER: @cfengine LOCATION: Oslo, Norway, EU

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/wkdP  Windows dzone.com/r/hQyX  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris CA Release Automation is an  Solaris CFEngine Community is an open  AIX enterprise-class continuous delivery  AIX source infrastructure automation orchestration solution to design, framework, originally developed  Other (See companion site)  Other (See companion site) test, and automate complex out of an academic setting. STRENGTHS application release deployments STRENGTHS CFEngine helps engineers, • Plug-ins for automating with visibility into the entire process • Free and open source system administrators, and other and integrating third- for all stakeholders. Integrating under the GNU GPL stakeholders in an IT organization party solutions without technologies acquired from license manage and understand IT scripting Nolio, the CA LISA platform helps • Designed to scale infrastructure throughout its lifecycle. • Application-centric, enterprises simplify and standardize Continuously monitors Solutions running CFEngine use manifest-driven • the application release processes, and self-repairs IT decentralized, autonomous agents deployment provides systems reusable and repeatable increasing the speed of application to scale and automatically configure processes to simplify release cycles while improving • Able to use any IT systems. CFEngine was first application releases. . integrated policy editor built in 1993, well before its present • Graphical workflow for competitors in the CM space. creating deployment processes • Defines and manages release activities across Agent Model application components FREE TRIAL • Agents Installed FREE TRIAL

Supported CI Engines No free trial Open source

• Jenkins/Hudson • JetBrains TeamCity • Microsoft TFS

CFEngine Enterprise CM CircleCI CI, ARA Product Launch: Jan-94 TWITTER: @cfengine LOCATION: Oslo, Norway, EU Product Launch: Jul-12 TWITTER: @circleci LOCATION: San Francisco, CA, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/ALtU  Windows dzone.com/r/uNhy  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris CFEngine is an infrastructure  Solaris CircleCI is a hosted continuous  AIX automation framework that helps  AIX integration and deployment platform engineers, system administrators that can infer how to run CI and CD  Other (See companion site)  Other (See companion site) and other stakeholders in an from the details of a user’s code STRENGTHS IT organization manage and STRENGTHS based on the first test. It supports • Designed to scale, with understand IT infrastructure • Build machines are web applications on a Linux stack, up to 5,000 hosts per throughout its lifecycle. It takes managed as part of the including Ruby, Python, Node, PHP, server hub systems from build to deploy, product and Java applications. This includes • Perfect security record manage, and audit. CFEngine was • Automatic test optimized build speed, automatic based on nist.gov first built in 1993, well before its parallelization test parallelization, SSH access to standards present competitorsin the CM space. • SSH access to builds builds, built-in artifact management, • ”Sketches” help users In contrast to its competitors, it Built-in artifact tight GitHub integration, and instant perform sysadmin • originates from an academic setting management setup of new repositories. tasks without in-depth knowledge of the rather than a commercial one. • Tight GitHub integration product • Runs virtual machines • CFEngine AS provides with user’s choice of training and consulting database.

Agent Model Supported CI Engines FREE TRIAL CircleCI FREE TRIAL • Agents Installed • Free trial limited to 25 managed 2 weeks unlimited hosts. Community edition also available.

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 27 dzone.com/research/continuousdelivery CloudBees Continuous Delivery Platform CI Codeship CI Product Launch: Jan-11 TWITTER: @CloudBees LOCATION: Woburn, MA, USA Product Launch: May-11 TWITTER: @codeship LOCATION: Cambridge, MA, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/aVAt  Windows dzone.com/r/CWKT  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris The CloudBees Continuous Delivery  Solaris Codeship is a hosted Continuous  AIX Platform provides a range of Jenkins-  AIX Integration and deployment service. based Continuous Integration and Users can connect their Github or  Other (See companion site)  Other (See companion site) Continuous Delivery solutions on BitBucket repositories and Codeship STRENGTHS premise, in the cloud, or in a hybrid STRENGTHS will automatically run the user’s tests • Jenkins-based configuration. The CloudBees • Integration with GitHub when they push new commits. Upon Continuous Delivery Platform offers solutions for both and BitBucket successful completion of the tests, lifecycle development and DevOps teams • Web based users can also deploy to staging • On premise, public to utilize Jenkins and Platform as a configuration or production systems, so their cloud, or hybrid cloud Service technology in a configuration Completely hosted applications are always up to date configuration and • that best meets their needs. The solution with the latest changes. hosting options CloudBees CD Platform can also be • Build pipelines for • Hosted iOS and Android extended to work with technologies builds for mobile deployment Continuous Delivery across the application lifecycle. • Manage multiple Jenkins configurations across the enterprise

Hosting Options Hosting Options

• On premise, public FREE TRIAL • Cloud hosted FREE TRIAL cloud, or hybrid cloud 30-day free trial 50 builds per month for free

ElectricCommander Drone.io CI ElectricAccelerator CI, ARA Product Launch: Feb-14 TWITTER: @droneio LOCATION: San Francisco, CA, USA Product Launch: Sep-06 TWITTER: @electriccloud LOCATION: Sunnyvale, CA, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/JvfV  Windows dzone.com/r/Tdv6  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Drone is a Continuous Integration  Solaris Electric Cloud helps organizations  AIX solution built on Docker, an open  AIX accelerate the delivery of high- source tool to pack, ship, and run quality software. They automate and  Other (See companion site)  Other (See companion site) any application as a lightweight parallelize manual build, test, and STRENGTHS container. Every build is virtualized in STRENGTHS deploy processes across pools of • Single binary file a Docker container. Drone provides • Visual authoring of cloud infrastructure to significantly installation with only dozens of pre-built Docker images complex multi-path boost DevOps productivity and Docker as a dependency for nearly every popular language workflows throughput. Software-driven • Drone is open source and database, eliminating the need • Ability to create companies like Intel, Cisco, data-driven dynamic Dashboard optimized to setup and configure your build Qualcomm, Siemens, HP, GE, SpaceX, • processes for large monitors or TV environment. Drone is free and open GAP, E*TRADE, and GM use Electric screens source with a cloud-hosted offering • Dynamic creation of Cloud to reduce costs and accelerate steps, procedures and Drone CLI for local available. innovation. • workflows builds that run like cloud-based builds • Process-specific user impersonation and fine- grained access control

Hosting Options Supported CI Engines

• On-premise and cloud FREE TRIAL • Jenkins/Hudson FREE TRIAL hosted • Microsoft TFS Open Source 30-day free trial

PAGE 28 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

Enterprise Chef CM EpicForce Leroy ARA Product Launch: Jun-11 TWITTER: @chef LOCATION: Seattle, WA, USA Product Launch: Dec-13 TWITTER: @LeroyDeploy LOCATION: Carrboro, NC, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/rfNp  Windows dzone.com/r/47ks  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Enterprise Chef models IT  Solaris Leroy is an application deployment  AIX infrastructure and application  AIX automation engine, written in C++, delivery as code, providing an that is free to use. It allows engineers  Other (See companion site)  Other (See companion site) integrated tool chain and automation to describe the entire deployment, STRENGTHS platform. Ruby on Rails developers STRENGTHS properties, and application configs • Pushes workloads to the will find Chef’s syntax especially • Agents compiled for using a simple XML format. This edges of the network familiar, as will many users with each platform allows the application deployment • Efficiently uses programming backgrounds. • Multi-credential agent to be treated like a versioned server resources and Enterprise Chef enables rapid model without having to component in the repository, bandwidth provisioning and deployment of store user credentials like the rest of the code. Leroy is • RBAC, multi-tenant IT resources and the automated • Agents contain an relatively new, but it is being used support, and delivery of applications and services embedded Python in production environments at cryptographic validation at massive scale. scripting environment companies like Fleetmatics. for clients and servers • Application and • Out-of-box integrations deployment engine with popular tools configs are in one scope and programmatic extensions for other tools Supported CI Engines • Jenkins/Hudson Agent Model FREE TRIAL FREE TRIAL • JetBrains TeamCity • Agents Installed Open source version available Free solution

Hudson Continuous Integration Server CI IBM UrbanCode Deploy ARA Product Launch: May-06 TWITTER: @hudsonci LOCATION: Ottawa, Canada Product Launch: Sep-11 TWITTER: @UrbanCode LOCATION: Armonk, NY, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/3hjG  Windows dzone.com/r/sh3j  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Hudson is an open source  Solaris IBM UrbanCode Deploy (formerly  AIX Continuous Integration (CI) server,  AIX uDeploy) is an application hosted by the Eclipse Foundation. deployment automation solution  Other (See companion site)  Other (See companion site) Hudson’s job is to automate the to deploy and test new builds. It STRENGTHS process of building, testing and STRENGTHS integrates with many CI and test • Self-contained easy to deploying software over a wide • Direct support for tools to drive rapid feedback, and is install by running the range of technologies and platforms mobile, distributed, and designed to facilitate collaboration WAR file in an open and extensible manner. mainframe components in agile development environments. • Massive plugin Hudson has a low barrier to entry • Builds, database UrbanCode Deploy also provides ecosystem and is simpler to set up and run than changes, config, etc audit trails, versioning, and bundled into snapshots Architected to meet the equivalent manual processes. approvals. The tool also tracks which • for consistent testing the needs of enterprise The Jenkins CI engine is a fork of the components make up an application scale development Hudson project. • Support for true so they can be deployed and active-active horizontal Java-based, runs • scalability and failover monitored together. on a wide range of environments • Tracks any discrepancies that could cause configuration drift

Hosting Options Supported CI Engines

• On-premise and cloud FREE TRIAL • Jenkins/Hudson FREE TRIAL hosted • JetBrains TeamCity Open Source 30-day free trial • Microsoft TFS • Rational Team Concert • AnthillPro • Rational Build Forge

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 29 dzone.com/research/continuousdelivery

Inedo BuildMaster ARA JetBrains TeamCity CI Product Launch: Oct-09 TWITTER: @inedo LOCATION: Berea, OH, USA Product Launch: Oct-06 TWITTER: @ LOCATION: Czech Republic, EU

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/bWCK  Windows dzone.com/r/dJPM  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Inedo BuildMaster utilizes approval  Solaris TeamCity is a continuous integration  AIX and promotion requirements to  AIX and build server. It features out- efficiently automate deployment of-the-box integration with many  Other (See companion site)  Other (See companion site) processes from source control version control systems and it STRENGTHS through production. It uses STRENGTHS has dedicated “no-configuration” • Designed for use by automatic builds to create builds • Reporting determines integration with many build tools entire team and run unit tests at specific times, new failures and assigns and testing frameworks in Java, .NET, investigations • Flexibility & Extensibility trigger builds via source control and Ruby. TeamCity also features activity, ping a URL using another • Support for non-linear reporting both in the web browser Growing integrations • build pipelines library tool, or integrate with an existing CI and IDE plugins. system. Users can see the state of • Reuse settings with Can manage from • templates, hierarchical source control through all planned and past releases, with projects, and parameter production detailed historical data and auditing. references Web UI allows users • • Bundled with other to set up as many JetBrains tools like deployment plans as IntelliJ IDEA they need

Supported CI Engines Hosting Options

• Jenkins/Hudson FREE TRIAL • On-premise FREE TRIAL • JetBrains TeamCity 45-day Full-Featured Enterprise Free professional version. Can build • Microsoft TFS Trial or Non-expiring Feature- up to 20 different builds. Some limited Express Edition (5 users, 10 plugins are open source. application, 5 servers)

MidVision RapidDeploy ARA Octopus Deploy ARA Product Launch: Jun-08 TWITTER: @RapidDeployTM LOCATION: London, UK, EU Product Launch: Jul-12 TWITTER: @octopusdeploy LOCATION: Australia

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/ytX9  Windows dzone.com/r/KdTv  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris RapidDeploy is an application  Solaris Octopus Deploy is a deployment  AIX release automation originally created  AIX automation system built for .NET for a large financial institution. It developers. Octopus works with  Other (See companion site)  Other (See companion site) features integrations with many a build server to enable secure, STRENGTHS popular source control, build, and STRENGTHS automated releases of ASP.NET • Agent and agentless repository tools, and has support • Built-in conventions applications and Windows Services deployments for many middleware, messaging, for .NET configuration into test, staging and production settings and transforms • File & API based and database solutions. It provides environments, whether they are deployments to support self-service deployment capabilities • Conventions for IIS in the cloud or on-premises. Red cloud and legacy as well. Integrations are available configuration and Gate’s Deployment Manager is based platforms Windows Service for monitoring and ITSM, as well as on a copy of the Octopus Deploy deployment • Broad range of support image-based provisioning and cloud codebase. out-of-the-box for third- providers. • Encrypted database party tools • Secure variable • Supports several management for storing compliance and audit sensitive settings capabilities • Comprehensive REST API Supported CI Engines • Jenkins/Hudson FREE TRIAL Supported CI Engines FREE TRIAL • CruiseControl 5 nodes free forever or unlimited 30 • JetBrains TeamCity All features free for 45 days. Free • RTC day free trial. • Microsoft TFS version available for 5 projects or 10 agents.

PAGE 30 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

Open Source Chef CM Puppet Enterprise CM Product Launch: Jan-09 TWITTER: @chef LOCATION: Seattle, WA, USA Product Launch: Feb-11 TWITTER: @puppetlabs LOCATION: Portland, OR, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/RrL7  Windows dzone.com/r/j9GR  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Open Source Chef is an open-source  Solaris Puppet Enterprise is an IT  AIX configuration management tool,  AIX automation product that gives and is the basis of Chef Enterprise. system administrators the ability to  Other (See companion site)  Other (See companion site) Chef allows users to manage the automate repetitive tasks, deploy STRENGTHS infrastructure behind applications STRENGTHS critical applications, and proactively • Free and open source by automatically configuring each • DSL that is easy to learn manage infrastructure on premise resource to the desired state. and use for operations or in the cloud. Puppet Enterprise Integrations with • teams and developers Amazon EC2 and Users can also create QA and pre- automates tasks at any stage Rackspace production environments. • A secure declarative of the IT infrastructure lifecycle, model that enables true Able to support multiple including: discovery, provisioning, • simulation runs environments OS and application configuration • Tight integration Chef servers can management, orchestration, and • with VMware, Cisco, manage over 10,000 reporting. Microsoft, and others. nodes • Large, active community of users and contributors

Agent Model Agent Model

• Agents Installed FREE TRIAL • Agents Installed FREE TRIAL Open source Free up to ten nodes, Open source project available

Puppet Open Source CM Rocket ALM Hub ARA Product Launch: Jan-05 TWITTER: @puppetlabs LOCATION: Portland, OR, USA Product Launch: Jan-01 TWITTER: @rocket LOCATION: Waltham, MA, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/TMv6  Windows dzone.com/r/yaX9  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Puppet Open Source is an IT  Solaris The Rocket ALM Hub is an end-  AIX automation product under the  AIX to-end solution that provides Apache 2.0 license that gives multi-platform application lifecycle  Other (See companion site)  Other (See companion site) system administrators the ability to management automation for STRENGTHS automate repetitive tasks using a STRENGTHS deployment automation. Rocket • Free and open source declarative, model-based approach • Multi-platform focuses on organizations managing under Apache 2.0 to IT automation. Puppet Open management multiple releases, ensuring license Source automates tasks at many • Point and click process applications are deployed to the • Large, active stages of the IT infrastructure setup correct locations with the most community of users and lifecycle, including: provisioning, Integrated issue tracking up-to-date builds. The Hub provides contributors • OS and application configuration automation, visibility to everyone, • Integrated service desk • DSL that is easy to management, and orchestration. security, and repeatable process learn and use for both • Automated promotion, creation. The software also includes operations teams and build, and deployment developers reporting and IT compliance. • A secure declarative model that enables true simulation runs

Agent Model Supported CI Engines

• Agents Installed FREE TRIAL • Jenkins/Hudson FREE TRIAL Open source Free trials are limited by time

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 31 dzone.com/research/continuousdelivery

Rundeck ARA SaltStack Enterprise CM Product Launch: Jul-11 TWITTER: @rundeck LOCATION: Redwood City, CA, USA Product Launch: Feb-11 TWITTER: @saltstackinc LOCATION: Salt Lake City, UT, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/HGQ4  Windows dzone.com/r/jxGR  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Rundeck is an open source utility  Solaris SaltStack Enterprise delivers an  AIX designed to help operations teams  AIX infrastructure communication automate routine operational bus for real-time infrastructure  Other (See companion site)  Other (See companion site) procedures and deployment. automation, cloud provisioning, and STRENGTHS Teams can collaborate through STRENGTHS orchestration, as well as application • Designed for Rundeck to share how processes are • High-speed, always-on stack configuration management. automating all ongoing automated, delegate execution of communication bus SaltStack is known for its speed operational tasks those processes, and share visibility • Instantly react to events and its ability to scale up to • Fully open source into operational activity. Rundeck spanning networks, data support environments with tens of software project also includes access control, centers, or continents thousands of systems, unmodified, • Extensible through first workflow building, scheduling, • Infrastructure-wide with no performance or functional and third-party plugin logging, notifications, and plugins deployment and degradation. SaltStack Enterprise systems for integration with external data orchestration is identical to the open source • Built for enterprise scale sources and tools. • Self-healing capabilities Salt project, but offers enterprise concerns, including Imperative or support. security and auditing • declarative model

Supported CI Engines Agent Model

• Jenkins/Hudson FREE TRIAL • Can Install Agents or Go FREE TRIAL Agentless • Atlassian Bamboo Open source software Open source

Semaphore CI Serena Release Manager ARA Product Launch: Mar-12 TWITTER: @semaphoreapp LOCATION: Serbia, EU Product Launch: Apr-08 TWITTER: @serenasoftware LOCATION: San Mateo, CA, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/yHX9  Windows dzone.com/r/7Nuh  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Semaphore lets developers create a  Solaris Serena Release Manager is an ARA  AIX continuous delivery workflow with  AIX and configuration management a testing and deployment solution solution that allows developers  Other (See companion site)  Other (See companion site) in the cloud, built on dedicated to manage the implementation of STRENGTHS hardware. Semaphore automatically STRENGTHS software changes in the production • Quick setup process configures the build and deploy • Insights through environment. Release Manager is through GitHub environment for a growing list dashboards and designed to support the application integration of languages and frameworks. It reporting lifecycle from • Automatic project works without any change in source • Approval and demand to deployment, including configuration that works code. Collaborators can follow the notification system can planning and tracking the steps without any changes in be calendar-based, rule- progress on a live project timeline. of releasing applications, greater source code based, or on-demand Semaphore requires that you use visibility into those processes, and a • Parallel testing GitHub for source control, and it is • Reusable processes way to enforce release policies. capabilities for that can be tailored languages and tightly integrated with their system. to release type with frameworks supporting audit trail • Live project timeline • Single UI that supports and server deployment multi-platform apps history Supported CI Engines • Jenkins/Hudson Hosting Options FREE TRIAL FREE TRIAL • JetBrains TeamCity • Cloud hosted 30-day free trial 30-day free trial • CloudBees

PAGE 32 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

Shippable CI Thoughtworks Go CI Product Launch: Mar-14 TWITTER: @BeShippable LOCATION: Seattle, WA, USA Product Launch: Jul-10 TWITTER: @thoughtworks LOCATION: Chicago, IL, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/khs3  Windows dzone.com/r/N3pH  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Shippable is a continuous integration  Solaris ThoughtWorks Go is a Continuous  AIX and deployment cloud service. It is  AIX Delivery solution that offers free to use for private repositories development, business, and  Other (See companion site)  Other (See companion site) on GitHub or BitBucket and operations team members a STRENGTHS natively supports Docker. Users STRENGTHS common platform for collaboration. • Utilizes dedicated can sign in with their GitHub or • Deployment Pipelines as Go automates and streamlines Docker containers for BitBucket credentials and turn a first-class concept the build-test-release cycle for each user CI on for repositories they want • Dashboard provides continuous delivery of applications. • Users can trigger to enable. Shippable offers real automatic, traceable It was built with the Continuous multiple builds to test time notifications, matrix builds, bug reporting Delivery book’s principles in mind. language runtimes and integrated test/code coverage Allows for custom Go is now open source as of March environments • visualizations, and build history. automation of testing 2014, and the source code will be • Integrated test results functions available in April 2014. and code coverage Model complex visualization • workflows using job, • Offers granular team- tasks, stages, and based permissions for pipelines each member

Hosting Options Hosting Options

• On-premise and cloud FREE TRIAL • On-premise FREE TRIAL hosted Unlimited builds, unlimited public Open Source repositories, up to 5 private repositories, 1 parallel build minion

XebiaLabs XL Deploy Travis CI CI ARA Product Launch: Feb-11 TWITTER: @travisci LOCATION: Berlin, Germany, EU Product Launch: May-11 TWITTER: @xebialabs LOCATION: Boston, MA, USA

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/94zr  Windows dzone.com/r/M3aA  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Travis CI is a hosted continuous  Solaris XL Deploy is an agentless application  AIX integration and deployment platform  AIX release automation solution. that began life as a free platform XL Deploy combines intelligent  Other (See companion site)  Other (See companion site) for open source projects. Now deployment automation, the STRENGTHS Travis CI is a fully hosted product, STRENGTHS richest content set for enterprise • Integration and supporting both Linux and Mac/ • Agentless architecture middleware, and seamless collaboration around iOS build environments. Travis CI integration into the application Declarative automation GitHub pull requests • requires GitHub as its version control that eliminates scripting delivery ecosystem. XL Deploy’s • Support for Mac and iOS system, and it tightly integrates and workflow design unique, agentless architecture means builds with GitHub to test pull requests. • Fully automated rollback less maintenance, lower costs, and • Large community Travis CI also has integrations with and recovery support for much more diverse contributions and several databases and supports most Enterprise security and deployment scenarios that will following • programming languages. centralized auditing become common with the Internet • Command line tooling • Visualize Continuous of Things. • Completely free for Delivery pipelines with open source projects template reports

Hosting Options Supported CI Engines

• On-premise and cloud FREE TRIAL • Jenkins/Hudson FREE TRIAL hosted • Atlassian Bamboo 100 builds Fully functional, time-limited • Microsoft TFS evaluation version. Trial license extensions available upon request.

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 33 dzone.com/research/continuousdelivery

Zend Server ARA ZeroTurnaround LiveRebel ARA Product Launch: Feb-08 TWITTER: @zend LOCATION: Cupertino, CA, USA Product Launch: May-11 TWITTER: @liverebel LOCATION: Estonia, EU

Supported Target Systems: FULL PROFILE LINK Supported Target Systems: FULL PROFILE LINK  Windows dzone.com/r/rVNp  Windows dzone.com/r/ULJf  Linux  Linux  Mac OS X DESCRIPTION  Mac OS X DESCRIPTION  Solaris Zend Server is an application server  Solaris ZeroTurnaround’s LiveRebel enables  AIX with a supported PHP runtime that  AIX the release of Java, .Net, PHP, and gives PHP developers, DevOps other apps automatically with  Other (See companion site)  Other (See companion site) engineers, and sysadmins tool configuration changes. Instead of a STRENGTHS integrations to automate and STRENGTHS constrictive plugin system, LiveRebel • Web APIs for integrating accelerate app delivery. Integrating • Automated rollbacks features a powerful command CI and config with tools like Jenkins, Chef, and with no downtime line with plenty of simple scripts management tools Nagios, Zend Server orchestrates • Version-controlled to help make flexible integrations • Multi-mode PHP app deployment automation, configuration with other tools. LiveRebel fits in app performance scalability, performance monitoring, management with continuous integration and optimization request analysis, and configuration • Self-contained provisioned infrastructure to help • Developer tools that management. deployment script create a full Continuous Delivery provide enhanced management with no pipeline that ties application builds debugging downtime to infrastructure deployments. • Long-term support for • Pre-written scripts to the open source PHP integrate with third runtime party tooling from many categories

Supported CI Engines Supported CI Engines

• Jenkins/Hudson FREE TRIAL • Jenkins/Hudson, FREE TRIAL Bamboo, and TeamCity • Atlassian Bamboo Free edtion. 30 days full access to all 2 free licenses Enterprise features.

PAGE 34 2014 GUIDE TO CONTINUOUS DELIVERY 2014 GUIDE TO CONTINUOUS DELIVERY

GLOSSARY OF TERMS

A Continuous Delivery - A software P production process where the software Agent - A program installed on specific can be released to production at any time Platform-as-a-Service (PaaS) - An physical servers in order to handle the with as much automation as possible for application-server-as-a-service (aPaaS) execution of various processes on that each step. provided over a network along with server. additional components that may include Continuous Deployment - A software middleware, development tools, and Agile Software Development - production process where changes are application controllers. An iterative and incremental software automatically deployed to production development workflow methodology with without any manual intervention. Production - The final stage in a several sub-methodologies. deployment pipeline where the software Continuous Integration (CI) - A will be used by the intended audience. Application Release Automation software development process where a (ARA) - A type of continuous software continuous integration server rebuilds a release solution that handles packaging branch of source code every time code is R and deploying applications into a variety committed to the source control system. of environments and ultimately into The process is often extended to include Rollback - An automatic or manual production. deployment, installation, and testing of operation that restores a database or applications in production environments. program to a previous defined state in B order to recover from an error. D Behavior-Driven Development S (BDD) - A deployment methodology Deployment - A term that refers to the that asserts software should be specified grouping of every activity that makes a Self-Service Deployment - A in terms of the desired behavior of the program available for use and moving it to feature where deployment processes are application, and with syntax that is the target environment. automated enough for developers to allow readable for business managers. project managers or even clients to directly Deployment Pipeline - An automated control push-button deployments. Build - A compiled version of software, process for getting software from version and typically a work in progress. control to the production environment Source Control - A system for storing, by moving those builds through multiple tracking, and managing changes to Build Agent - A type of agent used in stages of testing and deployment. software. This is commonly done through continuous integration that can be installed a process of creating branches (copies for locally or remotely in relation to the DevOps - An IT organizational safely creating new features) off of the continuous integration server. It sends and methodology where all teams in the stable master version of the software and receives messages about handling software organization, especially development then merging stable feature branches back builds. teams and operations teams, collaborate into the master version. This is also known and implement technology to increase as version control or revision control. Build Artifact Repository - A tool software production agility and achieve used to organize artifacts with metadata business goals. System Administrator (Sysadmin) - constructs and to allow automated A member of the operations department publication and consumption of those E who is responsible for the upkeep, artifacts. configuration, and stable operation Exploratory Testing - A manual of a server, multi-user computer, local C testing strategy where human testers area network (LAN), large program, or have the freedom to test areas where they community. Capacity Test - A test that is used to suspect issues could arise that automated determine the maximum number of users testing won’t catch. U a computer, server, or application can support just before failing. I Unit Testing - A testing strategy in which the smallest unit of testable code is Commit - An action that makes a set of Infrastructure-as-a-Service (IaaS) isolated from the rest of the software and potential revisions into permanent changes, - A self-service computing, networking, and tested to determine if it functions properly. also known as a “check in.” storage utility on-demand over a network. User Acceptance Test - The final phase Configuration Drift - A term for the Integration Testing - Testing that of where clients and general tendency of software and hardware occurs after unit testing, but before end-users determine whether the program configurations to drift, or become validation testing, where individual will work for the end-user in real world inconsistent, with the template version of software components are combined and scenarios. This stage is also known as beta the system due to manual ad hoc changes tested as a single group. testing. that are not introduced back into the template. Issue Tracking - Also known as bug V tracking or defect tracking, this is a Configuration Management - A process that allows programmers and Virtual Machine (VM) - A software term for establishing and maintaining quality assurance personnel to track the emulation of a physical computing resource consistent settings and functional attributes flow of defects and new features from that can be modified independent of the for a system. It includes tools for system identification to resolution. hardware attributes. administration tasks such as IT infrastructure automation (e.g. Chef, Puppet, etc.).

dzone.com/research/continuousdelivery | dzone.com | [email protected] | (919) 678-0300 PAGE 35 ONE Automation

Freedom to innovate ONE Automation is a business automation platform that transforms businesses.

Automate more. See more. Innovate more.

Watch our demo

ONE Automation means less complexity, more innovation Your business, development and operations teams are busier than ever. Confronted with an ever growing list of applications and infrastructure they are consumed with the complexity of running the business leaving them little time for innovative transformation.

ONE Automation lets you give them the one thing they need most - time.

Benefits Benefits Benefits for business for development for operations • Simple to use interface • Drag & drop deployment • Centralized script • Visibility of the entire workflows repository business process • Smart deployment model • Reuse the same workflows • Easily understand up and • Hundreds of built-in run across thousands of downstream applications books servers • See the availability of • Integration with source • Dynamic workload services that matter most control, binary repositories, management • Predict and respond to build and continuous • Self service catalog errors using analytics integration servers

What’s possible with ONE Automation Learn how Automic lets you go faster, for less cost whilst staying in control. ONE Automation is the only automation platform that combines all automation disciplines and operates across business, application and infrastructure layers. www.automic.com