Devops: Introducing Infrastructure-‐As-‐Code
Total Page:16
File Type:pdf, Size:1020Kb
DevOps: Introducing Infrastructure-as-Code Elisabetta Di Nitto, Damian A. Tamburri*, Michele Guerriero*, Matej Artac+, Tadej Borovsak+ *Politecnico di Milano +XLAB Corp. Roadmap o Session 1: DevOps In a Nutshell o Break! (5 mins) o Session 2: Infrastructure-as-code Explained through TOSCA o Session 3: Our proposal to connect Dev and Ops: the DICER tool Session 1 DevOps In a Nutshell Dev & Ops: the classical roles Dev focuses on o Developing features o Changing requirements o Releasing products Ops focuses on o Offering services o Providing guarantees and stability Dev vs. Ops “The system works correctly in the development machine but it does not on the operation machine” “10 days after successful deployment of last release, the system is overloaded” Who is guilty? Dev– team, or –Ops team? The problem: two disconnected groups • Dev works to apply releases • Ops resists to releases • Dev does not pay attention • Ops is aiming at to QoS guarantees guaranteering QoS • Changes are fundamental for the business! • QoS guarantees are needed too! DevOps • What is it: “Practices or tools that bridge the gap between development and operations” • Goal: Creates a collaborative mindset where a single team performs Dev and Ops àthe team must contain differentiated competences, background, etc. • Requires: • Culture management; • Automation tools; • Organisational as much as technical metrics • Continuous sharing artifacts, procedures, languages, approaches… DevOps Need 1: Process Alignment! - Unified Processes - Unified tooling Before DevOps Development IT Operations Tools Tools Development Test Production «Devs have to walk in Ops shoes and viceversa» DevOps QA – Quality Assurance Development & IT Operations Tools Development Test Production Collaborative Development Continuous Testing Continuous Monitoring The four DevOps values 1 2 Development and test Accelerated deploy on production-like using proper environment processes and tools 4 3 Continuous Continuous validation collaboration and of operation quality feedback IT performance and DevOps (1) Main IT performance metrics o Throughput o Deployment frequency o Lead time for release o Stability & Quality o Mean time to recover from failure IT performance and DevOps (2) Examples of DevOps practices that improve IT performance Throughput Stability & Quality Deployment frequency Mean time to recover • Continuous delivery • Version control • Version control • Monitoring system and application health Lead time for release • Version control • Continuous architecting • Continuous integration • Continuous testing Waterfall vs. Agile vs. DevOps Concerns Waterfall Agile DevOps Main Focus Design and Design and Whole application development development life-cycle Attention to Minimal or none Minimal or none Main concern Operation Role of Maintenance Maintenance seen as Maintenance seen as Replaces a running a redevelopment a redevelopment system with a running phase phase system Process Outcome A packaged system Various intermediate A continuously ready to be deployed demo versions of the updated running system system Quality Assurance Testing typically Test-driven Test-driven performed toward the development development and end of the process monitoring-driven operation with feedback to Dev Who focuses on DevOps (in some way)? All Others 21,9% Not … 1,3% Health-care 3,0% I don't … 2,1% NA 2,0% Retail 3,7% 10,000+ 15,8% I don't … 8,0% Government 4,5% 500-9,999 26,8% 10,000 > 8,5% Telecommunicati… 5,7% 5,000-… 4,9% Consulting 5,9% 100-499 21,8% ENTMT/Media 6,8% 20-99 17,1% 2,000-… 8,4% Finance/Banking 7,4% 10-19 5,8% 500-… 16,9% Education 7,5% 5-9 3,6% 100-499 23,0% Web Software 10,9% 1-4 5,8% <100 28,3% Technology 22,7% Industry Company Size by # of Size of IT infrastructure by # of Employees servers All Others 11,1% Other 5,2% Release Engineering 1,2% VP 0,9% Quality Assurance 1,3% Director 4,8% Information Security 1,4% Manager 8,3% Network Operations 1,9% System Engineer 23,4% C-level Executive 2,3% Build Engineer 5,2% Consultant 5,6% DevOps Engineer 31,3% DevOps 16,0% Automation 10,5% Dev/Eng 28,8% Architect 10,3% IT Ops 30,4% Departments DevOps Roles 2014 State of DevOps Report, Puppet Labs (9200+ respondents) Puppet Labs Findings and Survey recommendations o Clear correlation between high IT performance and strong business performance o Suggestions for improvement: Practitioners Managers Cross-functional • Work with other teams • Build trust collaboration • Find ways to build empathy • Encourage practitioners to move • Make invisible work visible between departments • Facilitate collaboration Climate of learning • Learn by sharing knowledge • Create and acquire training budget • Always bring back what you learned • Create a climate of learning and • Prepare for postmortems sharing • Make it safe to fail Tools • Automate the things that are painful • Make sure your team chooses the tools • Make monitoring a priority DevOps general advantages • Shorten time-to-value • Quick and regular release of software features • Improve quality • Mitigate risks • Increase collaboration in the team DevOps Organisational Changes What to avoid: anti-patterns Dev Ops Dev DevOps Ops Separate Silos Separate DevOps Silo Dev DevOps Ops We don’t need Ops From M. Skelton. What Team Structure is Right for DevOps to Flourish? 2013 http://blog.matthewskelton.net/2013/10/22/what-team-structure-is-right-for-devops-to-flourish/ http://web.devopstopologies.com DevOps Organisational Changes What to do: adoption patterns Dev Ops Dev Ops Dev DevOps Ops Smooth collaboration Fully Embedded Infrastructure as a Service Dev DevOps Ops Dev DevOps Ops DevOps as a Service Temporary DevOps Team From M. Skelton. What Team Structure is Right for DevOps to Flourish? 2013 http://blog.matthewskelton.net/2013/10/22/what-team-structure-is-right-for-devops-to-flourish/ http://web.devopstopologies.com DevOps processes and toolchain - Continuous Architecting Def. “architect for test, build and deploy, take quality attributes into account, take advantage of feedback from runtime” [1] - Continuous Integration Def. “merge all developer work-copies to a shared mainline frequently” [4] Examples. Apache Jenkins, Hudson, etc. - Continuous Testing Image by Kharnagy (Own work) [CC BY-SA 4.0 Def. “run tests as part the build pipeline (http://creativecommons.org/licenses/by-sa/4.0)], so that every check-in and deployment is via Wikimedia Commons validated” [3] Examples. Selenium+GitHub+LI-API, etc. DevOps process and toolchain 1.Configuration Management 5. Self- 2. Server Adaptation Provisioning 3. Application 4. Monitoring Deployment Our focus Infrastructure Host Host Network Middleware Apache Tomcat MySQL Application Mod_proxy WAR Schema A large number of tools A large number of tools A large number of tools One common characteristic o Code! Example 1 To define what a web node should do [Chef] { “name”: “web”, “description”: “Role used for the web tier”, “chef_type”: “role”, “json_class”: “Chef::Role”, “run_list”: [ “recipe[timezone]”, “recipe[apt]”, “recipe[nginx]” ], “default_attributes”: {}, “override_attributes”: {} } One common characteristic o Code! Example 2 To run a VM on OpenStack and to install a mongo shell on it [Terraform] resource "openstack_compute_instance_v2" "mongod_host" { count = "3" region = "" name = "mongod_host" image_name = "${var.image_name}" flavor_name = "${var.flavor_name}" key_pair = "tf-keypair-1" security_groups = ["mongo_security_group"] network { uuid = "${openstack_networking_network_v2.tf_network.id}" } ... provisioner "remote-exec" { scripts = [ "scripts/install_mongo.sh" "start_mongod.sh" ] } } From computation-specific machines By Alessandro Nassiri - Museo della Scienza e della Tecnologia "Leonardo da Vinci", CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=47910919 To programmable machines To programmable management of machines To run a VM on OpenStack and to install a mongo shell on it [Terraform] resource "openstack_compute_instance_v2" "mongod_host" { count = "3" region = "" name = "mongod_host" image_name = "${var.image_name}" flavor_name = "${var.flavor_name}" key_pair = "tf-keypair-1" security_groups = ["mongo_security_group"] network { uuid = "${openstack_networking_network_v2.tf_network.id}" } ... Becomes another subject of provisioner "remote-exec" { scripts = [ this same lifecycle "scripts/install_mongo.sh" "start_mongod.sh" ] } } Infrastructure as a Code Session 2 Infrastructure-as-code Explained through TOSCA Towards standard Infrastructure-as-Code è An Application Deployment Topology, i.e., “a graph of physical artefacts that need support for several lifecycle phases (e.g., procurement, installation, configuration, deployment, undeployment, teardown, etc.)” [6] Infrastructure Host Host Network Middleware Apache Tomcat MySQL Application Mod_proxy WAR Schema Towards standard Infrastructure-as-Code è Infrastructure-as-code, i.e., “a blueprint detailing physical artefacts, all scripts for all lifecycle phases and all artefacts needed for deployment” [6] IasC Infrastructure Host Host Network Blueprint Referenced in IasC Middleware Middleware Apache Tomcat MySQL Scripts (e.g., Chef, Puppet, etc.) Included in Deploy Application artefacts (e.g., Mod_proxy WAR Schema JARs, etc.) Where Does TOSCA fit into? TOSCA: Here’s What We’ve Seen there… o An application topology o 3 layers Infrastructure Host Host Network o Infrastructure (Cloud or DC objects) o Platform or Middleware (App containers) o Application modules, schemas and Middleware Apache Tomcat MySQL configurations Application Mod_proxy