Continuous Delivery Pipelines Automating the Value Stream Through Continuous Release
Total Page:16
File Type:pdf, Size:1020Kb
Continuous Delivery Pipelines Automating the value stream through Continuous Release Kevin A. Lee – [email protected] Senior Solutions Architect DevOps and Release Management Webinars (UK) •Automating DevSecOps: How to embed security into your continuous delivery pipelines •Enterprise DevOps: Release Management for the multi-modal Enterprise •Continuous Delivery Pipelines: Automating the value stream through continuous release https://www.microfocus.com/campaign/serena-release-management/ Agenda 01. Application Release Challenges 02. Implementing Continuous Delivery 03. Micro Focus Solutions 04. Demonstration 05. Q&A Application Release Challenges Modern Software Lifecycle Disruptions METHODOLOGY DevOps Adoption Waterfall Iterative Agile DevOps APPLICATIONS Software Complexity Monolithic Client-Server N-Tier Microservices + APIs INFRASTRUCTURE Cloud Transformation Mainframe Open Systems Virtualization/Cloud Cloud Native Shift to Continuous Releases Release Waterfall Requirements Design Implementation Verification Release Deploy Deploy Deploy Deploy (Internal Release) (Internal Release) (Internal Release) (Internal Release) Release Agile (Scrum) R D I V D R D I V D R D I V D R D I V D R D I V Release Iteration Deploy Deploy Deploy Release (Internal Release) (Internal Release) (Internal Release) Hybrid Agile Planning R D I V D R D I V D R D I V D Stabilization Release ScrumFall Deploy Deploy Deploy Deploy Deploy Deploy Deploy Deploy Deploy Deploy Kanban R D I V R D I V R D I V R D I V R D I V R D I V R D I V R D I V R D I V R D I V Feature Adoption of DevOps DevOps (development and operations) is an enterprise software development phrase used to mean a type of agile relationship between development and IT operations. The goal of DevOps is to change and improve the relationship by advocating better communication and collaboration between these two business units. (webopedia.com) Release Management? Adoption of DevOps – Many different meanings People - Culture Systems Thinking Value Stream Management Continuous Feedback Multidisciplinary Teams Autonomous Teams Experimentation & Learning Process - Techniques Agile Continuous Delivery Application Performance Management Shift Left Microservices Release Management Containerization Infrastructure as Code Tools - Automation Value Stream Management Customer Time to Value (end-to-end lead time) PPM Agile – Development Lead Time DevOps - Deployment Lead Time ITSM Ideate Create Release Operate Plan Develop Test Provision Deploy Support Change is checked in Change is running to version control in production . A value stream is the sequence of activities an . Organizations have focused on reducing waste by organization undertakes to deliver on a customer need optimising Agile and now DevOps Lead Times . Value stream per product, application or service . But still struggle with visibility & management at cross product (portfolio) level and optimizing value in approval centric IT environment (service) Most Organisations are “Multi-modal” Web-Scale Agile Traditional IID* Waterfall Continuous / Experimental Systems of Innovation Systems are mostly dependent Governance Systems of Change Differentiation Everyone needs to embrace DevOps Systems of Record *Iterative and Incremental Development And need to Balance Agility and Control “Move fast without breaking things” Governance, Sarbanes-Oxley, Segregation of Information Risk and PCI-DSS, HIPAA … Duties Security, Audit Compliance Multi-platform Distributed Mainframe Mobile Development Risk Integration to Integration to Dependent Core Systems, third-party Back Office Applications providers Systems Innovation Multiple Apps Shared Schedule Access running on to Test Environments Infrastructure Customer Performance Representative Testing Environments Implementing Continuous Delivery What is Continuous Delivery? Developer • potentially be able to release "at any moment" a version into production Commit • Kanban style-focus - small batch sizes, short iterations, fast feedback • manage and collate releasable artefacts in artefact repository • incorporate manual tasks, approvals where necessary Compile Release Source Code Test Deployment Pipeline On Demand Tests Deploy Approval(s) Version Control Build Artifact Repository Integration UAT Production Continuous Delivery Continuous Deployment = fully automated Developer push to production Example Continuous Delivery Capabilities Agile / Kanban Continuous Configuration Deployment Test Data Release Performance Integration Automation Automation Automation Migration Coordination Monitoring Auto provisioning of Convergent - self “Production” like healing Integration to environments for infrastructure Rollback automation Change Automated build, testing Automated Management Real User artifact staging and customer reference Monitoring code level testing data migrations Automated Automated infrastructure performance testing provisioning for of all devices environments Self-service Environment deployments into Scheduling and environments Planning Environment Automated API-level inventory testing Integration to Automated schema Synthetic deployment and instance data Planning and Monitoring pipeline Fully automated migrations Execution of Artefact repository “single-click” Automated multiple/dependent for releasable deployments functional testing on applications binaries all devices (runbook) Collaboration/Communication Continuous Delivery Challenges Multiple Hybrid Release on Large Teams Scale Applications Enterprise Demand Governance, Sarbanes-Oxley, Segregation of Information PCI-DSS, HIPAA Risk and Duties Security, Audit Compliance … Technology .NET, Java etc. Mainframe Mobile Integration to Integration to Core Systems, third-party Dependencies Back Office providers Systems Automated Dependencies Infrastructure Provisioning Culture Silos Scaled Agile Framework (SAFe)? Continuous Delivery & Release Management Release Release Train Planning Stabilization Release (recurring frequency, i.e. every month/quarter) Deploy Deploy Deploy (Internal Release) (Internal Release) (Internal Release) App A R D I V D R D I V D R D I V D Deploy Deploy Deploy (Internal Release) (Internal Release) (Internal Release) App B R D I V D R D I V D R D I V D Deploy Deploy Deploy Deploy Deploy Deploy App C R D I V R D I V R D I V R D I V R D I V R D I V … Automate for All Technologies Type Examples Automation Process Flat File HTML, PHP File Transfer Scripted (direct) Shell, Batch File Transfer, Execute Directly Scripted (command) SQL,MQ,Forms File Transfer, Incremental, Adhoc Execution Installer MSI,RPM,DAC File Transfer, Execute via Command Extract from Environment A, Transfer and Load into Environment A via Extract and Load Siebel, SAP, Informatica Commands WARs,EARs Application Archive File Transfer, Deploy into App Server via APIs (WebSphere,JBoss) Container Docker Pull Images from Tag, Stop/Start Container via Commands Configuration Property Files File Transfer, Replace Tokens Batch Jobs Mainframe Cobol Promote, Schedule Execution Adhoc Deployment Manifest Example <?xml version="1.0" encoding="ISO-8859-1"?> <release> Release details with “deployment” and <name>Adhoc Deployment Example</name> “rollback” sections <description>Release file for System A</description> <system>System A</system> type=script|sql|file <subsystem>Customer Portal</subsystem> Determines type of file and what to <date>24/06/2017</date> automate <version>1.2</version> <deployment> Version=[SCM Version] Input manually or automatically <files> from SCM <file name="somefile.sh" type="script" version="1.6“ source="somedir/bin" destination="$SOME_HOME/bin" copy="true" chmod="755" execute="true"/> copy=true|false … Copy file from source to </files> destination, chmod if necessary </deployment> <rollback> Execute=true|false Execute the file automatically … </rollback> </release> Decouple Release from Deployment •New versions deployed onto a limited set of Rolling servers to see how they perform •Typically load balancer points at multiple current Deployments versions and one instance of new “canary” release •Running versions of your app in “blue” production Blue-Green environment •New versions deployed to “green” environment, Deployment switched over (via “load balancer” on successful deployment, test …. Configuration •No code deployment, only make changes to Updates configuration files (debugging / feature flags) •Turn already deployed features on/off through Feature Flags updates in configuration Implement immutable infrastructure . Next level on from Blue-Green deployment . Deploy onto existing infrastructure! . Programmatically spin up new servers for each new application deployment . Guarantees validity / security of infrastructure and remediates configuration drift . Really needs Infrastructure as a Service platform. Can similarly be implemented using Update Instantiate Secure Base Image with (Run) New containerization (Docker) – but base image Image new App Image needs to be secure. Harden Continuous Delivery infrastructure Deployment Store and bound manage credentials deployable securely stored artefacts Cloud Environments Artefact Repository Database Server Credential Store (DAl/Third Party) (Oracle, SQL Server) (DA/HashiCorp Vault) Agents CD Server needs DB for Unidirectional Secure Communication audit trail JMS/7918 Deployment Mutual through Agents, HTTPS/8443 Authentication Relays, SSH Agent Relay(s) Continuous Delivery Agents Version Server Source Control Build Tools Control Dimension CM everything SVN/Git Jenkins SSH/22 Agents Manage and TeamCity lockdown