The EA's Guide to Reliable Infrastructure Deployments
Total Page:16
File Type:pdf, Size:1020Kb
Welcome THE EA’S GUIDE TO RELIABLE CLOUD DEPLOYMENTS When development starts moving into the cloud, it’s understandable to want to move fast and see results quickly. Enterprises are often encouraged to deliver those first few deployments at pace, sometimes without thinking about longevity. This can cause enterprises to struggle with a lack of comparability between projects. This inhibits your ability to effectively measure the success of your deployments and can cost your organisation a lot of time and resources. At Dootrix, we believe it’s essential to tackle standardisation as soon as possible. By having reliable infrastructure deployments in place, you can achieve cost savings in a number of areas. Once you’ve got deployment standards in place, new projects require less work. But by encoding security practices into your deployments, you’re reducing mistakes and the opportunity for human error as well as security risks. With standardisation, comes clearer and better accounting - understanding where your costs lie and what’s driving them becomes much easier. But how do you approach standardisation? In this eBook, we break down our strategy in easy steps that will help you create high quality, standardised, repeatable and reliable infrastructure deployments and accelerate your journey into the cloud. We hope this eBook is useful, please don’t hesitate to get in touch if you have any questions or would like to find out more! Mark Vallins, Head of DevOps PAGE 2 OF 30 CONTENTS 1 SETTING THE SCENE 2 INFRASTRUCTURE AS CODE 3 AGILE 4 VERSION CONTROL 5 CONVENTIONS 6 STANDARDS 7 TESTING 8 AUTOMATION 9 DESTRUCTIVE CHANGES 10 CONCLUSION PAGE 3 OF 30 Setting the scene WHAT DO WE WANT TO ACHIEVE? We want our cloud deployments to be: Repeatable Reliable Standardised Secure Not only do we want our code deployments to meet these requirements, we also want our infrastructure deployments to meet them. However, in reality we often find that we deviate from this vision. The stresses of deadlines and a fear of change can back us into a corner, leaving us managing deployments by hand in a piecemeal manner, we therefor lose faith in our ability to manage change effectively. We tend to have code deployments ‘wrapped-up’, we follow Agile processes and we implement continuous integration and continuous delivery using deployment tools like Azure DevOps. We should be doing the same for our infrastructure, in many cases the tools and processes that we already use can be put to use again. In short, to achieve the desired cloud deployments, we want: A process that allows us to treat infrastructure the same way as code Infrastructure definitions under version control Infrastructure definition conventions Infrastructure standards and patterns Infrastructure testing Automated PAGE 4 OF 30 Setting the scene WHAT DO WE WANT TO ACHIEVE? The processes described here are especially suited to Platform as a Service (PaaS) deployments where configuration is naturally exposed as properties on resources. Ask yourself this question: Can you confidently expect to be able to tear down your infrastructure and re-build it from scratch, automatically and without issue? For any environment, especially development environments, you really should be able to reset (delete and start-again) that environment at any point to avoid cumulative errors from experiments or other updates that cause the environment to deviate from the design. Destroy Deploy PAGE 5 OF 30 2: INFRASTRUCTURE AS CODE PAGE 6 OF 30 Infrastructure as Code WHAT DO WE WANT TO ACHIEVE? Infrastructure as code is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools.1. From a PaaS perspective perhaps we should re-phrase this in Platform as code terms as: the process of managing and provisioning platform resources through machine-readable definition files rather than interactive configuration tools. A number of infrastructure as code definition formats are supported across the cloud platforms: LANGUAGE FORMAT PLATFORM CloudFormation JSON, YAML Amazon Web Services (AWS) ARM Templates JSON Microsoft Azure Terraform HCL, JSON AWS, Microsoft Azure, Google Cloud Platform Javascript, Typescript, Pulumi AWS, Microsoft Azure, Google Cloud Python 3, Go Platform 1. Wikipedia PAGE 7 OF 30 ARM Template Example Each of these formats allows you to define your infrastructure in a file or set of files that can be shared, reviewed and used as part of an automated deployment process (just like your source code). Here’s a simple example of an Azure Resource Manager Template that describes an Azure storage account. This example is self-contained and minimal, but I don’t recommend you write your templates in this way (e.g. inline values for names and SKUs) for reasons that we will discuss later). { "$schema": "https://schema.management.azure.com/schemas/ 2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": {}, "variables": {}, "resources": [ { "type": "Microsoft.Storage/storageAccounts", "apiVersion": "2018-07-01", "name": "examplestorageaccount", "location": "[resourceGroup().location]", "tags": { "displayName": "examplestorageaccount" }, "sku": { "name": "Standard_LRS" }, "kind": "StorageV2", "properties": {} } ], "outputs": {} } PAGE 8 OF 30 3: AGILE PAGE 9 OF 30 Agile You may be working to an Agile process, but even if you aren’t you probably manage your code requirements in a tool like JIRA or Azure DevOps. You should be managing your infrastructure code in the same way. 1. Define the infrastructure requirements 2. Break down and refine those requirements 3. Assign tasks to individuals 4. Track infrastructure commits against stories/tasks 5. Review changes 6. Test changes 7. Merge 8. Deploy 9. Repeat Breaking down infrastructure into separate tasks to allow for parallel development is made easier when the infrastructure definition isn’t one monolith of code, modularising the definition is critical for collaboration and re-use, we’ll cover this soon. “YOU SHOULD BE MANAGING YOUR INFRASTRUCTURE CODE IN THE SAME WAY AS YOUR SOFTWARE CODE” PAGE 10 OF 30 4: VERSION CONTROL PAGE 11 OF 30 Version Control A number of popular version control tools and services are available for managing your definitions locally and remotely. Use the same platform that your team members use for code, choose a well-known platform if you don’t have one already. Git might have a steep learning curve but it is ubiquitous and there is plenty of documentation available. Make sure you get your infrastructure definitions under version control, the benefits are obvious: History Recovery Backup Collaboration Understanding Infrastructure definitions under version control can participate in the same processes as your code, you can organise changes in branches, peer review definitions and merge changes to development and production branches via pull requests. Your team have oversight of the infrastructure deployment Changes can be reviewed and approved PAGE 12 OF 30 HISTORY The genesis and modifications of your infrastructure definitions are tracked, history enables the related concepts of recovery and understanding. RECOVERY View and recover any version, want to know what’s changed in the last 7 days, that’s easy. Roll-back mistakes and deployments, broken something recently but not sure when, roll-back to a version that you know worked last week. BACKUP Local and remote copies of infrastructure definitions are available, whether working with a distributed version control system or a centralised one, there are always multiple copies of the infrastructure definition should you lose your local copy. COLLABORATION Changes can come from multiple sources. Changes are merged in a controlled fashion. Others can view and review. UNDERSTANDING A freedom to experiment, access to history and collaboration help understanding. The ability of others to view the current and previous states of the infrastructure definition help provide context to the decisions and intention of the current state. INFRASTRUCTURE DEFINITIONS UNDER VERSION CONTROL CAN PARTICIPATE IN THE SAME PROCESSES AS YOUR CODE. YOUR TEAM HAVE OVERSIGHT OF THE INFRASTRUCTURE DEPLOYMENT AND CHANGES CAN BE REVIEWED AND APPROVED. PAGE 13 OF 30 5: CONVENTIONS PAGE 14 OF 30 Conventions You have coding conventions, coding standards, don’t you? You should have infrastructure definition conventions too, ask yourself the following questions: File names Are file names predictable and well-defined? Can new team members navigate the architecture? File structure How are your definition files arranged? Where are support scripts to be found? Are there standard locations for modular resource definitions? Resource names Can team members recognise the type and purpose of a resource? Can team members recognise the project that a resource belongs to? Can team members recognise who owns the resource? Do resource names naturally avoid naming conflicts across projects and regions? File layout Is the file consistently laid out? Are layout conventions followed? Are constraints correctly expressed? Are inputs validated and cleaned? Extraneous conventions Specific to a particular infrastructure language, custom functions or DSLs for example. If you have answers for most of these then you probably have a standard approach to writing and storing infrastructure definitions, are those conventions published and available to all team members? Does your chosen platform allow you to enforce your conventions? If so, make use of it to ensure that all current and future deployments gain the benefit of convention. PAGE 15 OF 30 6: STANDARDS PAGE 16 OF 30 Standards Do you find that your team members are writing the same resource definitions again and again? How do you know that each instance is correct or that they all follow internal guidelines? Rather than rely on published resource guidelines think about enforcing resource definitions using a modular approach. If your infrastructure language supports multiple files per architecture (it really should) then you can begin to create a library of components adhering to your internal standards that will help the whole team move faster and with fewer mistakes.