Welcome

THE EA’S GUIDE TO RELIABLE CLOUD DEPLOYMENTS

When development starts moving into the cloud, it’s understandable to want to move fast and see results quickly. Enterprises are often encouraged to deliver those first few deployments at pace, sometimes without thinking about longevity.

This can cause enterprises to struggle with a lack of comparability between projects. This inhibits your ability to effectively measure the success of your deployments and can cost your organisation a lot of time and resources.

At Dootrix, we believe it’s essential to tackle standardisation as soon as possible. By having reliable infrastructure deployments in place, you can achieve cost savings in a number of areas. Once you’ve got deployment standards in place, new projects require less work. But by encoding security practices into your deployments, you’re reducing mistakes and the opportunity for human error as well as security risks. With standardisation, comes clearer and better accounting - understanding where your costs lie and what’s driving them becomes much easier.

But how do you approach standardisation?

In this eBook, we break down our strategy in easy steps that will help you create high quality, standardised, repeatable and reliable infrastructure deployments and accelerate your journey into the cloud.

We hope this eBook is useful, please don’t hesitate to get in touch if you have any questions or would like to find out more!

Mark Vallins, Head of DevOps

PAGE 2 OF 30 CONTENTS

1 SETTING THE SCENE 2 INFRASTRUCTURE AS CODE 3 AGILE 4 VERSION CONTROL 5 CONVENTIONS 6 STANDARDS 7 TESTING 8 AUTOMATION

9 DESTRUCTIVE CHANGES

10 CONCLUSION PAGE 3 OF 30 Setting the scene

WHAT DO WE WANT TO ACHIEVE?

We want our cloud deployments to be:

Repeatable Reliable Standardised Secure

Not only do we want our code deployments to meet these requirements, we also want our infrastructure deployments to meet them. However, in reality we often find that we deviate from this vision.

The stresses of deadlines and a fear of change can back us into a corner, leaving us managing deployments by hand in a piecemeal manner, we therefor lose faith in our ability to manage change effectively.

We tend to have code deployments ‘wrapped-up’, we follow Agile processes and we implement continuous integration and continuous delivery using deployment tools like Azure DevOps.

We should be doing the same for our infrastructure, in many cases the tools and processes that we already use can be put to use again.

In short, to achieve the desired cloud deployments, we want:

A process that allows us to treat infrastructure the same way as code Infrastructure definitions under version control Infrastructure definition conventions Infrastructure standards and patterns Infrastructure testing Automated

PAGE 4 OF 30 Setting the scene

WHAT DO WE WANT TO ACHIEVE?

The processes described here are especially suited to Platform as a Service (PaaS) deployments where configuration is naturally exposed as properties on resources.

Ask yourself this question: Can you confidently expect to be able to tear down your infrastructure and re-build it from scratch, automatically and without issue?

For any environment, especially development environments, you really should be able to reset (delete and start-again) that environment at any point to avoid cumulative errors from experiments or other updates that cause the environment to deviate from the design.

Destroy Deploy

PAGE 5 OF 30 2: INFRASTRUCTURE AS CODE

PAGE 6 OF 30 Infrastructure as Code

WHAT DO WE WANT TO ACHIEVE?

Infrastructure as code is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools.1.

From a PaaS perspective perhaps we should re-phrase this in Platform as code terms as:

the process of managing and provisioning platform resources through machine-readable definition files rather than interactive configuration tools.

A number of infrastructure as code definition formats are supported across the cloud platforms:

LANGUAGE FORMAT PLATFORM

CloudFormation JSON, YAML Amazon Web Services (AWS)

ARM Templates JSON Microsoft Azure

Terraform HCL, JSON AWS, Microsoft Azure, Google Cloud Platform

Javascript, Typescript, Pulumi AWS, Microsoft Azure, Google Cloud Python 3, Go Platform

1. Wikipedia

PAGE 7 OF 30 ARM Template Example

Each of these formats allows you to define your infrastructure in a file or set of files that can be shared, reviewed and used as part of an automated deployment process (just like your source code).

Here’s a simple example of an Azure Resource Manager Template that describes an Azure storage account.

This example is self-contained and minimal, but I don’t recommend you write your templates in this way (e.g. inline values for names and SKUs) for reasons that we will discuss later).

{ "$schema": "https://schema.management.azure.com/schemas/ 2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": {}, "variables": {}, "resources": [ { "type": "Microsoft.Storage/storageAccounts", "apiVersion": "2018-07-01", "name": "examplestorageaccount", "location": "[resourceGroup().location]", "tags": { "displayName": "examplestorageaccount" }, "sku": { "name": "Standard_LRS" }, "kind": "StorageV2", "properties": {} } ], "outputs": {} }

PAGE 8 OF 30 3: AGILE

PAGE 9 OF 30 Agile

You may be working to an Agile process, but even if you aren’t you probably manage your code requirements in a tool like JIRA or Azure DevOps. You should be managing your infrastructure code in the same way.

1. Define the infrastructure requirements 2. Break down and refine those requirements 3. Assign tasks to individuals 4. Track infrastructure commits against stories/tasks 5. Review changes 6. Test changes 7. Merge 8. Deploy 9. Repeat

Breaking down infrastructure into separate tasks to allow for parallel development is made easier when the infrastructure definition isn’t one monolith of code, modularising the definition is critical for collaboration and re-use, we’ll cover this soon.

“YOU SHOULD BE MANAGING YOUR INFRASTRUCTURE CODE IN THE SAME WAY AS YOUR SOFTWARE CODE”

PAGE 10 OF 30 4: VERSION CONTROL

PAGE 11 OF 30 Version Control

A number of popular version control tools and services are available for managing your definitions locally and remotely.

Use the same platform that your team members use for code, choose a well-known platform if you don’t have one already.

Git might have a steep learning curve but it is ubiquitous and there is plenty of documentation available.

Make sure you get your infrastructure definitions under version control, the benefits are obvious:

History Recovery Backup Collaboration Understanding

Infrastructure definitions under version control can participate in the same processes as your code, you can organise changes in branches, peer review definitions and merge changes to development and production branches via pull requests.

Your team have oversight of the infrastructure deployment Changes can be reviewed and approved

PAGE 12 OF 30 HISTORY The genesis and modifications of your infrastructure definitions are tracked, history enables the related concepts of recovery and understanding. RECOVERY View and recover any version, want to know what’s changed in the last 7 days, that’s easy. Roll-back mistakes and deployments, broken something recently but not sure when, roll-back to a version that you know worked last week.

BACKUP Local and remote copies of infrastructure definitions are available, whether working with a distributed version control system or a centralised one, there are always multiple copies of the infrastructure definition should you lose your local copy.

COLLABORATION Changes can come from multiple sources. Changes are merged in a controlled fashion. Others can view and review.

UNDERSTANDING A freedom to experiment, access to history and collaboration help understanding. The ability of others to view the current and previous states of the infrastructure definition help provide context to the decisions and intention of the current state.

INFRASTRUCTURE DEFINITIONS UNDER VERSION CONTROL CAN PARTICIPATE IN THE SAME PROCESSES AS YOUR CODE. YOUR TEAM HAVE OVERSIGHT OF THE INFRASTRUCTURE DEPLOYMENT AND CHANGES CAN BE REVIEWED AND APPROVED.

PAGE 13 OF 30 5: CONVENTIONS

PAGE 14 OF 30 Conventions

You have coding conventions, coding standards, don’t you? You should have infrastructure definition conventions too, ask yourself the following questions:

File names Are file names predictable and well-defined? Can new team members navigate the architecture?

File structure How are your definition files arranged? Where are support scripts to be found? Are there standard locations for modular resource definitions?

Resource names Can team members recognise the type and purpose of a resource? Can team members recognise the project that a resource belongs to? Can team members recognise who owns the resource? Do resource names naturally avoid naming conflicts across projects and regions?

File layout Is the file consistently laid out? Are layout conventions followed? Are constraints correctly expressed? Are inputs validated and cleaned?

Extraneous conventions Specific to a particular infrastructure language, custom functions or DSLs for example.

If you have answers for most of these then you probably have a standard approach to writing and storing infrastructure definitions, are those conventions published and available to all team members?

Does your chosen platform allow you to enforce your conventions? If so, make use of it to ensure that all current and future deployments gain the benefit of convention.

PAGE 15 OF 30 6: STANDARDS

PAGE 16 OF 30 Standards

Do you find that your team members are writing the same resource definitions again and again? How do you know that each instance is correct or that they all follow internal guidelines?

Rather than rely on published resource guidelines think about enforcing resource definitions using a modular approach.

If your infrastructure language supports multiple files per architecture (it really should) then you can begin to create a library of components adhering to your internal standards that will help the whole team move faster and with fewer mistakes.

Enforce naming conventions Make sure that new resources are automatically assigned names that are compliant to your organisation’s policies (you might automatically concatenate values like region, project and owner into a standard name format).

Encoding shared functions Repeatable custom functions defined within the DSL of the infrastructure definition format can be written once and shared with other files to help make tasks easier (for example,

Does the platform support enforcement of standards? If so, make use of it to ensure that all current and future deployments gain the benefits of standardisation.

Standards help define and enable: Best security practice Resource discovery Cost control Reduction in configuration errors

PAGE 17 OF 30 RESOURCE CONFIGURATION & PATTERNS

Resource configuration Modularisation helps manage standards configuration for resources should be set in modular resource and support files, allowing team members to concentrate on how resources are connected rather than how they are configured. Use the capabilities of your chosen platform tools to define self-contained, pre-configured resources.

Set optional values on resources to preferred defaults for your team and organisation, reduce choice when creating new instances of the same resource. Set security options to strong defaults to prevent future issues later down the line. Manage resource options through built-in mechanisms for generating compliant names and relationships.

Ultimately, create a bucket of common components, commonly configured that can be used as architecture building blocks.

Patterns Use a library of standardised resources together to create standard patterns for the purposes of security or functionality. Create patterns and deployments that can be customised through unique pattern parameters.

For example, if you regularly create input queues and processing functions in your architectures, why not set this up as a common pattern?

Bucket of Standard Patterns Components

PAGE 18 OF 30 7: TESTING

PAGE 19 OF 30 Testing

It is important to test your infrastructure definition files’ static and dynamic properties:

Definition testing Deployment testing

The infrastructure definition files should be regularly (ideally automatically) tested as part of your development process, make testing an acceptance criteria for your pull requests.

Definitions should be tested for: Well-formed Valid Definition Security

Well-formed Is the file correctly structured and formatted? For example, is a JSON file a valid JSON file? This doesn’t test the correctness of the domain-specific data but does at least tell you that the file format and structure is correct.

Ideally, the tools that you use to create and edit your infrastructure definitions will recognise and support the file format.

However, it is always worth a second automated check-in a build pipeline as you can’t always guarantee that files are edited in a recommended environment.

Valid definition Are the domain-specific aspects of the file correct? Is the data within the file in the correct format with the expected sections and labels for the platform format. You might consider extending this capability beyond that of the platform tools to include checks for your own conventions (e.g. naming).

Security Are the correct security patterns in place, are resources secure by default? This task is made easier when using a standard library of resource definitions. Some platform definitions can be security checked using third-party tools.

PAGE 20 OF 30 RESOURCE CONFIGURATION & PATTERNS

Post Deployment Validation

Continually validate deployed cloud resources and configuration against the infrastructure definition. Such testing may be performed regularly against a deployed system to ensure that the state does not drift away from the desired configuration. Your chosen platform may use other terms for this, but it’s a useful technique that ensures that your infrastructure definition is not only used as the blueprint for deployment but may also be used as a test and enforcement tool.

Some forms of desired configuration testing may also support automated correction when configuration drifts, this is something that you may wish consider.

“YOU SHOULD CONTINUALLY VALIDATE DEPLOYED CODE RESOURCES AND CONFIGURATION AGAINST THE INFRASTRUCTURE DEFINITION”

PAGE 21 OF 30 8: AUTOMATION

PAGE 22 OF 30 Automation

I’m going to assume that you are using a version control system, you are right? But are you taking advantage of that to automate much of the rest of the process? When you make changes to your definition are you doing any of the following?

Deploying without having someone else review changes Deploying from any branch Downloading a copy of the definition to your local workstation and deploying from there Skipping any level of testing

If you are you will eventually run into issues like:

Not knowing which version of the definition the last deployment was made from Not knowing who made the last set of changes Not allowing other team members to deploy confidently Failing to document the deployment process

PAGE 23 OF 30 CONTINUOUS INTEGRATION / CONTINUOUS DELIVERY

Consider a Continuous Integration / Continuous Delivery (CI/CD) tool to automate your process as much as possible, allowing you to: 1. Commit a definition change to a new version control branch. 2. Initiate a pull-request for that change. 3. Have definition file tests run automatically on the code as part of the pull-request trigger. 4. Ensure that someone else views and approves the changes. 5. On approval see the change merged into a deployable branch. 6. On merge have a deployment process automatically triggered. 7. The deployment process automatically tests the definition files before running them - this is a gated process requiring success to continue. 8. The deployment process uses the platform’s tools to deploy or update resources in a target environment. 9. Successful deployment results in notifications to the team and optionally triggers further processes for code deployment or environment validation.

PAGE 24 OF 30 9: DESTRUCTIVE CHANGES

PAGE 25 OF 30 Destructive Changes

Cloud platforms may support multiple deployment modes for resources, which you use depends on your environment and intent.

INCREMENTAL DEPLOYMENT Incremental deployment allows you to update existing and add new resources to a deployment, it generally won’t delete any existing resources and is probably the default mode that you will want to adopt for production resources. You may find that you will have to validate and potentially clean-up the resources by hand after an incremental update - where an old resource may no longer be required, for example.

DESTRUCTIVE DEPLOYMENT Each time the infrastructure definition files are used for destructive deployment the environment will contain only those resources that are in the definition and configuration will be reset on those resources. This mode of deployment is useful for resetting development and test environments where you want to be sure that the infrastructure depends only on those resources currently in the definition file, you will probably follow this kind of deployment with a fresh code deployment - as existing content will have been wiped when old resources were deleted.

DESTRUCTIVE DEPLOYMENT IS USEFUL FOR RESETTING DEVELOPMENT AND TEST ENVIRONMENTS WHERE YOU WANT TO BE SURE THAT THE INFRASTRUCTURE DEPENDS ONLY ON THE RESOURCES IN THE DEFINITION FILE.

PAGE 26 OF 30 10: CONCLUSION

PAGE 27 OF 30 WHAT’S NEXT

So, there you have it. The Enterprise Architect’s Guide to Reliable Cloud Deployments. The companies that manage to standardise effectively are able to benefit from reliable, repeatable and secure deployments that help drive their businesses forward.

In summary, improving the reliability, correctness, security and management of your infrastructure as code definition files may require you to think about and implement the following:

Treat infrastructure as code (IaC) development as a project Ensure that all source files are held under version control Use version control and project management tools to enable change management Define, document and apply standards and conventions to your definition files Consider your testing strategy and implement protections against errors Automate deployment validation and testing Understand and use relevant infrastructure deployment modes Apply these principles to your specific platform and tools

Prioritising deployment standards in the midst of digital transformation is a challenge for many reasons, but with the right approach, organisations can not only deliver better quality infrastructure deployments, but they can also reduce costs and accelerate their journey to the cloud.

We hope you found our insights valuable – that we’ve sparked some ideas, and that you get a chance to put them into practice soon.

PAGE 28 OF 30 dootrix.com/cloud

Where do we go from here? BOOK A FREE INFRASTRUCTURE REVIEW NOW!

During our infrastructure review, our expert team will work closely with your technical and business stakeholders to help build a customised roadmap to accelerate your journey into the cloud.

Our team of digital activation specialists and software developers will assess either solution or enterprise architecture to help you build an effective strategy and deliver a high-level backlog to ensure the most effective approach.

Half day one: immersion with stakeholders to understand the ambitions, challenges and commercial context

Half day two: architecture review and completion of a full written report

You will come away with a high quality roadmap to activate your own digital transformation Book now at [email protected]

PAGE 29 OF 30 Want to accelerate your digital transformation? Let’s talk

+44 (0) 2392 001 990 WWW.DOOTRIX.COM [email protected]

PAGE 30 OF 30