Understanding Containers on AWS Containers on AWS and why they matter Understanding Containers on AWS Every so often there is a game changer in technology that completely disrupts how people operate. Over the last six years that game changer has been — in the past two years it has been containers (often interchangeably known as Docker).

Strictly speaking, containerization is not a new concept; as early as on , while machine learning specialists are actively 1979 the concept of containerization began with , which isolates contemplating how to generate and train their predictive models and restricts namespaces of a Unix process (and its children) to a new efficiently leveraging container orchestration tools. Containers have location in the file system. begun to underpin the evolution of the different distributed computing disciplines. Docker has offered a certain “ease of adoption” for containers in current technology. Around since 2014, Docker has been adopted into business While containers as a packaging solution has a strong impact on the strategy instead of being relegated to just a technological trend. modern “DevOps” culture, they’ve also gained popularity because they go hand-in-hand in the continuous evolution of software architecture. Why Containers Matter?

A conversation on containers usually starts with the technology, but quickly transforms to a conversation of change management, safe and scalable collaboration across different groups, community-driven initiatives and other disciplines that are normally kept separate. These days, data scientists are talking about deploying their data workloads

Every so often there is a game changer in technology that completely disrupts how people operate.

Understanding Containers on AWS | 2 Before Containers Monolithic applications were once the norm.

As servers were tedious and difficult to configure, it was worthwhile to reduce the pain by using as few servers as possible—combining front-end user interfaces, business logic, and backend services into one single package deployed to a single server. Every major upgrade translated directly to downtime. Servers were expensive both in terms of upfront investment and ongoing maintenance costs; less was more.

While fewer servers with larger installations looked reasonable on paper, as the complexity of the system increased and demands of new features hastened, the development and testing of such complex applications broke down and easily grinded to a halt. The more complex the system was the more time is required on testing. Releases became slower and larger in scope, with more features slotted in for fear of “missing the boat”, and production rollout became a multi-hour/day affair.

These methods were unsustainable. As Internet-based business became more prominent, users expectations meant that the haphazard way of massive deployment needs to be completely eradicated. The “” paradigm was introduced to address this problem. The complex installation of all features every release was broken down into independent releases of much smaller components. As long as the agreed upon interfaces did not change, it became the prerogative of individual component owners to update and change the logic as seen fit instead of needing to worry about every single test case of in the test suite.

Understanding Containers on AWS | 3 Microservices and Virtual Machines

While microservices were a good response to these complex applications, it was not free from challenges. For example, with each microservice on an independent release cycle having slightly different dependencies or running on different versions of an , it was necessary to provision dedicated servers for each of the services. By breaking the monolithic applications to microservices, the number of servers would exponentially increase from tens to hundreds, and the monolithic days exponentially to tens or hundreds.

Better Internet connectivity also meant users and systems were no longer siloed. Peak usage no longer meant hundreds or more calls within the hour, but within a minute. The tens or hundreds of servers to serve the full collection of microservices now had to be increased to hundreds and thousands.

The rise of virtual machines helped solve the problem. Sitting on top of hardware, the is able to create and run one or more virtual machines on the same hardware. Each of the virtual machines can operate independently on the shared resources, and can be configured using configuration management tools. This increases the number of servers on the same hardware, therefore more microservices can run on the same hardware.

Running microservices on virtual machines is not a perfect solution to every use case. Managing a fleet of VMs and introduces its own set of overhead challenges around managing load, machine density, horizontal and vertical scaling, as well as configuration drift and OS maintenance. Configuration management tools such as Chef, Puppet, and coupled with platforms such as OpsWorks and AWS SSM can often eliminate or greatly reduce these challenges and for many use cases. For some applications, this overhead is a fair trade for the flexibility required, especially for large but still-evolving enterprise applications. However, as applications evolve further towards the segmentation of microservices, this balance between management overhead and hosting flexibility can skew unfavorably. For developers focused on maintaining a large set of many smaller systems, a new solution was required.

Understanding Containers on AWS | 4 Why do containers work well with the cloud? One prominent feature of Containers and Microservices cloud computing is elasticity: cloud infrastructure can scale up and down based on demand. Raising a new to service the For many, that solution was Docker. demand is fine, but it takes minutes, which can result in significant loss in business. Scaling up containers takes seconds 1 instead of minutes, Docker represents a user-friendly container deployment methodology. meeting scaling demands much more efficiently. The speediness of By adopting Docker, multiple applications can run on the same virtual container deployment aligns with the requirement of the cloud that machine/bare metal server. Since Docker packages all of an demands rapid changes. application's dependencies within a single image, conflicting dependencies between different services can exist. As long as the Because containers run on an abstraction layer on top of virtual services share the same kernel as the host machine, the different Docker machines, it is further separated from the underlying compute resources. processes run harmoniously with one another. Now the hundreds and A Docker image that can run on-premise can also run on AWS and other thousands of machines can drop drastically back while not sacrificing environments. Since cloud transcends physical locations and service the release independence and integrity of the applications. providers, containers work well with the cloud. Cloud native architectural patterns state that scaling horizontally (more servers) is preferable to Another advantage of containers is immutability and therefore scaling vertically (more powerful servers). Docker provides a way to run consistency. Upgrading a containerized application is equivalent to the applications across different servers easily. Because containers are stopping an existing process and starting a new one based on a newer immutable, it poses less operational costs with less margin of error. As image. This removes the potential drift in configurations. The removal of there are increasing demands on hybrid cloud computing for disaster configuration ambiguities also helps introduce a more streamlined recovery and high availability purpose, a deployment mechanism that process. Since the dependencies are already packaged within the promises working across the different physical environments is certainly container image, the overhead is drastically reduced. This is analogous very attractive. to compiled binary applications where all dependencies are encapsulated at build time. Container versus Container Orchestration While running one container is indeed easy, managing a number of Containers and Cloud-Nativeness containers - or generally what is known as “container orchestration” Go Hand-in-Hand - is a lot more complex.

Another movement that helped promote adoption of containers is Container orchestration can be a heavy operational overhead. Unless cloud computing. With the advent of cloud, “cloud-native” technologies your core business is managing container processes, mastering the became the building blocks for developing cloud-based applications. managing and orchestrating of containers often does not increase the Containers are an enabling technology, facilitating encapsulation of bottom line. The good news is that because of the popularity of independent services, fully utilized compute infrastructure, scalability, containers, a number of people have been trying to solve these and rapid development. problems, and services like AWS have been introducing a variety of 1 Because of the overlay file system, which supports inter-application and inter-version sharing solutions to help orchestrate containers.

Understanding Containers on AWS | 5 Cross-Provider Containers on AWS All-in AWS Services (incl. On Premise)

Amazon Elastic Complex While it has always been possible to run containers directly on Amazon EC2 Container Kubernetes on EC2 Requirements instances, AWS recognised the need to relieve users from undifferentiated Services (ECS) operating activities, allowing businesses to focus on their core products and services, as opposed to focusing on mastering container orchestration. Key Advantages: Key Advantages: There are a number of managed container orchestration services on AWS such as Support for Windows Users retain full control and containers configurations of the clusters Amazon EKS (Elastic Container Services for Kubernetes) and AWS Fargate, both of Dedicated ENI for tasks Very configurable to support which were announced during re:Invent 2017. Along with AWS Elastic Container for network segregation, highly diversified workload while more traditional Service (ECS), these services maximize coverage and cover the diverse needs of docker0 and host based Leverages the robust, AWS users. networking still supported scalable and highly available AWS services Support for spot instances and spot fleet Active and rapidly growing To the right is a summary diagram of the various container options available to community to support most Logging aggregation needs AWS customers. Each has its own specific advantages and is best suited for with CloudWatch logs particular use cases. Together, they serve the full spectrum of container needs on natively supported AWS and help prove AWS’ commitment to the wide assortment of container approaches being adopted by customers. Amazon Elastic Simpler AWS Fargate Kubernetes Service Requirements Amazon Elastic Container Service(ECS): (EKS) Born and Raised in AWS Key Advantages: Key Advantages: Zero scaling and Best way to run K8s workload on The first managed container services on AWS was ECS, which was first announced availability management AWS: Provider agnostic while in 2014 but has since gone through many changes. for container workflow engineered for AWS Fits well into event-driven Best practice for K8s on AWS & scheduled workloads without the operational Elastic Container Service provides a rich experience for running containers. A Excellent choice for overhead ad-hoc / temporary product that is born and raised in AWS, ECS has completely embedded the DNA of scenarios such as dynamic No rework for Dev & Test systems application/service succinct integration with other AWS services. Logging needs can be answered by deployments already running Well-suited to production on other infrastructure providers the awslog option that points straight to CloudWatch Logs, autoscaling and deployments of simple applications where Automatic platform patching monitoring are concisely integrated with CloudWatch, while permissions and management overhead and easy upgrades of other solutions would outweigh benefit High scalability and availability Runs in AWS accounts (not with minimal costs user accounts) to further minimise infrastructure management security are heavily backed by IAM and security groups. Given Fargate is by far the simplest way to start running containers on AWS, as it container instances are specialized EC2 instances, any needs for spot poses managing overhead on scaling the containers and the underlying instances and spot fleets are also supported by ECS. On top of that, if instances. The open question is if there are specific workloads best suited you’re looking for a CI/CD experience within AWS, Elastic Beanstalk for Fargate. Lambda is best suited for event-driven or scheduled short can be used to manage the ECS workloads easily. duration workloads. Given the price point and features currently supported, Fargate appears to be following the same pattern of serving ECS is well integrated with the rest of the AWS ecosystem. If your scheduled and short term workload. workload is fully dependent on AWS native services, ECS can be a great tool for further ease and simplicity. Amazon Elastic Container Service AWS Fargate: the Marriage of for Kubernetes (EKS): Easily Operating Serverless and Containers Kubernetes on AWS

AWS Fargate was announced as the container orchestration tool with Originating from an internal container platform at named Borg, no management. Currently supported for the Elastic Container Service Kubernetes has grown into a widely-adopted and growing ecosystem (ECS) (while EKS support is pending), Fargate provisions and configures that has won the support of a vast community of engineers. It is so compute and network infrastructure while developers simply specify popular that all of the cloud providers on the Gartner magic quadrant in CPU and memory requirements. 2018 2 provide different levels of support to it.

In a nutshell, Fargate is serverless deployment for containers. Instead of Running Kubernetes on AWS is attractive for numerous reasons. Many developing function code in one of the supported languages like when people choose Kubernetes for is its open architecture: because of the running Lambda functions, you can now just run the image directly in a pluggable provider interface, Kubernetes can run both on premise and serverless fashion. Existing ECS task definitions can be reused to set up across different cloud providers, making the application management Fargate workloads. The Fargate workloads will run the containers driven code reusable regardless of the underlying infrastructure. It is a by events such as schedules, event-driven patterns, or feature-rich container orchestration platform that supports widely automation-driven instantiation for on-demand workloads. The user is different workloads ranging from highly fluctuating stateless services to liberated from the overhead associated with managing the entire containerized persistent backends and tools supporting such workloads. container hosting platform aside from ensuring the sizing and access Furthermore, it fully aligns security paradigms 3 to create a highly requirements of the container itself. The containers run in VPCs in AWS scalable yet secure container deployment experience. accounts, while users can set up network load-balancers and application load balancers pointing directly to the target group consisting of the Fargate containers in order to run a load-balanced workload. 2 https://www.gartner.com/doc/reprints?ct=150519&id=1-2G2O5FC 3 Namespaces, , linux capabilities

Understanding Containers on AWS | 7 While Kubernetes has internal support for container scaling, service giving the team more time to focus on developing applications discovery and config management within the platform; running and services, and thereby reducing mean time to market of the Kubernetes on AWS allows the platform to leverage some key AWS differentiating products and services. Meanwhile, application services such as cluster autoscaler integration with autoscaling groups, developers and operators can still retain full control on the nodes Amazon Elastic Load Balancer (ELB), Amazon Elastic Block Store (EBS) where the containers are run. volumes, Amazon Route53, Parameter Store/Secrets Manager. For an organization that is already operating on AWS, many of the existing Also with the official support on AWS, the plugins running on the platform operating paradigms and provisioning tools can be extended to also run are more closely aligned with the AWS core design. Traffic across pods Kubernetes with relative ease. Running Kubernetes on AWS is an on different worker nodes closely follow the VPC network and security opportunity to take advantage of the feature-rich container platform model, making the process very efficient. Logging and analytics are and robust scalable on-demand infrastructure. tightly integrated with AWS platform tools such as Amazon CloudWatch. Plus, certain industry-specific compliance requirements are resolved by Many organisations have adopted Kubernetes, however engineers are using EKS. Currently EKS is HIPAA-eligible as well as ISO and PCI-DSS Level 1. often flummoxed by the steep learning curve. While tools such as Kops and Kubeadm can help ease the creation and upgrade of the By becoming an active contributor to the Kubernetes project, AWS has Kubernetes clusters on AWS, configuring the master nodes, addressed potential concerns that EKS is going to diverge from troubleshooting network errors, managing etcd backup/restoration, and Kubernetes development. The releases onto EKS follow the official performing other control plane related tasks takes a lot of education. Kubernetes releases very closely. If a workload runs on Kubernetes on Engineers can quickly find themselves descending into a rabbit hole if premise on a version that is supported by EKS, it is expected to run they adopt Kubernetes without fully appreciating the complexity of smoothly on EKS too. maintaining a highly available Kubernetes control plane. Many engineers working on Kubernetes often spend too much time reading documentation and user blogs, working through tutorials and source code and catching up on the latest messages on Kubernetes GitHub issues and Slack channels. This coupled with the rapidly changing nature of the Kubernetes platform with the multitude of features, enhancements and add-ons make navigating in the Kubernetes world even more difficult.

That is why Amazon EKS is a positive tool for users who only have conventional orchestration needs. With the managed Kubernetes control plane, the indispensable – and often most challenging – pieces of the Kubernetes cluster 4 is in the hands of the AWS managed services,

4 More exactly, etcd, kube-apiserver, kube-scheduler, the controllers and a number of addons that need to run on the masters

Understanding Containers on AWS | 8 anything that requires setting up specific features of the cluster by Amazon EC2-native Workloads: configuring parameters into the Kube API server, Kubernetes on EC2 Specific Requirements Need is still the optimal solution. Specific Tools As technologies progress, what is the standard today may well shift because of evolving needs. What container workloads on EC2 provides Running container workloads on EC2 instances is analogous to running is the fundamental basis that the workload will function on AWS. It may databases on AWS using the example of a SQL Server database. If you be more management overhead or more flexibility in configuration—the need a SQL Server database that fits into the operating parameter of key is it’s supported and it can be done. RDS SQL server, you should leverage RDS; but if you have specific requirements for SQL Server Analysis or Reporting services, the ideal solution is still running the SQL server on EC2 instances. It is the usual Containers on Windows balance between control and delegation. There are more limitations with managed services, while the user gains peace of mind; conversely, Modern containerization is largely an evolution of isolation approaches with flexibility the user has to assume more operational responsibility. The that emerged from various Unix-like operating systems. As such, it can be same logic applies to running container workloads on EC2 directly, be it a common (though decreasingly so) misconception that through Kubernetes or other container orchestration platforms. containerization in the cloud is restricted to Linux environments. Sensing the importance of technology, began experimenting with When working on a greenfield project, it is easy to decide between the containerization with the early versions of 2016. most popular choices i.e. ECS or EKS; however, there are considerably more constraints when managing a publicly released product that Throughout the 2016 lifecycle, and especially with the late 2018 release serves critical workloads and has service level agreement obligations. of Windows Server 2019, customers heavily invested in the Windows Before Kubernetes becomes arguably the cross-platform standard, and/or .NET ecosystems can enjoy robust first-class support of workloads have been deployed onto Mesosphere (or DC/OS), Docker containerization. In supported versions of Windows Server (and Windows Swarm, OpenShift and Racher for a variety of reasons such as add-on 10), Docker containers based on Windows Server images are supported support and hybrid deployment requirements. Each of these solutions in much the same way as they are on Linux systems. Windows offers a has its merits and shortcomings. Philosophical debates aside, production similar process-level isolation, called "Windows Server Containers" where workloads still need to be supported while migration decisions to either containers share the kernel of the underlying host, with other resources EKS or ECS are being made. That means running those container abstracted through the container runtime. This gives Windows containers workloads on EC2. the same behavior and management patterns through Docker that are available in Linux, and allows Windows containers to enjoy a vast array Even in the case of Kubernetes, if the non-descript Kubernetes cluster of hosting and management options in the cloud. works for you, the most effective way to run is using EKS. If you have specific compliance, performance, or management requirements,

Understanding Containers on AWS | 9 Microsoft did take the evolution one step further by offering a second As Windows containers are first-class citizens in the Docker/container isolation option, known as Hyper-V isolation, which runs containers ecosystem, many of the same options we've already discussed are individually in an optimized VM preventing the need to share the OS available. Similar to Linux, when specialized workloads require it, a kernel, and offering VM-level security protections to a container that Docker system can be built on top of an EC2 instance (or fleet of may need such isolation. Isolation model is a runtime feature, so the instances) directly. In the Windows world, this offers one unique item of containers themselves do not differ across models. We'll cover this in a flexibility. As mentioned before, Windows offer a 2nd level of isolation, little more detail, but the key takeaway is that Windows containers are known as Hyper-V isolation which provide VM levels of security for just Docker containers running Windows base images, and offer two containers. As its name suggests, this is a core feature of the hypervisor different isolation options, with Windows Server Containers (process and requires direct access to the hardware, which many cloud options isolation) being the most similar to the model Docker uses in Linux. abstract. However, using AWS EC2 Bare Metal instances, you can build your own Hyper-V nodes running Docker, and take advantage of With differing operating systems, there are a few minor considerations Hyper-V isolation in the cloud. This option also gives you the ability to run that need to be taken into account when deploying Windows Linux containers on Windows nodes, as Hyper-V isolation is providing a containers. The main limitation/restriction comes from the fact that complete VM and can therefore support native Linux workloads. This is a containerization is largely an abstraction offered by the OS. As such, highly specialized use case, but another great example of the flexibility containers are mostly limited to running on a host running the same OS of the AWS platform. architecture. Linux VMs need to run on Linux Hosts, and Windows VMs need to run on Windows hosts. Fortunately, many of the management & Amazon ECS has provided support for Windows containers since 2016, hosting platforms have made this consideration largely a solved issue now providing various OS options (2016, 2019) and optimized AMIs for a and handle the node OS on the backend, allowing containers to be streamlined deployment. Amazon ECS has a few caveats with Windows managed together. Windows container images can also be quite large containers, mostly aligning to the general restrictions described earlier. in size, due again to differences in OS architecture. This is improving, with Additionally, some of the AWS integration features provided by the host various versions of Windows Server being offered for this use case. In proxy software on the node may have limitations or operational Server 2019, there are 3 Windows versions - Server, Server Core, and Nano differences in a Windows environment. These caveats are constantly Server. In past versions, Core & Nano were somewhat similar, but in 2019, being evolved/re-visited, and the most accurate information can be Nano has been repositioned to be the clear best base image for found at https://docs.aws.amazon.com/AmazonECS/latest/ Windows containers. Core offers a UI-less streamlined server that is developerguide/ECS_Windows.html. perfect for use as a container host, and Server offers the traditional UI-based server experience Windows Server is known for. There are also At the time of this writing (early 2019), ECS Fargate does not support some additional platform restrictions or differences, covered briefly in the Windows containers. following sections. However, outside of this small list, containers on Windows provide the same benefits, usage model, and management options as their Linux brethren.

Understanding Containers on AWS | 10 Kubernetes introduced beta support for Windows containers in v1.9. With this, developers can continue to select "the best tool for the job", As of version 1.14, Windows container support is a production-level moving use cases that are better served by serverless approaches to feature. In Kubernetes, containers must run on a node with a matching those platforms and ensuring the container deployments are optimized OS. Beyond that limitation, all core features work the same as they do and modernized to best serve the applications that can best benefit with Linux containers, which minor implementation differences at from this technology. hardware level features such as storage and networking. Adding Windows containers to a Kubernetes cluster is as simple as adding a All Container Workloads are Windows-based node to the cluster. Kubernetes control nodes and master components must run on Linux, and the roadmap does not supported on AWS suggest Windows support for cluster management. At the time of this writing (early 2019) Amazon EKS supports Windows containers in As is the norm for AWS, every service in the portfolio serves a specific beta-level deployments, as EKS is currently running Kubernetes 1.12. workload. With the container orchestration services – Amazon EKS, AWS has pledged continued support for future versions of Kubernetes Amazon ECS and AWS Fargate – AWS is doubling down on making sure and features supported by those releases. that all containerized workloads are welcome on AWS. There are different services for different workflows, just pick the one that works One last item to consider when planning a Windows container strategy best for your solution. on AWS is to look at the applications themselves. For legacy Windows applications that don't have active development, or modern/evolving In the rapid evolution of containerized workflows there is a lot of applications heavy based on .NET Framework runtimes, Windows excitement and benefit. If you are curious about where to start or containers are a fantastic and flexible option. However, for applications how to evolve, check out these containers resources from AWS and leveraging .NET (but not Windows or Framework-only libraries), a lot Onica. To speak with an Onica representative, contact the Onica of customers are choosing .NET Core as the first step to their team at [email protected] or 1-833-664-2280. modernization plans. .NET core offers all the options we've covered here, but adds a few interesting new options in the cloud ecosystem. .NET core is platform independent and will run well on Linux-based containers. For some, this provides a great path towards better scaling options without the limitations of Windows Server licensing. Microsoft SQL server also now runs on Linux, which can further offer licensing independence where applicable. Also, .NET Core is a first-class language in AWS Lambda, the workhouse of the AWS Serverless platform. Modernizing to .NET Core on containers opens a path towards full serverless application design.

Understanding Containers on AWS | 11 © 2019, Onica, Inc. and , Inc. All rights reserved.