Simplifying Release Engineering for Multi-Stacked Container-Based Services a Constraint-Based Approach Lars Haugan Master’S Thesis Spring 2015
Total Page:16
File Type:pdf, Size:1020Kb
Simplifying Release Engineering for Multi-Stacked Container-Based Services A constraint-based approach Lars Haugan Master’s Thesis Spring 2015 Simplifying Release Engineering for Multi-Stacked Container-Based Services Lars Haugan 18th May 2015 ii Abstract Today, large and complex solutions are needed to provide the services that users require, where cloud-based solutions have been the panacea to help solve this problem. However, these solutions, are complex in the nature of their implementation, and the need for a standardized way of handling the services are in order. This thesis aims to explore the possibilities of simplifying release engin- eering processes, with the usage of multi-stacked container-based ser- vices. A model is designed with the goal of reducing the complexity in release engineering processes. It enables restriction of possible outcomes by enabling constraints to specify the wanted state of an environment, and enforces a single method approach towards achieving a more uniform environment. The model has been implemented in a prototype, which enables the documentation, configuration and orchestration of the deployed services, that are deployed with the usage of Docker containers. Through the implementation, the validity of the designed model is verified, and the complexity of the release engineering processes are reduced. iii iv Contents 1 Introduction 1 2 Background 5 2.1 Services on the Web . .5 2.1.1 Technology of the web . .6 2.1.2 Changes in methodology and technology . .6 2.2 Service architectures . .7 2.2.1 Components of complex service architectures . .9 2.3 Release engineering . 10 2.3.1 Common challenges . 12 2.3.2 Continuous delivery and deployment . 12 2.4 IT operations methodologies . 13 2.4.1 DevOps . 14 2.5 Cloud computing solutions . 15 2.6 Containers and container technologies . 15 2.6.1 Containers compared to VMs . 16 2.6.2 How does Containers work . 17 2.6.3 chroot jails . 18 2.6.4 Linux Containers - LXC . 18 2.6.5 Docker . 18 2.6.6 Rocket . 19 2.6.7 New cloud solutions for containers . 19 2.7 Relevant research . 20 2.7.1 TOSCA . 21 2.7.2 Towards the holy grail of continuous delivery . 21 2.7.3 The Deployment Production Line . 21 2.7.4 MLN . 22 2.8 Usage Scenarios . 22 2.8.1 Service state management . 23 2.8.2 The release of a new service . 23 2.8.3 Deployment of a new version . 23 2.8.4 Management of the service . 24 3 Approach 25 3.1 Design phase . 26 3.1.1 Model scope and properties . 27 3.1.2 Model . 27 v 3.1.3 Expected outcomes . 29 3.2 Implementation phase . 29 3.2.1 Environment . 29 3.2.2 Prototype . 30 3.2.3 Apprising properties . 31 3.2.4 Expected results . 31 4 Design 33 4.1 Model overview . 33 4.2 Terminology and definitions . 34 4.2.1 Processes . 34 4.2.2 Actions . 35 4.2.3 Constraints . 35 4.2.4 Terminology . 36 4.3 State of the service environment . 42 4.3.1 Defining state . 43 4.3.2 Example environment . 44 4.3.3 Constraints of the state . 48 4.4 Processes . 51 4.4.1 Process 1: State of the environment and services . 52 4.4.2 Process 2: New service to be deployed . 54 4.4.3 Process 3: New version of a service to be deployed . 56 4.4.4 Process 4: Management of a running service . 58 4.5 Summary . 59 5 Implementation 61 5.1 The implementation and the decisions made . 61 5.1.1 Handling the state . 61 5.1.2 Docker . 62 5.1.3 Python and libraries . 62 5.1.4 Structure of the prototype . 63 5.1.5 Getting started with the prototype . 65 5.2 Getting ready for deployment . 66 5.3 Process 2: New service to be deployed . 68 5.3.1 Adding the service . 69 5.3.2 New endpoint for the new service . 69 5.3.3 Creating a service tree . 70 5.4 Process 3: New version of a service . 70 5.4.1 Deploying the new version . 71 5.4.2 Changing the stack . 72 5.5 Process 1: Showing the state . 73 5.5.1 Presentation of services . 74 5.5.2 Presenting the other items . 74 5.6 Apprising properties . 75 5.6.1 Time to completion and needed steps . 75 5.6.2 Experience and understanding . 76 5.6.3 Reproducibility . 77 vi 6 Discussion 79 6.1 Implementation . 79 6.1.1 Apprising properties . 79 6.1.2 Reduced complexity through reduced variation . 81 6.1.3 Scalable solution . 81 6.1.4 Structured state, not unstructured state . 82 6.1.5 An expandable and Open Source solution . 82 6.1.6 Ease of implementation . 82 6.2 Model . 83 6.2.1 How its defined . 83 6.2.2 Simplicity, best practices or both? . 83 6.2.3 Enabling extensions . 84 6.3 Approach . 84 6.4 Related work . 85 6.4.1 Infrastructure as code . 85 6.4.2 Continuous integration and the role of orchestration 86 6.5 Impact . 86 6.5.1 How does it fit? . 87 6.5.2 Road ahead . 87 7 Conclusion 91 A Appendix 97 A.1 Prototype . 97 A.1.1 Python files . 98 A.2 The webapp . 123 A.3 HAProxy Config updater . 124 vii viii List of Figures 2.1 The development of infrastructure and methodology . .7 2.2 Service Oriented Architecture with ESB . .8 2.3 Continuous- delivery and deployment . 13 2.4 VM architecture and Container architecture . 16 3.1 Outline of the approach . 26 3.2 Service environments . 27 4.1 Model overview . 34 4.2 Service tree with three services . 40 4.3 Service tree example . 41 4.4 Multiple service trees example . 41 4.5 Entity relation of items . 43 4.6 Large service environment . 45 4.7 Infrastructure for environments . 46 4.8 Relation between endpoint, service and stacks . 46 4.9 Complete service tree . 47 4.10 The specific service tree . 48 4.11 State of an small service environment . 48 4.12 View state process . 53 4.13 New service process . 54 4.14 New version of service . 57 4.15 Replacing container . 58 5.1 Building a new version . 66 5.2 New service process diagram . 68 5.3 New version process diagram . 71 6.1 Thesis on the stack . 87 ix x List of Tables 4.1 Table with constraints of state . 51 xi xii Acknowledgements First and foremost I would like express my sincere gratitude towards my supervisor Kyrre Begnum, for his expertise and inspiration, that has helped me during this project. His encouragement, enthusiasm, creativity and constructive suggestions, is greatly appreciated. Next I would like to thank Morten Iversen, Fredrik Ung, and Ratish Mohan for sticking around with me for the duration of this thesis, and their endless inspiration and input. I would also like to acknowledge my coworkers, for their input and inspiration throughout the masters, for which I am far better off than without. In addition I would like to extend my gratitude to my family, for the support and understanding during this thesis. Last but not least, to the person who has helped me endlessly through the entire course of this masters program, Katrine Dunseth, thank you for the patience and love you have shown. Sincerely, Lars Haugan xiii xiv Chapter 1 Introduction Our society is today being rapidly digitized. With a rapid growth of Internet users over the last decade, the number of users of the Internet, has increased by almost 60% [1]. Today, there are almost 3 billion people that have access to the Internet [1, 2]. But the users that are already connected, have increasing requirements to the services already available, and the once that are not yet digitalized. Large services are today used and are depended upon by millions. Online banking services, newspapers, and social media sites are just a few. As worlds larges social media site, Facebook has on average 890 million daily active users each day [3], which is an astonishing 30% of the active Internet users. With this volume, and huge expectancy from the users, new challenges appear in the process of providing architecture and services. One of the trends over the last year has been the digitalization of currency and mobile payment solutions. This show that there is an expectancy of uptime and reliability of the services provided to the users. Furthermore, new features are expected at a more rapid rate than what were previously expected. Large and complex solutions are needed to provide the services the users require. Cloud-based solutions have over the last years appeared as a panacea to this problem. These solutions are only part of the solution, as they are complex in the nature of their implementation. Today’s applications are more complex with loose coupling and complex relations, and yet they must be developed more quickly than ever. This.