A Guide to Setup Your Own Kubernetes Cluster... with Gesticlean Up! Documentation Release 1.0.0
Total Page:16
File Type:pdf, Size:1020Kb
A guide to setup your own Kubernetes Cluster... With GestiClean Up! Documentation Release 1.0.0 David Berardozzi May 15, 2017 Contents 1 Introduction 1 2 Goal 3 3 Credits 5 4 Author 7 5 Openstack 9 5.1 Project..................................................9 5.2 SSH keys................................................. 10 5.3 Download secrets............................................. 10 5.4 Horizon dashboard............................................ 11 5.5 Prerequisites............................................... 11 5.6 API’s clients............................................... 12 6 Juju 13 6.1 Install Juju................................................ 14 6.2 Cloud configuration........................................... 14 6.3 Bootstrapping.............................................. 15 6.4 High availability............................................. 17 6.5 Backup and restore............................................ 17 6.6 Neutron networks (Optional)....................................... 18 6.7 Dashboard................................................ 19 6.8 Model................................................... 19 7 Kubernetes 21 7.1 Deploying................................................ 22 7.2 High availability............................................. 26 7.3 kubectl.................................................. 26 7.4 Dashboards................................................ 26 7.5 Namespaces............................................... 26 7.6 We are working on Openstack and we love logs (Optional for Openstack part, see next chapter)... 27 7.7 Ceph persistent storage pool....................................... 29 7.8 kubectl exec and port-forward...................................... 30 7.9 Pulling private images from Dockerhub................................. 31 7.10 Logs................................................... 31 i 7.11 Troubleshooting............................................. 32 7.12 Deeeeeestrrroooy tests.......................................... 33 7.13 YES SECURITY MATTERS!...................................... 35 7.13.1 Your API and dashboard are accessible... from ANYONE!!.................. 35 7.13.2 Ok you have done great, but your password is WEAK!!!................... 36 7.13.3 Paranoid security issues..................................... 36 8 Self maintained high availability Postgresql Cluster 37 8.1 Secure store............................................... 38 8.1.1 Option A: Going with Consul.................................. 38 8.1.2 Option B: Going with etcd................................... 39 8.2 Stolon................................................... 40 8.2.1 Stolon preparation....................................... 40 8.2.2 Stolon deployment....................................... 42 8.2.3 Tests and operation....................................... 42 8.3 Connection pooling........................................... 43 9 File sharing system 45 9.1 Allow privileged containers and security precautions and allow nfs on nodes............. 46 9.2 Deploy NFS server............................................ 47 9.3 Creating and claiming NFS persistent volumes............................. 49 10 Postgresql connection pooling 53 10.1 Prepare.................................................. 53 10.2 Deploy.................................................. 58 10.3 Test.................................................... 59 10.4 Pooling modes.............................................. 59 11 Reverse proxy 61 11.1 Manage certificates............................................ 62 11.2 Activate default ingress controller.................................... 62 11.3 Config maps............................................... 62 11.4 Ingress.................................................. 63 11.5 Deploy.................................................. 64 11.6 Tests................................................... 64 11.7 External load balancing......................................... 64 12 GestiClean Up’ at last! (or your own magnificent incredible personal app) 65 12.1 Docker image............................................... 65 12.2 Database prerequisites.......................................... 67 12.3 Config map................................................ 68 12.4 Deployment preparation......................................... 68 12.5 Deploy.................................................. 69 12.6 Test.................................................... 70 ii CHAPTER 1 Introduction Dear reader, Here is the very first version of “a guide to setup a Kubernetes cluster on OVH”. It was written to deploy a 1.5.x Kubernetes Cluster. We are now in the process to test the new version 1.6.x, therefore some parts may already be outdated for this latest series. This doc was initially centered on our product GestiClean Up’, I tried to remove parts that are too much specific. Anyway, I think it could be adapted for other apps. Remember I have not the pretention to write a user manual nor the ultimate serious guide for Kubernetes. Instead, you may take this doc as a “personal diary” containing what I learnt testing Kubernetes and docker infrastructure for our own usage. I am not an english native speaker, sorry for the numerous spelling mistakes you will find. There must be a lot of things that you will do or explain much better than me. That is why you are very welcome to contribute freely in this bitbucket repo if you want to correct some of my false statements ;) Our stack is composed of: • javascript client • nginx reverse proxy and load balancer • nginx, uwsgi (emperor with pg module enabled), twisted for asynchronous communication (IoT) and python 2.7 (version 2.7 because twisted is not fully available in 3 series) • Postgresql This is a multi tenant app (one base code for all customers) controlled by a higher controller named SaasAdmin for now. Maybe apart for Twisted, this stack is pretty similar to a classic Flask, Web2Py, Django, Pyramid... setup, so I made this document public in case it could be useful to you (yes you who spent so much time googling around to find the light or at least the beginning of a candle). This work is the result of extensive researches, incommensurable tries and fails, wrong path, U turns, joy and sadness. I hope this would be useful to those who are taking the path to deploy their all brand new cloud ready web app. Some of my statements may (should) be balanced with specialists advices, but this is just “our path to success” in building our production infrastructure for our company named Inforum. Feel free to comment and enhance. It is sprinkled with my modest, subjective and very restrictive point of view on several tools and methods that I balanced as much as I 1 A guide to setup your own Kubernetes Cluster... With GestiClean Up! Documentation, Release 1.0.0 could with alternatives that I enjoined you to explore. You are very welcome to complete my thoughts with your own experiences or making corrections to my false statements if there are any. For information we also considered Cloud Foundry (through Pivotal services), Marathon (through OVH Paas services), Docker Swarm and the excellent Jelastic (proprietary). Regarding Jelastic, you should definitely give a try to this one with one of their numerous providers if you are looking for a simple (almost magical) way to deploy your code with minimal adaptations. I won’t extend this (well long enough) document with the factors that made us choose Kubernetes. This may be detailed in an other post if require later. The choice has been made to provide a solution self packaged and ready for the latest web technologies in order to power our software GestiClean Up’. GestiClean Up’ is a point of sale software specialized for the dry cleaning market (yes there is also a small room for this). But it can accept modules (community or corporate) to adapt it to any market. It is a Saas app, accessible through any modern web browser. We built our own framework specialized in managing process steps. Sorry this is not open source and still in extensive development. But we plan to provide all tools to let a community build their own modules. Our software includes a custom proxy for connected objects called the UCS (Unified Communication System) and the so called EasyPlug. The EasyPlug can transform any old fashion RS232 device (or other interfaces) in a connected object (yes one could find a lot of specific peripherals in a dry cleaning shop like conveyors, marking printers that are not at all prepared for the internet of things). You will noticed that we are using here the excellent OVH Public Cloud. This is one of the consequence of being a very satisfied customer for years and having the great opportunity to be part of their Digital Launchpad Program. This work was initially shared with the OVH’s Digital Launchpad Program members as a “gift” to this community. 2 Chapter 1. Introduction CHAPTER 2 Goal Give all the resources to build a high availability GestiClean Up’ (or whatever app) cluster fully portable and agnostic to cloud providers, providing that Kubernetes is installable (see our website for details about GestiClean Up’). The global “philosophy” was to use modern technologies, and to find all the possible tools that could ease their deployment, and most important to me, their maintenance and scalability. This is the answer to the question “why using Juju instead of being satisfied with Openstack’s API?” for example. Oh, and I forgot that it should minimize costs of ownership. Note: A Kubernetes