IN PRACTICE Ian Miell Aidan Hobson Sayers FOREWORD BY Ben Firshman MANNING www.allitebooks.com Docker in Practice www.allitebooks.com www.allitebooks.com Docker in Practice IAN MIELL AIDAN HOBSON SAYERS MANNING SHELTER ISLAND www.allitebooks.com For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: [email protected] ©2016 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. Manning Publications Co. Development editor: Cynthia Kane 20 Baldwin Road Technical development editors: Alain Couniot PO Box 761 and Robert Wenner Shelter Island, NY 11964 Copyeditor: Andy Carroll Proofreader: Melody Dolab Technical proofreader: José San Leandro Typesetter: Gordan Salinovic Cover designer: Marija Tudor ISBN 9781617292729 Printed in the United States of America 12345678910–EBM –212019181716 www.allitebooks.com brief contents PART 1DOCKER FUNDAMENTALS ....................................................1 1 ■ Discovering Docker 3 2 ■ Understanding Docker—inside the engine room 19 PART 2DOCKER AND DEVELOPMENT .............................................41 3 ■ Using Docker as a lightweight virtual machine 43 4 ■ Day-to-day Docker 65 5 ■ Configuration management—getting your house in order 103 PART 3DOCKER AND DEVOPS...................................................143 6 ■ Continuous integration: speeding up your development pipeline 145 7 ■ Continuous delivery: a perfect fit for Docker principles 169 8 ■ Network simulation: realistic environment testing without the pain 186 v www.allitebooks.com vi BRIEF CONTENTS PART 4DOCKER IN PRODUCTION ...............................................213 9 ■ Container orchestration: managing multiple Docker containers 215 10 ■ Docker and security 262 11 ■ Plain sailing—Docker in production and operational considerations 291 12 ■ Docker in production—dealing with challenges 308 www.allitebooks.com contents foreword xv preface xvii acknowledgments xix about this book xx about the cover illustration xxiii PART 1DOCKER FUNDAMENTALS........................................1 Discovering Docker 3 1 1.1 The what and why of Docker 5 What is Docker? 5 ■ What is Docker good for? 7 ■ Key concepts 8 1.2 Building a Docker application 10 Ways to create a new Docker image 11 ■ Writing a Dockerfile 12 Building a Docker image 13 ■ Running a Docker container 14 Docker layering 16 1.3 Summary 18 Understanding Docker—inside the engine room 19 2 2.1 Docker’s architecture 20 vii www.allitebooks.com viii CONTENTS 2.2 The Docker daemon 21 TECHNIQUE 1 Open your Docker daemon to the world 22 TECHNIQUE 2 Running containers as daemons 23 TECHNIQUE 3 Moving Docker to a different partition 26 2.3 The Docker client 27 TECHNIQUE 4 Use socat to monitor Docker API traffic 27 TECHNIQUE 5 Using ports to connect to containers 29 TECHNIQUE 6 Linking containers for port isolation 31 TECHNIQUE 7 Using Docker in your browser 33 2.4 Docker registries 34 TECHNIQUE 8 Setting up a local Docker registry 35 2.5 The Docker Hub 36 TECHNIQUE 9 Finding and running a Docker image 37 2.6 Summary 39 PART 2DOCKER AND DEVELOPMENT.................................41 Using Docker as a lightweight virtual machine 43 3 3.1 From VM to container 44 TECHNIQUE 10 Converting your VM to a container 44 TECHNIQUE 11 A host-like container 47 TECHNIQUE 12 Splitting a system into microservice containers 49 3.2 Managing services on your containers 52 TECHNIQUE 13 Managing the startup of your container’s services 53 3.3 Saving and restoring your work 55 TECHNIQUE 14 The “save game” approach to development 55 TECHNIQUE 15 Docker tagging 57 TECHNIQUE 16 Sharing images on the Docker Hub 59 TECHNIQUE 17 Referring to a specific image in builds 61 3.4 Environments as processes 62 TECHNIQUE 18 The “save game” approach to development 62 3.5 Summary 64 Day-to-day Docker 65 4 4.1 Volumes—a persistent problem 66 TECHNIQUE 19 Docker volumes—problems of persistence 66 TECHNIQUE 20 Distributed volumes with BitTorrent Sync 67 www.allitebooks.com CONTENTS ix TECHNIQUE 21 Retain your container’s bash history 69 TECHNIQUE 22 Data containers 71 TECHNIQUE 23 Remote volume mounting using sshfs 74 TECHNIQUE 24 Sharing data over NFS 76 TECHNIQUE 25 Dev tools container 78 4.2 Running containers 79 TECHNIQUE 26 Running GUIs within Docker 79 TECHNIQUE 27 Inspecting containers 81 TECHNIQUE 28 Cleanly killing containers 83 TECHNIQUE 29 Using Docker Machine to provision Docker hosts 84 4.3 Building images 87 TECHNIQUE 30 Injecting files into your image using ADD 88 TECHNIQUE 31 Rebuilding without the cache 90 TECHNIQUE 32 Busting the cache 92 4.4 Staying ship-shape 93 TECHNIQUE 33 Running Docker without sudo 93 TECHNIQUE 34 Housekeeping containers 94 TECHNIQUE 35 Housekeeping volumes 95 TECHNIQUE 36 Detaching containers without stopping them 97 TECHNIQUE 37 Using DockerUI to manage your Docker daemon 98 TECHNIQUE 38 Generate a dependency graph of your Docker images 99 TECHNIQUE 39 Direct action—execute commands on your container 101 4.5 Summary 102 Configuration management—getting your house in order 103 5 5.1 Configuration management and Dockerfiles 104 TECHNIQUE 40 Create reliable bespoke tools with ENTRYPOINT 104 TECHNIQUE 41 Avoid package drift by specifying versions in your build 106 TECHNIQUE 42 Replacing text with perl -p -i -e 107 TECHNIQUE 43 Flattening images 109 TECHNIQUE 44 Managing foreign packages with alien 111 TECHNIQUE 45 Reverse-engineer a Dockerfile from an image 113 5.2 Traditional configuration management tools with Docker 116 TECHNIQUE 46 Traditional: using make with Docker 116 www.allitebooks.com x CONTENTS TECHNIQUE 47 Building images with Chef Solo 118 TECHNIQUE 48 Source-to-image builds 122 5.3 Small is beautiful 128 TECHNIQUE 49 Dockerfile tricks for keeping your build small 128 TECHNIQUE 50 Tricks for making an image smaller 131 TECHNIQUE 51 Tiny Docker images with BusyBox and Alpine 133 TECHNIQUE 52 The Go model of minimal containers 134 TECHNIQUE 53 Using inotifywait to slim containers 137 TECHNIQUE 54 Big can be beautiful 139 5.4 Summary 141 PART 3DOCKER AND DEVOPS ......................................143 Continuous integration: speeding up your development 6 pipeline 145 6.1 Docker Hub automated builds 146 TECHNIQUE 55 Using the Docker Hub workflow 146 6.2 More efficient builds 149 TECHNIQUE 56 Speed up I/O-intensive builds with eatmydata 149 TECHNIQUE 57 Set up a package cache for faster builds 151 TECHNIQUE 58 Running Selenium tests inside Docker 154 6.3 Containerizing your CI process 158 TECHNIQUE 59 Containing a complex development environment 158 TECHNIQUE 60 Running the Jenkins master within a Docker container 162 TECHNIQUE 61 Scale your CI with Jenkins’ Swarm plugin 164 6.4 Summary 168 Continuous delivery: a perfect fit for Docker principles 169 7 7.1 Interacting with other teams during the CD pipeline 170 TECHNIQUE 62 The Docker contract—reducing friction 170 7.2 Facilitating deployment of Docker images 173 TECHNIQUE 63 Manually mirroring registry images 173 TECHNIQUE 64 Delivering images over constrained connections 174 TECHNIQUE 65 Sharing Docker objects as TAR files 176 7.3 Configuring your images for environments 177 TECHNIQUE 66 Informing your containers with etcd 178 CONTENTS xi 7.4 Upgrading running containers 180 TECHNIQUE 67 Using confd to enable zero-downtime switchover 181 7.5 Summary 185 Network simulation: realistic environment testing without the 8 pain 186 8.1 Container communication—beyond manual linking 187 TECHNIQUE 68 A simple Docker Compose cluster 187 TECHNIQUE 69 A SQLite server using Docker Compose 190 TECHNIQUE 70 Finding containers via DNS with Resolvable 194 8.2 Using Docker to simulate real-world networking 197 TECHNIQUE 71 Simulating troublesome networks with Comcast 197 TECHNIQUE 72 Simulating troublesome networks with Blockade 200 8.3 Docker and virtual networks 204 TECHNIQUE 73 Setting up a substrate network with Weave 204 TECHNIQUE 74 Docker networking and service features 208 8.4 Summary 211 PART 4DOCKER IN PRODUCTION...................................213 Container orchestration: managing multiple Docker 9 containers 215 9.1 Simple single-host Docker 217 TECHNIQUE 75 Managing your host's containers with systemd 217 TECHNIQUE 76 Orchestrating your host's containers with systemd 220 9.2 Multi-host Docker 223 TECHNIQUE 77 Manual multi-host Docker with Helios 223 TECHNIQUE 78 A seamless Docker cluster with Swarm 229 TECHNIQUE 79 Using a Kubernetes cluster 234 TECHNIQUE 80 Building a framework on Mesos 239