IN ACTION SECOND EDITION Jeff Nickoloff Stephen Kuenzli FOREWORD BY Bret Fisher MANNING Docker running three containers on a Linux system Container Container Container space A space B space C Database Command line Docker CLI Web server User space Hello World Docker daemon Operating system IO CPU Memory Network interface Persistent storage Devices www.EBooksWorld.ir Praise for the first edition “All there is to know about Docker. Clear, complete, and precise.” —Jean-Pol Landrain, Agile Partner Luxembourg “A compelling narrative for real-world Docker solutions. A must-read!” —John Guthrie, Pivotal, Inc. “An indispensable guide to understanding Docker and how it fits into your infrastructure.” —Jeremy Gailor, Gracenote “Will help you transition quickly to effective Docker use in complex real-world situations.” —Peter Sellars, Fraedom “. a superlative introduction to, and reference for, the Docker ecosystem.” —Amazon reader www.EBooksWorld.ir Docker in Action SECOND EDITION JEFF NICKOLOFF STEPHEN KUENZLI FOREWORD BY BRET FISHER MANNING SHELTER ISLAND For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: [email protected] ©2019 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. Manning Publications Co. Development editor: Jennifer Stout 20 Baldwin Road Technical development editor: Raphael Villela PO Box 761 Review editor: Aleksandar Dragosavljevic´ Shelter Island, NY 11964 Project editor: Janet Vail Copy editor: Sharon Wilkey Proofreader: Keri Hales Technical proofreader: Niek Palm Typesetter: Dennis Dalinnik Cover designer: Marija Tudor ISBN: 9781617294761 Printed in the United States of America For Jarrod Nickoloff and William Kuenzli contents foreword xiii preface xv acknowledgments xviii about this book xx about the authors xxii about the cover illustration xxiii Welcome to Docker 1 1 1.1 What is Docker? 3 “Hello, World” 3 ■ Containers 5 ■ Containers are not virtualization 5 ■ Running software in containers for isolation 6 ■ Shipping containers 7 1.2 What problems does Docker solve? 8 Getting organized 9 ■ Improving portability 10 Protecting your computer 11 1.3 Why is Docker important? 12 1.4 Where and when to use Docker 13 1.5 Docker in the larger ecosystem 14 1.6 Getting help with the Docker command line 14 vii viii CONTENTS PART 1PROCESS ISOLATION AND ENVIRONMENT- INDEPENDENT COMPUTING ................................17 Running software in containers 19 2 2.1 Controlling containers: Building a website monitor 20 Creating and starting a new container 21 ■ Running interactive containers 22 ■ Listing, stopping, restarting, and viewing output of containers 23 2.2 Solved problems and the PID namespace 25 2.3 Eliminating metaconflicts: Building a website farm 28 Flexible container identification 28 ■ Container state and dependencies 31 2.4 Building environment-agnostic systems 34 Read-only filesystems 34 ■ Environment variable injection 37 2.5 Building durable containers 40 Automatically restarting containers 41 ■ Using PID 1 and init systems 42 2.6 Cleaning up 44 Software installation simplified 47 3 3.1 Identifying software 48 What is a named repository? 48 ■ Using tags 49 3.2 Finding and installing software 50 Working with Docker registries from the command line 50 Using alternative registries 51 ■ Working with images as files 52 Installing from a Dockerfile 53 ■ Using Docker Hub from the website 54 3.3 Installation files and isolation 56 Image layers in action 57 ■ Layer relationships 58 Container filesystem abstraction and isolation 59 Benefits of this toolset and filesystem structure 60 Weaknesses of union filesystems 60 Working with storage and volumes 62 4 4.1 File trees and mount points 63 4.2 Bind mounts 64 4.3 In-memory storage 67 CONTENTS ix 4.4 Docker volumes 68 Volumes provide container-independent data management 70 Using volumes with a NoSQL database 71 4.5 Shared mount points and sharing files 73 Anonymous volumes and the volumes-from flag 74 4.6 Cleaning up volumes 77 4.7 Advanced storage with volume plugins 78 Single-host networking 80 5 5.1 Networking background (for beginners) 81 Basics: Protocols, interfaces, and ports 81 ■ Bigger picture: Networks, NAT, and port forwarding 82 5.2 Docker container networking 83 Creating a user-defined bridge network 84 ■ Exploring a bridge network 86 ■ Beyond bridge networks 88 5.3 Special container networks: host and none 89 5.4 Handling inbound traffic with NodePort publishing 91 5.5 Container networking caveats and customizations 93 No firewalls or network policies 93 ■ Custom DNS configuration 93 ■ Externalizing network management 97 Limiting risk with resource controls 99 6 6.1 Setting resource allowances 100 Memory limits 101 ■ CPU 102 ■ Access to devices 105 6.2 Sharing memory 105 Sharing IPC primitives between containers 106 6.3 Understanding users 107 Working with the run-as user 108 ■ Users and volumes 111 Introduction to the Linux user namespace and UID remapping 113 6.4 Adjusting OS feature access with capabilities 114 6.5 Running a container with full privileges 116 6.6 Strengthening containers with enhanced tools 117 Specifying additional security options 118 6.7 Building use-case-appropriate containers 119 Applications 119 ■ High-level system services 120 Low-level system services 120 x CONTENTS PART 2PACKAGING SOFTWARE FOR DISTRIBUTION .......123 Packaging software in images 125 7 7.1 Building Docker images from a container 126 Packaging “Hello, World” 126 ■ Preparing packaging for Git 127 ■ Reviewing filesystem changes 128 ■ Committing a new image 129 ■ Configuring image attributes 130 7.2 Going deep on Docker images and layers 131 Exploring union filesystems 131 ■ Reintroducing images, layers, repositories, and tags 134 ■ Managing image size and layer limits 137 7.3 Exporting and importing flat filesystems 139 7.4 Versioning best practices 141 Building images automatically with Dockerfiles 144 8 8.1 Packaging Git with a Dockerfile 145 8.2 A Dockerfile primer 148 Metadata instructions 149 ■ Filesystem instructions 153 8.3 Injecting downstream build-time behavior 156 8.4 Creating maintainable Dockerfiles 159 8.5 Using startup scripts and multiprocess containers 162 Environmental preconditions validation 163 ■ Initialization processes 164 ■ The purpose and use of health checks 166 8.6 Building hardened application images 167 Content-addressable image identifiers 168 ■ User permissions 169 ■ SUID and SGID permissions 171 Public and private software distribution 174 9 9.1 Choosing a distribution method 175 A distribution spectrum 175 ■ Selection criteria 176 9.2 Publishing with hosted registries 178 Publishing with public repositories: “Hello World!” via Docker Hub 179 ■ Private hosted repositories 181 9.3 Introducing private registries 183 Using the registry image 186 ■ Consuming images from your registry 187 9.4 Manual image publishing and distribution 188 A sample distribution infrastructure using FTP 190 9.5 Image source-distribution workflows 194 Distributing a project with Dockerfile on GitHub 194 CONTENTS xi Image pipelines 197 10 10.1 Goals of an image build pipeline 198 10.2 Patterns for building images 199 All-in-one images 200 ■ Separate build and runtime images 201 ■ Variations of runtime image via multi-stage builds 202 10.3 Record metadata at image build time 204 Orchestrating the build with make 205 10.4 Testing images in a build pipeline 209 10.5 Patterns for tagging images 212 Background 212 ■ Continuous delivery with unique tags 213 Configuration image per deployment stage 214 ■ Semantic versioning 215 PART 3HIGHER-LEVEL ABSTRACTIONS AND ORCHESTRATION......................................217 Services with Docker and Compose 219 11 11.1 A service “Hello World!” 220 Automated resurrection and replication 222 ■ Automated rollout 224 ■ Service health and rollback 226 11.2 Declarative service environments with Compose V3 229 A YAML primer 231 ■ Collections of services with Compose V3 233 11.3 Stateful services and preserving data 237 11.4 Load balancing, service discovery, and networks with Compose 239 First-class configuration abstractions 244 12 12.1 Configuration distribution and management 245 12.2 Separating application and configuration 247 Working with the config resource 249 ■ Deploying the application 250 ■ Managing config resources directly 251 12.3 Secrets—A special kind of configuration 255 Using Docker secrets 257 xii CONTENTS Orchestrating services on a cluster of Docker hosts 13 with Swarm 264 13.1