STANDARDISE DEVELOPMENT ENVIRONMENTS AND MACHINE IMAGES WITH PACKER Marcelo Pinheiro http://salizzar.net - @salizzar SUMMARY
• Motivation
• Installation
• How it works
• Inside Templates
• Some examples
• Our experience
• FAQ MOTIVATION
• How to take control inside the following environment issues?
• Common workstation problems (HD failure, dead computer)
• Different OS’s
• Extra machine configuration to enable devs to work (programming languages, databases, plugins, etc), taking one or two days to be ready-to-code
• “Works on my machine” syndrome MOTIVATION
• Vagrant or Docker. Period.
• Embrace virtualisation
• Each offers a way to pre-setup your VM with necessary libraries, databases and so on
• No more development databases in your pre-staging DB server, additional dependencies
• A try to make development environment more similar to production MOTIVATION
• It sounds good, but… how to take control over each application VM?
• Sometimes your team needs to use some tools that are not available in official package repository (or are too old), forcing to manual configuration after up a VM
• Even running a VM, developer personal choices can contaminate the application (example: rspec add-ons, irb plugins)
• Some developers don’t have knowledge about Chef / Puppet recipes
• How to maintain Vagrant Custom Boxes / Docker Custom Images when you need to add / change tools, repositories or configs? MOTIVATION
• For sysadms / sysops:
• How to export a new machine image to your virtualisation server (KVM, Xen, VMWare, etc) when a new OS release is launched without “dist-upgrade”?
• How to automate it? MOTIVATION
• Packer for the rescue
• Written in Go
• Owner: Mitchell Hashimoto (Vagrant, Serf)
• http://www.packer.io INSTALLATION
• http://www.packer.io/downloads.html
• Download zipped binaries for your OS
• OSX
• Linux
• Windows
• FreeBSD
• OpenBSD INSTALLATION
• Move binaries to your /usr/local/bin, ~/bin, whatever
• It’s done. HOW IT WORKS
• Packer recipes are JSON files
• Validate template:
• $ packer validate your_recipe.json
• Run template:
• $ packer run your_recipe.json INSIDE TEMPLATES
• Packer templates have the following structure:
• Variables
• Builders
• Provisioners
• Post-processors INSIDE TEMPLATES: VARIABLES
• User-defined variables to be used along template
• Can be declared in a custom file INSIDE TEMPLATES: VARIABLES
"variables": {! "box_ostype": "centos",! "box_osversion": "6.5",! "box_nick": "6.5",! "box_arch": "x64",! "box_type": "base",! "box_memory": "512",! "box_cpus": "1",! ! "iso_arch": "x86_64",! "iso_type": "netinstall",! "iso_md5": "939fd1d87c11ffe7795324438b85adfb",! ! "ssh_user": "vagrant",! "ssh_pass": "vagrant",! "hostname": "vagrant-centos-6.5",! "domain": "vagrantup.com"! } INSIDE TEMPLATES: VARIABLES
# my_variables.json! {! "type": "vmware-iso",! "vm_name": "mybox-vmw“,! "guest_os_type": “centos",! "disk_size": “4096”,! ! (… other definitions here…)! }! INSIDE TEMPLATES: BUILDERS • Create a machine image from scratch
• Download a ISO from official OS mirror, select a base image to start
• Set CPU cores, memory size, disk size
• See documentation for further details (a lot of options) INSIDE TEMPLATES: BUILDERS
"builders": [! {! "type": "virtualbox-iso",! "iso_url": "http://an.repository.com/an-image.iso",! "iso_checksum": "an-checksum",! "iso_checksum_type": "md5",! "http_directory": "http",! "ssh_username": "root",! "ssh_password": "apassword",! "ssh_wait_timeout": "100000s",! "shutdown_command": "echo {{ user `ssh_user` }} | sudo halt -p",! ! "boot_command": [! "
• Minimal set of packages
• Disk partition
• Network
• Timezone INSIDE TEMPLATES: BUILDERS • For CentOS: Kickstart
• For Debian: Preseed
• For Windows:
• Windows Automated Installation Kit (AIK)
• Microsoft Deployment Toolkit (MDT) INSIDE TEMPLATES: BUILDERS
• Available builders: !
• QEMU - KVM and Xen • OpenStack (experimental) • Google Compute • VMWare Engine
• Virtualbox • Amazon EC2
• Docker • Digital Ocean INSIDE TEMPLATES: BUILDERS / VMWARE & VIRTUALBOX
• VMWare:
• vmware-iso: create from scratch
• vmware-vmx: create from a base VMX file
• Virtualbox:
• virtualbox-iso: create from scratch
• virtualbox-ovf: create from a base OVF file INSIDE TEMPLATES: BUILDERS / QEMU
• Create KVM / Xen images from scratch
• Packer depends on qemu-system-x86_64, available only on Debian at this time as a binary
• CentOS have qemu-kvm, but you need to manually override all Packer default options INSIDE TEMPLATES: BUILDERS / DOCKER
• Creates a Docker image by pulling a existent, starting a container, provision it and exports a .tar file
• Provision without Dockerfile INSIDE TEMPLATES: BUILDERS / OTHERS • For other builders, you simply need to inform:
• username / password, API key
• base image
• zone and other related information
• See Packer documentation INSIDE TEMPLATES: PROVISIONERS • After the setup of a machine image, it’s time to configure it
• Here is where magic happens:
• Add packages, useful scripts
• Standardise config files
• Apply existent recipes from a CM INSIDE TEMPLATES: PROVISIONERS
• Available provisioners:
• Shell Scripts
• File Uploads
• Ansible
• Chef Solo
• Puppet
• Salt INSIDE TEMPLATES: PROVISIONERS / SHELL SCRIPTS
• Most simple way to setup machine
• Run apt-get, yum and friends INSIDE TEMPLATES: PROVISIONERS / SHELL SCRIPTS
"provisioners": [! {! "type": "shell",! "execute_command": "echo 'root' | sh '{{ .Path }}'",! "scripts": [! "scripts/locale.sh",! "scripts/elrepo.sh"! ]! },! {! "type": "shell",! "pause_before": "30s",! "execute_command": "echo 'root' | sh '{{ .Path }}'",! "scripts": [! "scripts/vagrant.sh",! "scripts/sudoers.sh"! ]! }! ] INSIDE TEMPLATES: PROVISIONERS / FILE UPLOADS
• Need to set default configuration files or upload some custom packages (.tar, .deb / .rpm) to be installed later?
• Upload them and after process with a shell script or CM recipe INSIDE TEMPLATES: PROVISIONERS / FILE UPLOADS
"provisioners": [! {! "type": "shell",! "execute_command": "echo 'root' | sh '{{ .Path }}'",! "scripts": [! "scripts/lamp/vagrant.sh",! "scripts/lamp/apache2.sh",! "scripts/lamp/php5.sh",! "scripts/lamp/mysql.sh"! ]! },! {! "type": "file",! "source": "files/lamp-vagrant/vhost",! "destination": "/etc/apache2/sites-available/lamp-php"! },! {! "type": "shell",! "execute_command": "echo 'root' | sh '{{ .Path }}'",! "script": "scripts/lamp/enable-vhost"! }! ] INSIDE TEMPLATES: PROVISIONERS / OTHERS
• The following provisioners requires installation before run:
• Ansible
• Puppet
• Salt
• Chef Solo is installed by Packer if not present
• At this time, all provisioners are executed in client mode (no remote server) INSIDE TEMPLATES: PROVISIONERS / OTHERS
"provisioners": [! {! "type": "ansible-local",! "playbook_file": "recipes/ansible/lamp.yml"! },! {! "type": "chef-solo",! "cookbook_paths": [ "recipes/chef/lamp" ]! },! {! "type": "puppet-masterless",! "manifest_file": "recipes/puppet/lamp"! },! {! "type": "salt-masterless",! "local_state_tree": "recipes/salt/lamp"! }! ] INSIDE TEMPLATES: POST-PROCESSORS • After create / setup a machine image, you can:
• Convert to a Vagrant Custom Box
• Locally add it as a Docker container
• Publish in a Docker registry
• Publish in a vSphere endpoint INSIDE TEMPLATES: POST-PROCESSORS / VAGRANT
• Defines a box output name
• You can attach a Vagrantfile template and other template files (cookbooks)
• Change compression rate if you want INSIDE TEMPLATES: POST-PROCESSORS / VAGRANT
"post-processors": [! {! "type": "vagrant",! “output": "lamp-vagrant.box"! }! ] INSIDE TEMPLATES: POST-PROCESSORS / DOCKER
• You can locally import a Docker image
• You can push a Docker image to a registry
• Needs manual login (automated soon)
• Important: Docker pushes a completely new image, not incremental INSIDE TEMPLATES: POST-PROCESSORS / DOCKER
"post-processors": [! {! "type": "docker-import",! "repository": "salizzar/packer",! "tag": "0.1"! },! "docker-push"! ] INSIDE TEMPLATES: POST-PROCESSORS / VSPHERE
• Upload to a vSphere endpoint INSIDE TEMPLATES: POST-PROCESSORS / VSPHERE
"post-processors": [! {! "type": "vsphere",! "host": "a-vsphere-host.com",! "username": "my_user",! "password": "my_password",! "cluster": "a-cluster",! “datacenter": "xyz",! "datastore": "zyx",! “resource_pool": "zyx",! "vm_folder": "images",! "vm_name": "lamp",! “vm_network": "staging"! },! ] SOME EXAMPLES
• It’s time to see some code!
• All examples are available on:
• https://github.com/salizzar/packer-examples OUR EXPERIENCE Ivan IV Vasilyevich (the Terrible) OUR EXPERIENCE
• Prepare to argue (sometimes fight :)
• It’s hard to change development tradition of premature optimisation, ultra-high performance, personal choices, “nightly build” syndrome
• Create a culture first OUR EXPERIENCE
• Sometimes the better choice must be autocracy-based
• Use OS package system ASAP (or backport / automate installation if package not exists)
• Introduce to developers a wisdom to use the same package of programming language / DB / whatever that runs in production (!)
• If is old, upgrade your app to use a newer version
• The same for tools that “vendorize" your app libraries (maven, bundler, etc) OUR EXPERIENCE
• Make all applications ready-to-setup-and-run with one command
• Track all dependencies with Dockerfile or Vagrant Shell Scripts
• Bash scripts are more easy to setup than 3rd party CM tools at first time
• Adopt a convention to make all applications more similar as possible about their structure
• Code generators OUR EXPERIENCE
• Divide to conquer
• Adopt a bottom-up strategy
• Minor systems that are easy to setup
• Minor teams
• Start to apply with more systems and greater teams
• Standardise ASAP OUR EXPERIENCE
• At this time, major systems in Locaweb PaaS are Vagrant-ready
• git clone, vagrant up, vagrant ssh
• Docker in development
• Internally created a gem to apply standardisation of Rails apps:
• Packaging (Debian)
• Va g r a n t
• Packer recipes to create Vagrant custom boxes, using our mirrors FAQ
• Questions?
• New recipes available on:
• https://github.com/salizzar/packer-vmware THANK YOU! :)