Building Linux distribution packages with Docker Bruno Cornec HPE EMEA EG Presales Strategist WW Linux Community Lead, HPE Open Source Pro ession !"#0 – October 20'( #$#A Custo%ers Solution Inno&ation Center Grenoble Ma)ing the ne+ style o ,T a reality # o » './ years o success, +orld +ide programs, including Cloud Center o Excellence, C Big Data Center o Excellence, Open Source Solutions ,nitiati!e, 0,SC to HP Intel Architecture Migrations, N ! Center o Excellence, EMEA Networking Customer 1isit Center and more » C Complete ,- 23$$/ systems, 4$$$/ net+ork ports, .$$/ -B storage5 o » Port olio o 3$/ ready to demo solutions +it* access to our ecosystem o Partners P » Complete test 6 !alidation en!ironment » Strategic partners*ip +it* Intel, '.7year long standing colla&oration » Strategic partners*ip +it* "ed Hat 87year colla&oration 2OSS,5 o % » e A uni9ue proo point in t*e industry +it* a pro!en ser!ice o:ering d e & i L Mission: Accelerate t*e adoption o new and inno!ati!e solutions &y creating simple and re+arding end7to7end customer experiences t*at &ene it our customers and partners, in a p o compelling and engaging colla&orative en!ironment. h s k …more information available at http://www.hpintelco.net r o ' Introducing m(sel) ● So t+are engineering and <nices since '=>>; – Mostly Con iguration Management Systems 2CMS5, Build systems, 9uality tools, on multiple commercial <nix systems – ?isco!ered Open Source 6 Linux 2OSL5 6 made irst contri&utions in '==4 – @ull time on OSL since '==5, irst as HP reseller t*en AHP ● Currently; – OSL -ec*nology Strategist, EMEA EG ,nno!ation Solution Center a)a HPB,ntel Solution Center, Greno&le – HP OSL Ad&ocate and Con!erged ,n rastructure A%bassador – WW Linux Community Lead or t*e HP Open Source Profession – POSS con erence, OpenStac)# r and A@<L board member# Con erences at WW le!el at LinuxCon, Linux#con #au, ### – Mondo0escue, Project7Builder#org, <<WL and P<SD Pro*ect Lead – LinuxCOE, mrepo, tellico, rinse, ossology, collectl, ,ronic contributor – @OSSBaEaarBSP?F and OSL +o&ernance ent*usiast – Mandri!a, Mageia, @edora packager ● And also; – Amateur singer 2Alto B -enor5, recorder player since '=8( and C*oir director since '=>8, C? collector 2($$$/5, Concerts, P*otograp*y 3 Change in • #xecutable code • Con)iguration • In)ra 0 en&iron%ent • Data De&Ops approach, • $onitoring Change in A continuous deli&er( pipeline • 1 • #xecutable code • Con)iguration • In)ra 0 en&iron%ent • Data • $onitoring • 1 ersion controlled Peer re&iewed Auto%ated tests -lots. Continuous Deliver( Pipeline /uul P N roCect7 odepool Builder#org Infrastructure as code Basics on packaging Linux Distribution , a pro*ect in itsel) Coherent packages set 2')74$)5 ta)en rom upstream proCects Package Manager Management tools Installation progra% Startup scripts Speci ic tools !unctional updates Security updates Community dri!en or Commercial 2HW certi ication, L-S, support5 10/17/16 Linux Distributions 2i%e line '=>3 % GN<B@SF ProCect 7 0# Stallman '==' 7 Linux '==2 % SLS – Peter Mc ?onald L# -or!alds '==3 % Slack+are % Patric) 1ol)erding '==3 % ?ebian Package '==4 – 0ed Hat Linux Marc E+ing '==( % SuSE 6 Hast % @lorian La 0oc*e '==7 – 0ed Hat Package Manager '==( % ?ebian GG<BLinux Ian Murdoc) Eri) -roan 6 Mar) E+ing '==8 – Ad!anced Packaging -ool Brian White "$$' – SLES "$$" – 0HEL "$$3 – @edora "$$3 % H<M 7 Set* 1idal Warren -ogami "$$. % OpenSUSE "$$3 % <&untu Mar) S*tuttlewort* "$'. – ?G@ "$'$ – Mageia Anne Gicolas 10/17/16 7 So%e definitions ● So)tware Package, ● Application stored +it* its metadata and &uild receipt in an arc*i!e ormat# ● Pro!ides dependency in ormation at &uild and install time ● Package for%at, ● Linux pro!ides multiple ormat; rpm, de&, tgz, ip)g,### ● Open, &ased on tool li)e cpio# ● Associated +it* distri&ution amilies. ● Package %anager, ● Automates pac)ages installation, upgrade, con iguration, and remo!al in a consistent manner# ● Manages pac)age dependencies to install easily rom top o tree ● Package repositor(, ● Storage location rom +*ic* so t+are pac)ages are retrie!ed or installationBupdate ● Manages repository metadata, including dependencies ● Continuous Packaging, ● E!ery so t+are component is managed using so t+are pac)ages ● Pac)age &uild is done on t*e ly, as t*e so t+are is de!eloped Wh( still using packages toda( ? ● tar4g5 ormat ad!antages: ● "PM0deb ormat ad!antages ● 0elati!ely easy to handle ● Sta&le ● Sta&le ● Binary and source formats a!aila&le w/ multiarc* support ● Only " tools needed (tar and gEip5 ● Gati!e support or LSBB@HS ● Can integrate some files for metadata ● Provides metadata, build procedure, patches and upstream content ● tar4gz format dra+&acks ; ● Manages installation, upgrade, remo!al ● Go repository management ● SignatureBChec)sum support and veri ication ● Go dependency management ● Deployment ser!er a!aila&ility % Scripted methods ● Go easy update mechanism ● Baseline support ● Go signature support ● 0PM places e!erything in the .spec file and supports ● Limited chec)sum support su&pac)ages ● Go pac)age data&ase ● 0PM Package data&ase a!aila&le to query metadata ● "PM0deb ormat dra+&ac)s ● 0e9uire appropriate tools but in distro ● Porta&ility across OSes Building packages and repositories (rp% world. ● rp%build ● Build src#rpm and arc*#rpm pac)ages rom t*e SPEC ile ● -a)e in account con ig iles, cron jo&s, init scripts, log rotation, shell con ig ● rpmbuild -ba pkg.spec ● DO NOT BUILD AS root ● ● Signing RP$s ● Ensures aut*enticity o t*e pro!ider and pac)age integrity ● 0e9uires GPG con iguration and macros in JHOMEB#rpmmacros ● rpm --addsign pkg.src.rpm pkg.arch.rpm ● rp%lint ● Chec) rpm pac)age common errors +rt distri&ution policies ● rpmlint pkg.src.rpm pkg.arch.rpm ● createrepo ● Separate command 2not a yum option5 ● Create a yum repository rom a directory +it* pac)ages ● createrepo . Packaging best practices ●Ha&e a working installation procedure ● Use con igure i possi&le or language &uild tools 2setup#py, Ma)e ile#PL, ###5 ● Ha!e a ile &ased install +or)ing targeting BusrBlocal ● Use !aria&les or all target directories 2Betc, BusrBs*are, BusrB&in, BusrBli&, ###5 ● Script t*e &uild and install p*ases i necessary to share &et+een !arious tools ● ● 7now your package deli&erables ● Dno+ +*at you +ant to deli!er and +*ere, +*at is optional# Pac)age accordingly ● Dno+ your external dependencies ● Go source components should *a!e t*eir o+n separated pac)ages ● A!oid including ot*er projects code inline# Use dependency or re7pac)age separately i or) needed# ● Generate repositories to ease your consumers' li e ● ?ecide on a license ●A deli&er( is a tree o) packages ● ,ncreasing num&er o pac)ages reduces &uild time on t*e long run ● ,ncrease complexity at pac)age de inition, not at install time, t*an)s to pac)age managers ● ?ecide +*et*er you +ant meta7pac)ages 2prC7all, prC7net, prC7tape, prC7data, ###5 Basics on Docker containers Linux Containers OS Virt OS Virtualization (LXC / Docker containers) – Applications in a Eone are isolated rom t*e ot*ers 2c*root on steroLds5 • Security 2as secure as your )ernel5 • Gamespaces 2root, <,?, G,?, net+or), P,?, ,PC, mount points, *ostnameButs5 • Cgroup Granularity • Single OS image manages HW accesses • Per ormance 6 transparency neit*er emulation nor ull !irtualiEation • Get ilter or net+ork isolation 2GA- and port +d5 • AuFS o!erlays isolated ile systems on top o a p*ysical ile system – Ot*er tec*nos a!aila&le • 17Ser!er • VirtuoEEo • Open1M *ttps;BB&log.doc)er#com/+p7contentBuploadsB"$'3B$4Bdoc)er7execdriver7diagram.png Docker In Brief • ision, Build, S*ip, and 0un Any App Any+*ere • Both, a company and an open source ecosystem 2under t*e Apac*e license 1"#$ since "$'45 • Docker is a tool chain written in Go t*at simpli ies t*e creation and management of containers# • Paradig% shi)t; Nne+” pac)aging model or deploying applications and services using Linux containers 2Google *as &een doing t*is or yearsP " &illion B +ee)5 • Containers, A sel 7contained porta&le en!ironment or pac)aging, deploying, and executing applications and services# Contains all execution dependencies or a service 2single process 7Q single container5 • alue prop • Better resource utili5ation, 1Ms !irtualiEe t*e CP<, – 0elia&le deployment containers !irtualiEe process2es5 (lightweight5 – More e:icient deployment • Si%pli)ied Application Li)ec(cle $anage%ent; – 0educed deployment latency – Build and con igure once 2integrated ?e!Ops5 – ?eploy any+*ere 2laptop, ser!er, cloud, R5 – More e:icient use o system resources – ,- ocuses on managing t*e &ase in rastructure 'hat is Docker value8add ? Goal: run applications in containers in a neutral, lightweig*t and porta&le +ay Bundle, E!erything pac)ed together La(ers; ,mages 2ro5 6 containers 2r+5 using a union FS "egistry; pu&lic/private registry of s*ared images Dockerfile; descriptive &uild of an image olumes; loop&ack mounting host FS into container Ports; expose container services port to the host Portable; created once, run e!erywhere 2Linux &ased5 $anagement; 0ES- AP,BCLI Solomon Hy)es, Doc)er Inc# A la(ered approach I%ages Local or remote re erence content to initiate a container Multiple images can be layered adding content at each time using Copy on Write FS Cache to speed up repeated operations Containers Last layer providing rw access to t*e cumulated set o images 1 application + its deps == 1 container Building packages with Docker 'h( building distribution packages with Containers vs V$s3 Container like V$s brings isolation Go pollution of your running environment Easily scratch and redo i problems Easier refinement & automation o t*e &uild environment with the Docker file Containers like V$s brings multi8distribution support Easy to build or anot*er distri&ution than yours <se ul also for your o+n distro: not e!erybody uses an unsta&le distro Containers can use nati&el( your ho%e director( Allo+ sharing of your pac)age sources for local and in container build Allo+ sharing of your .rpmmacros, .rpmrc files, Mageia SSH keys or Fedora certs $s are mandator( is you need a di9erent kernel Basic Docker workflow to build distribution packages Images Mageia Cauldron Mageia 5 ..