Continuous Packaging is also mandatory for DevOps Bruno Cornec HP EMEA EG Presales Strategist WW Linux Community Lead, HP Open Source Profession v 3#$ % August $&'( $%$A Customers Solution Innovation Center Grenoble Ma)ing the ne+ style o ,T a reality $ o » './ years o success, +orld +ide programs, including Cloud Center o Excellence, C Big Data Center o Excellence, Open Source Solutions ,nitiati!e, 0,SC to HP Intel Architecture Migrations, N ! Center o Excellence, EMEA Networking Customer 1isit Center and more » C Complete ,- 23&&/ systems, "&&&/ net+ork ports, .&&/ -B storage4 o » Port olio o 3&/ ready to demo solutions +it* access to our ecosystem o Partners P » Complete test 5 !alidation en!ironment » Strategic partners*ip +it* Intel, '.6year long standing colla7oration » Strategic partners*ip +it* #ed Hat 86year colla7oration 2OSS,4 o m » e A uni9ue proo point in t*e industry +it* a pro!en ser!ice o:ering d e v i ' Mission: Accelerate t*e adoption o new and inno!ati!e solutions 7y creating simple and re+arding end6to6end customer experiences t*at 7ene it our customers and partners, in a p o compelling and engaging colla7orative en!ironment. h s k …more information available at http://www.hpintelco.net r o & Introducing myself ● So t+are engineering and <nices since '=>>; – Mostly Configuration Management Systems 2CMS4, Build systems, 9uality tools, on multiple commercial <nix systems – ?isco!ered Open Source 5 Linux 2OSL4 5 made first contri7utions in '==" – @ull time on OSL since '==., first as HP reseller t*en AHP ● Currently; – OSL -ec*nology Strategist, EMEA EG ,nno!ation Solution Center aka HPB,ntel Solution Center, Greno7le – HP OSL Advocate and Con!erged ,n rastructure Am(assador – WW Linux Community Lead or t*e HP Open Source Profession – POSS con erence, OpenStack#fr and A@<L (oard mem(er. Con erences at WW le!el at LinuxCon, Linux#con #au, #.. – Mondo0escue, ProCect6Builder#org, <<WL and P<SD Pro)ect 'ead – LinuxCOE, mrepo, tellico, rinse, ossology, collectl, ,ronic contri(utor – @OSSBaEaarBSP?F and OSL *overnance ent*usiast – Mandri!a, Mageia, @edora packager ● And also; – Amateur singer 2Alto B -enor4, recorder player since '=8( and C*oir director since '=>8, C? collector 2(&&&/4, Concerts, P*otograp*y 4 Change in • $/ecuta(le code • Configuration • Infra 0 environment • Data DevOps approach+ • %onitoring Change in A continuous delivery pipeline • 1 • $/ecuta(le code • Configuration • Infra 0 environment • Data • %onitoring • 1 ersion controlled Peer revie"ed Automated tests ,lots- Continuous Delivery Pipeline .uul P N roCect6 odepool Builder#org Infrastructure as code Basics on packaging Linu/ Distribution + a pro)ect in itself Coherent packages set 2')6"&)4 ta)en rom upstream proCects Package Manager Management tools Installation program Startup scripts Speci ic tools !unctional updates Security updates Community dri!en or Commercial 2HW certi ication, L-S, support4 8/23/16 Linu/ Distributions 2ime line '=>3 % GN<B@SF ProCect 6 0# Stallman '==' 6 Linux '==2 % SLS – Peter Mc ?onald L# -or!alds '==3 % Slack+are % Patric) 1ol)erding '==3 % ?ebian Package '==" – 0ed Hat Linux Marc E+ing '==( % SuSE 5 Hast % @lorian La 0oc*e '==7 – 0ed Hat Package Manager '==( % ?ebian GG<BLinux Ian Murdoc) Eri) -roan 5 Mar) E+ing '==8 – Ad!anced Packaging -ool Brian White $&&' – SLES $&&$ – 0HEL $&&3 – @edora $&&3 % H<M 6 Set* 1idal Warren -ogami $&&. % OpenSUSE $&&3 % <7untu Mar) S*tuttlewort* $&'. – ?G@ $&'& – Mageia Anne Gicolas 8/23/16 8 Some definitions ● Soft"are Package+ ● Application stored +it* its metadata and 7uild receipt in an arc*i!e ormat# ● Pro!ides dependency in ormation at 7uild and install time ● Package format+ ● Linux pro!ides multiple ormat; rpm, de7, tgz, ip)g,### ● Open, 7ased on tool li)e cpio# ● Associated +it* distri7ution amilies. ● Package manager+ ● Automates pac)ages installation, upgrade, con iguration, and remo!al in a consistent manner# ● Manages pac)age dependencies to install easily rom top o tree ● Package repository+ ● Storage location rom +*ic* so t+are pac)ages are retrie!ed or installationBupdate ● Manages repository metadata, including dependencies ● Continuous Packaging+ ● E!ery so t+are component is managed using so t+are pac)ages ● Pac)age 7uild is done on t*e ly, as t*e so t+are is de!eloped Why using packages ? ● tar4g5 ormat ad!antages: ● 0elati!ely easy to handle ● Sta7le ● Only $ tools needed (tar and gEip4 ● Can integrate some files for metadata ● tar4gz format dra+7acks ; ● Go repository management ● Go dependency management ● Go easy update mechanism ● Go signature support ● Limited chec)sum support ● Go pac)age data7ase Why using packages ? ● #P%0de( ormat ad!antages ● Sta7le ● Binary and source formats a!aila7le w/ multiarch support ● Gati!e support or LSBB@HS ● Provides metadata, build procedure, patc*es and upstream content ● Manages installation, upgrade, remo!al ● Signature/Checksum support and verification ● Deployment ser!er a!aila7ility % Scripted methods ● Baseline support ● 0PM places e!erything in the .spec file and supports su7packagesI ● 0PM Package data7ase a!aila7le to query metadata ● #P%0de( ormat dra+7acks ● 0e9uire appropriate tools but in distro ● Porta7ility across OSes Building packages and repositories (rpm "orld- ● rpm(uild ● Build src#rpm and arc*#rpm pac)ages rom t*e SPEC ile ● -a)e in account con ig iles, cron jo7s, init scripts, log rotation, shell con ig ● rpmbuild -ba pkg.spec ● DO NOT BUI'D AS root ● ● Signing RP%s ● Ensures aut*enticity o t*e pro!ider and pac)age integrity ● 0e9uires GPG con iguration and macros in JHOMEB#rpmmacros ● rpm --addsign pkg.src.rpm pkg.arch.rpm ● rpmlint ● Chec) rpm pac)age common errors +rt distri7ution policies ● rpmlint pkg.src.rpm pkg.arch.rpm ● createrepo ● Separate command 2not a yum option4 ● Create a yum repository rom a directory +it* pac)ages ● createrepo . Packaging best practices ●Have a working installation procedure ● Use con igure i possi7le or language 7uild tools 2setup#py, Ma)e ile#PL, ###4 ● Ha!e a ile 7ased install +or)ing targeting BusrBlocal ● Use !aria7les or all target directories 2Betc, BusrBs*are, BusrB7in, BusrBli7, ###4 ● Script t*e 7uild and install p*ases i necessary to share 7et+een !arious tools ● ● 7no" your package deliverables ● Dno+ +*at you +ant to deli!er and +*ere, +*at is optional# Pac)age accordingly ● Dno+ your external dependencies ● Go source components should *a!e t*eir o+n separated pac)ages ● A!oid including ot*er projects code inline# Use dependency or re6pac)age separately i or) needed# ● Generate repositories to ease your consumers' li e ● ?ecide on a license ●A delivery is a tree of packages ● ,ncreasing num7er o pac)ages reduces 7uild time on t*e long run ● ,ncrease complexity at pac)age de inition, not at install time, t*an)s to pac)age managers ● ?ecide +*et*er you +ant meta6pac)ages 2prC6all, prC6net, prC6tape, prC6data, ###4 Basics on project-builder.org &hat is pro)ect8(uilder4org9s goal ? Make upstream projects life easier with regards to packaging their software &hat is pro)ect8(uilder4org9s itch ? Make my life easier with regards to packaging my soft"are 6sers / Sysadmins want packaged software NO =&M ● Ease of use: G<I, CLI ● ?istri7ution compliance ● ,ntegration wit* deployment tools ● Lag between SW and Pkg a!aila7ility U L 1&M ● -est of alp*aB7etaB1CS SW ● -oo many: ● distri7utions, ● !ersions, ● pac)age formats, L 1M ● D tools to generate pac)ages ● tools to manage repositories 8/23/16 6pstream benefits from Continuous packaging Packaging should be a pro)ect concern as well as coding, testing, installing, #... especially or smaller projects Packaging as your only way o delivery (not a dream) Minimal o!erhead, maximum bene it; Consistancy and reproducea7ility or de!s and users ?istri7ution & deployment server integration, ,mpro!ed deployment without risk of screwing up the system Packaging as a marketing acti!ity or t*e upstream project. Easy +ay to extend your user base, and impro!e your community relationship and is a “competiti!e ad!antage”# Ge+ mantra: “Package early: package al"aysQ THE SOL<-,OG IS ,GDEED CON2IN6O6S PAC7A*IN* (+*ate!er the tool4 8/23/16 Continuous Packaging Architecture Pac)agers Local Build Server % or VE Build Project Build + 0epository metadata Project 'ocal build ?e!elopers 0M Build @arm #emote build 2may host VMs4 8/23/16 *oals ProCect67uilder main goal: help package continuously 7eing agnostic CS agnostic+ no 1CS 7ut guys itKs $'st century no+, S1G, C1S, Mercurial, G,- and G,-BS1G, S1D#### OS agnostic; Linux: 0PM, deb, e7uild, slack 7ased, ### '.&+ distro tuples made and counting % repositories or yum, urpmi, apt# Solaris p)g. HP6<F sd in roadmap Build environment agnostic; local, 1M 2REM<, D1M###4, 1E 2chroot, Doc)er, rpm7ootstrap, rinse, moc), debootstrap###4, 0M 27uild arm4 No pro)ect impact+ preser!es the md.sum o t*e delivered upstream sources# Can 7e completely external to the upstream proCect# A!oids duplication of code and metadata -HE SOL<-,OG IS ,GDEED CONTINUO6S PACKAGIN* 2with proCect67uilder#org S4 8/23/16 DEMONS-0ATION 8/23/16 *oodies Pro)ect8(uilder provides additional goodies Easy VMs/VEs/RMs installation/setup# <pdates not mandatory# Macro system with perl !aria7les to a!oid duplication of metadata S)eleton generation to help starting Manages package deli!ery up to your repository (ssh based4 wit* repository management (yum, apt, urpmi4 ,ntegration o tests in the process Manages patches/additional sources w*en not upstream C*ecks validity of packages built (lintian, rpmlint4 Easy creation of new versions