Open Build Service in Debian
It is impressive how much time and resources a team can save
李健秋 Andrew Lee [email protected] 3/Feb/2018 Andrew Lee 李健秋
Years of experience on RPM packaging.
Debian Developer since 2009.
More than 5 years of experience work in Build & Infrastructure team in Collabora. [email protected]
Open First Agenda ● Overview ● Benefits of using OBS ● OBS features and workflow ● The OBS Architecture ● How to setup OBS ● Tips for optimization
3 Agenda ● Overview ● Benefits of using OBS ● OBS features and workflow ● The OBS Architecture ● How to setup OBS ● Tips for optimization
4 Overview ● Classic Packaging
1 build Repository
Source amd64 Binary Packages Debian: Jessie
5 Overview ● Classic Packaging
2 builds Repositories Repository Binary Packages Source amd64 Binary Packages Debian: Jessie, Stretch
6 Overview ● Classic Packaging
Repositories Repository 4 builds Binary Repository Packages Binary Packages Source amd64, i386 Binary Packages Debian: Jessie, Stretch
7 Overview ● Classic Packaging
Repositories Repository Binary Repository Packages MESS! Binary Repository Packages How many builds? Binary Repository Packages Binary Repository Packages Binary Packages Source Amd64, i386, armhf, arm64 Binary Packages Debian: Jessie, Stretch, Sid Ubuntu: 16.04, 16.10, 17.04
8 Let's fix the mess! Overview ● Packaging with OBS
Source Amd64, i386, armhf, arm64 Debian: Jessie, Stretch Ubuntu: 16.04, 16.10, 17.04
10 Overview ● Packaging with OBS DONE!
Source Amd64, i386, armhf, arm64 Debian: Jessie, Stretch Ubuntu: 16.04, 16.10, 17.04
11 Agenda ● Overview ● Benefits of using OBS ● OBS features and workflow ● The OBS Architecture ● How to setup OBS ● Tips for optimization
12 The Benefit of Using OBS ● For existing and new software projects: - One source to multiple distros and archs - Automatically published binary repositories - Workflow for packaging and releasing needs
● For packagers: - Automatic, consistent and reproducible build
● For user: - Easy install and keep upgraded via apt-get
13 The Benefit of Using OBS ● For existing and new software projects: - One source to multiple distros and archs - Automatically published binary repositories - Workflow for packaging and releasing needs
● For packagers: - Automatic, consistent and reproducible build
● For user: - Easy install and keep upgraded via apt-get
14 For existing and new software projects
One source to multiple distros and archs.
15 For existing and new software projects
Automatically published binary repositories.
16 For existing and new software projects
Workflow for packaging and releasing needs.
17 For existing and new software projects
Workflow for packaging and releasing needs.
18 The Benefit of Using OBS ● For existing and new software projects: - One source to multiple distros and archs - Automatically published binary repositories - Workflow for packaging and releasing needs
● For packagers: - Automatic, consistent and reproducible build
● For user: - Easy install and keep upgraded via apt-get
19 For Packagers
Automatic, consistent and reproducible build.
20 The Benefit of Using OBS ● For existing and new software projects: - One source to multiple distros and archs - Automatically published binary repositories - Workflow for packaging and releasing needs
● For packagers: - Automatic, consistent and reproducible build
● For user: - Easy install and keep upgraded via apt-get
21 For Users
Easy install and keep Upgrade via apt-get.
22 Add repo to source.list: For Users deb https://repositories.apertis.org/apertis/ 16.12 target development sdk
Easy install and keep Upgrade via apt-get.
23 Agenda ● Overview ● Benefits of using OBS ● OBS features and workflow ● The OBS Architecture ● How to setup OBS ● Tips for optimization
24 The OBS features and workflow
● Build against multiple distros and archs ● Download on Demand Repositories (DoD) ● Revision control for source package ● Automatic source package processing ● Access control for source repositories ● Request and Review system
25 The OBS features and workflow
● Build against multiple distros and archs ● Download on Demand Repositories (DoD) ● Revision control for source package ● Automatic source package processing ● Access control for source repositories ● Request and Review system
26 OBS features and workflow
Build on multiple distros and archs.
27 The OBS features and workflow
● Build against multiple distros and archs ● Download on Demand Repositories (DoD) ● Revision control for source package ● Automatic source package processing ● Access control for source repositories ● Request and Review system
28 OBS features and workflow
Download on Demand Repositories (DoD).
29 OBS features and workflow
Download on Demand Repositories (DoD).
30 OBS features and workflow
Download on Demand Repositories (DoD).
31 The OBS features and workflow
● Build against multiple distros and archs ● Download on Demand Repositories (DoD) ● Revision control for source package ● Automatic source package processing ● Access control for source repositories ● Request and Review system
32 OBS features and workflow
Revision control for source package.
33 The OBS features and workflow
● Build against multiple distros and archs ● Download on Demand Repositories (DoD) ● Revision control for source package ● Automatic source package processing ● Access control for source repositories ● Request and Review system
34 OBS features and workflow
Automatic source package processing.
35 OBS features and workflow
Automatic source package processing.
36 OBS features and workflow
Automatic source package processing.
37 OBS features and workflow
Automatic source package processing.
38 OBS features and workflow
Automatic source package processing.
39 The OBS features and workflow
● Build against multiple distros and archs ● Download on Demand Repositories (DoD) ● Revision control for source package ● Automatic source package processing ● Access control for source repositories ● Request and Review system
40 OBS features and workflow
Access control for Source repositories.
41 The OBS features and workflow
● Build against multiple distros and archs ● Download on Demand Repositories (DoD) ● Revision control for source package ● Automatic source package processing ● Access control for source repositories ● Request and Review system
42 OBS features and workflow
Request and Review system.
43 OBS features and workflow
Request and Review system.
44 OBS features and workflow
Request and Review system.
45 Agenda ● Overview ● Benefits of using OBS ● OBS features and workflow ● The OBS Architecture ● How to setup OBS ● Tips for optimization
46 The OBS Architecture
● Backend Services - obs-server - obs-worker
● Frontend Ruby Rails App - obs-api
● Command line tools - osc - osc-dput-plugin
47 The OBS Architecture
48 The OBS Architecture
● Backend Services - obs-server - obs-worker
● Frontend Ruby Rails App - obs-api
● Command line tools - osc - osc-dput-plugin
49 The OBS Architecture
Backend Services.
50 The OBS Architecture
● Backend Services - obs-server - obs-worker
● Frontend Ruby Rails App - obs-api
● Command line tools - osc - osc-dput-plugin
51 The OBS Architecture
52 The OBS Architecture
Frontend Ruby Rails App.
53 The OBS Architecture
● Backend Services - obs-server - obs-worker
● Frontend Ruby Rails App - obs-api
● Command line tools - osc - osc-dput-plugin
54 The OBS Architecture
55 The OBS Architecture
Command line tools: osc and osc-dput-plugin
56 Let’s look at an example.
57 The OBS Architecture
Command line tools: osc and osc-dput-plugin
58 The OBS Architecture
Command line tools: osc and osc-dput-plugin
59 The OBS Architecture
Command line tools: osc and osc-dput-plugin
60 The OBS Architecture
Command line tools: osc and osc-dput-plugin
61 The OBS Architecture
Command line tools: osc and osc-dput-plugin
62 The OBS Architecture
Command line tools: osc and osc-dput-plugin
63 The OBS Architecture
Command line tools: osc and osc-dput-plugin
64 Modify your package as usual. Update changelog as usual for a new revision. The OBS Architecture And then generate source package: $ dpkg-buildpackage -S -us -uc -d
Command line tools: osc and osc-dput-plugin
65 The OBS Architecture
Command line tools: osc and osc-dput-plugin
66 The OBS Architecture
Command line tools: osc and osc-dput-plugin
67 The OBS Architecture
Command line tools: osc and osc-dput-plugin
68 The OBS Architecture
Command line tools: osc and osc-dput-plugin
69 The OBS Architecture
Command line tools: osc and osc-dput-plugin
70 The OBS Architecture
Command line tools: osc and osc-dput-plugin
71 The OBS Architecture
Command line tools: osc and osc-dput-plugin
72 The OBS Architecture
Command line tools: osc and osc-dput-plugin
73 For more information: $ osc help The OBS or Architecture $ man osc
Command line tools: osc and osc-dput-plugin
74 The OBS Architecture
Command line tools: osc and osc-dput-plugin
75 Agenda ● Overview ● Benefits of using OBS ● OBS features and workflow ● The OBS Architecture ● How to setup OBS ● Tips for optimization
76 How to Setup OBS
● OBS is now available in Debian Stretch! \o/
● You need to have Debian Stretch or Sid apt-get install [obs-server | obs-api | obs-worker]
● Read the README.Debian for the setup
77 How to Setup OBS
● OBS is now available in Debian Stretch! \o/
● You need to have Debian Stretch or Sid apt-get install [obs-server | obs-api | obs-worker]
● Read the README.Debian for the setup
78 OBS is now in Debian Stretch
Years of Debian Packaging efforts for OBS dependencies.
79 OBS is now in Debian Stretch \o/
80 How to Setup OBS
● OBS is now available in Debian Stretch! \o/
● You need to have Debian Stretch or Sid apt-get install [obs-server | obs-api | obs-worker]
● Read the README.Debian for the setup
● OBS in container
81 OBS install
Install obs-server The backend services
82 OBS install
Install obs-api The ruby rails app frontend
83 OBS install
Install obs-worker The builders
84 How to Setup OBS
● OBS is now available in Debian Stretch! \o/
● You need to have Debian Stretch or Sid apt-get install [obs-server | obs-api | obs-worker]
● Read the README.Debian for the setup
● OBS in container
85 OBS Setup Read: /usr/share/obs-api/README.Debian
86 OBS in containers
We are working on that: https://gitlab.collabora.com/obs/do cker-obs WIP docker images for obs-server, obs-api and obs-worker containers.
87 OBS in containers
● We are working on that:
https://gitlab.collabora.com/obs/docker-obs
88 Agenda ● Overview ● Benefits of using OBS ● OBS features and workflow ● The OBS Architecture ● How to setup OBS ● Tips for optimization
89 Tips for Optimization
● Merge-our-misc (MoM) ● Bug tracker hook ● Integrate with Jenkins to build package from git ● Integrate with LAVA to run autotest on actual hardware
90 Tips for Optimization
● Merge-our-misc (MoM) ● Bug tracker hook ● Integrate with Jenkins to build package from git ● Integrate with LAVA to run autotest on actual hardware
91 Merge-our-misc
Automated continuous package integration.
92 Merge-our-misc
3-way merge and submit request with new upstream package with local modifications.
93 Tips for Optimization
● Merge-our-misc (MoM) ● Bug tracker hook ● Integrate with Jenkins to build package from git ● Integrate with LAVA to run autotest on actual hardware
94 Bug tracker hook
Update bug status after package build.
95 Tips for Optimization
● Merge-our-misc (MoM) ● Bug tracker hook ● Integrate with Jenkins to build package from git ● Integrate with LAVA to run autotest on actual hardware
96 Integrate with Jenkins
Build packages from git.
97 Integrate with Jenkins
Build packages from git.
98 Integrate with Jenkins
Build packages from git.
99 Integrate with Jenkins
Build images from git.
100 Integrate with Jenkins
Build images from git.
101 Tips for Optimization
● Merge-our-misc (MoM) ● Bug tracker hook ● Integrate with Jenkins to build package from git ● Integrate with LAVA to run autotest on actual hardware
102 Integrate with Jenkins + LAVA
Run autotest on actual hardware.
103 More details
● Build a infrastructure of a Debian derivative – a Debian-derived platform for infotainment in automotive vehicles ● Tomorrow 9:00am ● Same room by Andrew Shadura
104 Recap ● Overview ● Benefits of using OBS ● OBS features and workflow ● The OBS Architecture ● How to setup OBS ● Tips for optimization
105 We're hiring: col.la/careers
Any questions?
106 Thank you for everyone who was involved
Andrew Shadura, Emanuele Aina, Emilio Pozuelo Monfort, Gustavo Noronha Silva, Gustavo Padovan, Héctor Orón Martínez, Justin Kim, Jordi Mallach, Luis Francisco Araujo Camarillo, Mark Filion, Simon McVittie, Sjoerd Simons, Travis Reitter, Yohei Yasukawa, 劉建緯 and many others.
107 Thank you!
李健秋 Andrew Lee [email protected] www.collabora.com
108