GitLab at DESY Git Services User Meeting Elena Gapon, DESY IT 31st August 2021 Topics Overview ◦ about GitLab Inc. ◦ DESY service GitLab ◦ project structure in GitLab ◦ Issues and Wiki ◦ container and package registry ◦ GitLab CI/CD service ◦ integration with other tools ◦ migrating repositories from Stash to GitLab ◦ last notes and outlook Questions and comments: https://codimd.desy.de/jt-azS0gQ9q5NAN1xfkpaw# GitLab at DESY | 31 August 2021 page 2/17 About GitLab Inc. Releases and Development ◦ all-remote company, across 67 countries ◦ monthly releases on 22nd ◦ public release kick-offs ◦ “run“ by GitLab Handbook, 8k+ pages ◦ an open source development process ◦ comprehensive documentation Community Business model and Versions ◦ > 3,000 contributiors, ~ 200 contributions/month ◦ community edition and enterprise paid tiers ◦ multiple communication channels (Youtube, Gitter,... meetups, feedback links) ◦ self-managed and SaaS current 1.429 employees 2011 2014 2016 ~ 30 m registered users first commit GitLab Inc. 140 employees ~ 100,000 organisations 2012 2015 2018 ? GitLab CI 9 employees 352 employees IPO GitLab.com GitLab at DESY | 31 August 2021 page 3/17 GitLab at DESY ◦ October 2020 - started as pilot ◦ since June 2021 - in production ◦ https://gitlab.desy.de ◦ Login with DESY account or via Helmholtz AAI ◦ ~ two updates / month, with short downtime ◦ Community Edition GitLab at DESY | 31 August 2021 page 4/17 Project in GitLab ◦ Project is created in a namespace This is different in ◦ Namespace is a group, a subgroup or a user name Bitbucket/Stash: ◦ Each repository is created in its own project each project in Bitbucket can contain multiple repositories Configuration options ◦ Project visibility public, private, internal ◦ Member roles guests, reporters, developers, maintainers, owners ◦ Repository default and protected branches, push restrictions, merge options ◦ Integration chats, notifications, 3rd party wiki, issue tracker, web hooks GitLab at DESY | 31 August 2021 page 5/17 Issues ◦ Labels ◦ Cross-linking merge requests, commits, other issues ◦ Multiple participants ◦ Multiple views on an issue list ◦ Add a Zoom meeting ◦ CSV export and import GitLab at DESY | 31 August 2021 page 6/17 Issue Boards GitLab at DESY | 31 August 2021 page 7/17 Wiki ◦ Mark down, incl. Math with KaTeX Diagramms and workflows ◦ WYSIWYG content editor ◦ RDoc, AsciiDoc, and Org ◦ Create: locally or in the web interface ◦ Cross-linking: merge requests, commits, issues ◦ Sidebar Navigation, TOC GitLab at DESY | 31 August 2021 page 8/17 Package Registry ◦ Package types: npm, Maven, PyPi, Ruby germs, NuGet, Conan, Composer Helm charts (Kubernetes applications) Generic packages ◦ public or private, protected by access rules ◦ Create and upload packages from the local machine, e.g. ◦ … or use GitLab CI/CD to build packages GitLab at DESY | 31 August 2021 page 9/17 GitLab CI/CD What is Continuous Integration/Delivery (CI/CD)? "Continuous Integration is the practice of integrating code into a shared repository and building/testing each change automatically, as early as possible - usually several times a day." "Continuous Delivery adds that the software can be released to production at any time, often by automatically pushing changes to a staging system." GitLab at DESY | 31 August 2021 page 10/17 GitLab CI/CD pipelines a pipeline with five jobs ◦ automate recurring tasks ◦ run scripts in a repeatable way ◦ version control for build scenarios ◦ overview of the build history ◦ publish and share build artifacts job log GitLab at DESY | 31 August 2021 page 11/17 Container Registry ◦ public or private registry for docker images ◦ create and upload an image from the local machine, e.g. ◦ … or use GitLab CI/CD to build container images [follow-up] GitLab at DESY | 31 August 2021 page 12/17 Migration from Stash to GitLab UI 1) create a personal access token in Stash with read permission 2) optionally: create a group where the repositories will be imported to 3) new project > import project > Bitbucket server API list repositories in Bitbucket: GET /rest/api/1.0/projects/{projectKey}/repos import into GitLab: POST api/v4/import/bitbucket_server → https://gitlab.desy.de/gitlab/migration-bitbucket GitLab at DESY | 31 August 2021 page 13/17 Integration with Other Tools ◦ Build: ◦ Wiki Jenkins, Buildkite, TeamCity, Drone, Bamboo ◦ Web Hooks on events: ◦ Issue Tracking: push, merge request, release Bugzilla, EWM, Redmine, YouTrack, Jira job, pipeline, deployment issue, Wiki page, comments ◦ Notifications: Campfire, Discord, Google Chat, Mattermost, MS Teams, Slack, Unity Circuit, Webex Teams ◦ REST API resources Pushover (e.g. on smartphone) access to ~every entity in GitLab Emails of push and pipeline status changes GitLab at DESY | 31 August 2021 page 14/17 Questions we received ◦ Best practices – don’t commit large or binary files → use LFS ◦ Git repository – for code not for data ◦ GitLab Pages – pilot available ◦ Singularity registry – use build artifacts as workaround Roadmap ◦ Sync GitLab groups with DESY LDAP group ◦ Support CI/CD jobs on Windows ◦ Make CVMFS available in CI/CD jobs ◦ On-going user documentation enhancement GitLab at DESY | 31 August 2021 page 15/17 Contact and Related Links Contact [email protected] Mattermost DESY / GitLab Links User documentation https://gitlab.desy.de/documentation/wiki/-/wikis/User Company https://about.gitlab.com/company/ GitLab DevOps Tools https://about.gitlab.com/devops-tools/ Project members' roles https://docs.gitlab.com/ee/user/permissions.html#project-members-permissions Packages https://docs.gitlab.com/ee/user/packages/pypi_repository/ https://docs.gitlab.com/ee/user/packages/workflows/project_registry.html Container visibility https://docs.gitlab.com/ee/user/packages/container_registry/index.html#container-registry-visibility-permissions Job artifacts https://docs.gitlab.com/ee/api/job_artifacts.html#get-job-artifacts Pipelines https://docs.gitlab.com/ee/ci/pipelines/ Learning Paths https://about.gitlab.com/learn/ ci/cd pipelines https://www.youtube.com/watch?v=Jav4vbUrqII (the best introduction) https://about.gitlab.com/blog/2020/12/10/basics-of-gitlab-ci-updated/ Singularity with GitLab https://vsoch.github.io/2018/gitlab-singularity-ci/ GitLab at DESY | 31 August 2021 page 16/17 Thank you GitLab at DESY | 31 August 2021 page 17/17.