GitLab at DESY

Git Services User Meeting

Elena Gapon, DESY IT 31st August 2021 Topics Overview

◦ about GitLab Inc.

◦ DESY service GitLab

◦ project structure in GitLab

◦ Issues and Wiki

◦ container and package registry

◦ GitLab CI/CD service

◦ integration with other tools

◦ migrating repositories from Stash to GitLab

◦ last notes and outlook

Questions and comments: https://codimd.desy.de/jt-azS0gQ9q5NAN1xfkpaw#

GitLab at DESY | 31 August 2021 page 2/17 About GitLab Inc. Releases and Development

◦ all-remote company, across 67 countries ◦ monthly releases on 22nd ◦ public release kick-offs ◦ “run“ by GitLab Handbook, 8k+ pages ◦ an open source development process ◦ comprehensive documentation

Community Business model and Versions ◦ > 3,000 contributiors, ~ 200 contributions/month

◦ community edition and enterprise paid tiers ◦ multiple communication channels (Youtube, Gitter,... meetups, feedback links) ◦ self-managed and SaaS current 1.429 employees

2011 2014 2016 ~ 30 m registered users first commit GitLab Inc. 140 employees ~ 100,000 organisations 2012 2015 2018 ? GitLab CI 9 employees 352 employees IPO GitLab.com

GitLab at DESY | 31 August 2021 page 3/17 GitLab at DESY

◦ October 2020 - started as pilot

◦ since June 2021 - in production

◦ https://gitlab.desy.de

◦ Login with DESY account or via Helmholtz AAI ◦ ~ two updates / month, with short downtime

◦ Community Edition

GitLab at DESY | 31 August 2021 page 4/17 Project in GitLab

◦ Project is created in a namespace This is different in ◦ Namespace is a group, a subgroup or a user name /Stash: ◦ Each repository is created in its own project each project in Bitbucket can contain multiple repositories

Configuration options

◦ Project visibility public, private, internal

◦ Member roles guests, reporters, developers, maintainers, owners

◦ Repository default and protected branches, push restrictions, merge options

◦ Integration chats, notifications, 3rd party wiki, issue tracker, web hooks

GitLab at DESY | 31 August 2021 page 5/17 Issues

◦ Labels

◦ Cross-linking

 merge requests, commits, other issues

◦ Multiple participants

◦ Multiple views on an issue list

◦ Add a Zoom meeting

◦ CSV export and import

GitLab at DESY | 31 August 2021 page 6/17 Issue Boards

GitLab at DESY | 31 August 2021 page 7/17 Wiki

◦ Mark down, incl.

 Math with KaTeX

 Diagramms and workflows

◦ WYSIWYG content editor

◦ RDoc, AsciiDoc, and Org

◦ Create: locally or in the web interface

◦ Cross-linking: merge requests, commits, issues

◦ Sidebar Navigation, TOC

GitLab at DESY | 31 August 2021 page 8/17 Package Registry

◦ Package types:

 npm, Maven, PyPi, Ruby germs, NuGet, Conan, Composer

 Helm charts (Kubernetes applications)

 Generic packages

◦ public or private, protected by access rules

◦ Create and upload packages from the local machine, e.g.

◦ … or use GitLab CI/CD to build packages

GitLab at DESY | 31 August 2021 page 9/17 GitLab CI/CD

What is Continuous Integration/Delivery (CI/CD)?

"Continuous Integration is the practice of integrating code into a shared repository and building/testing each change automatically, as early as possible - usually several times a day."

"Continuous Delivery adds that the software can be released to production at any time, often by automatically pushing changes to a staging system."

GitLab at DESY | 31 August 2021 page 10/17 GitLab CI/CD

pipelines

a pipeline with five jobs ◦ automate recurring tasks

◦ run scripts in a repeatable way

◦ version control for build scenarios

◦ overview of the build history

◦ publish and share build artifacts

job log GitLab at DESY | 31 August 2021 page 11/17 Container Registry

◦ public or private registry for docker images

◦ create and upload an image from the local machine, e.g.

◦ … or use GitLab CI/CD to build container images

[follow-up]

GitLab at DESY | 31 August 2021 page 12/17 Migration from Stash to GitLab

UI

1) create a personal access token in Stash with read permission

2) optionally: create a group where the repositories will be imported to

3) new project > import project > Bitbucket server

API

 list repositories in Bitbucket: GET /rest/api/1.0/projects/{projectKey}/repos

 import into GitLab: POST api/v4/import/bitbucket_server

 → https://gitlab.desy.de/gitlab/migration-bitbucket

GitLab at DESY | 31 August 2021 page 13/17 Integration with Other Tools

◦ Build: ◦ Wiki  Jenkins, Buildkite, TeamCity, Drone, Bamboo

◦ Web Hooks on events: ◦ Issue Tracking:  push, merge request, release  , EWM, , YouTrack,  job, pipeline, deployment

 issue, Wiki page, comments ◦ Notifications:

 Campfire, Discord, Google Chat, Mattermost, MS Teams, , Unity Circuit, Webex Teams ◦ REST API resources

  Pushover (e.g. on smartphone) access to ~every entity in GitLab

 Emails of push and pipeline status changes

GitLab at DESY | 31 August 2021 page 14/17 Questions we received

◦ Best practices – don’t commit large or binary files → use LFS

◦ repository – for code not for data

◦ GitLab Pages – pilot available

◦ Singularity registry – use build artifacts as workaround

Roadmap

◦ Sync GitLab groups with DESY LDAP group

◦ Support CI/CD jobs on Windows

◦ Make CVMFS available in CI/CD jobs

◦ On-going user documentation enhancement

GitLab at DESY | 31 August 2021 page 15/17 Contact and Related Links

Contact [email protected]

Mattermost DESY / GitLab

Links

User documentation https://gitlab.desy.de/documentation/wiki/-/wikis/User Company https://about.gitlab.com/company/ GitLab DevOps Tools https://about.gitlab.com/devops-tools/ Project members' roles https://docs.gitlab.com/ee/user/permissions.html#project-members-permissions Packages https://docs.gitlab.com/ee/user/packages/pypi_repository/ https://docs.gitlab.com/ee/user/packages/workflows/project_registry.html Container visibility https://docs.gitlab.com/ee/user/packages/container_registry/index.html#container-registry-visibility-permissions Job artifacts https://docs.gitlab.com/ee/api/job_artifacts.html#get-job-artifacts Pipelines https://docs.gitlab.com/ee/ci/pipelines/ Learning Paths https://about.gitlab.com/learn/ ci/cd pipelines https://www.youtube.com/watch?v=Jav4vbUrqII (the best introduction) https://about.gitlab.com/blog/2020/12/10/basics-of-gitlab-ci-updated/ Singularity with GitLab https://vsoch.github.io/2018/gitlab-singularity-ci/

GitLab at DESY | 31 August 2021 page 16/17 Thank you

GitLab at DESY | 31 August 2021 page 17/17