Machine Learning Cloud Getting Started Guide

Machine Learning Cloud Getting started guide

March 4th, 2020 Tübingen Outline • Who, what, how, … what again? o Our Cloud Infrastructure o Openstack, Storage, ... • General basics o SSH o DNS & TLS, … • Openstack basics: images, flavors, security groups, networks, floating IPs • Usage Terms, • VMs & Baremetal basics • How to create, connect and use • VM Hardening • Difference VM vs. Baremetal • GPUs (Driver, CUDA, within Docker) • Storage: Cinder, Quobyte, S3, WebDAV • Cloud-based services: Gitlab, Nextcloud, … • Docker • Support Who are we ...

[email protected] [email protected] [email protected] [email protected]

07071 - 29 70282 Openstack – ML-Cloud

Physical infrastructure ML-Cloud - TTR2 server room

• 14x 1G Switches & 10x 40G Switches • 2x Provisioningmachines • 2x Loadbalancer • 3x Openstack controllers

• 11x Storage servers (~1.8 PB, Quobyte software) • 12x GPU - Hypervisors with 4x Tesla V100 32GB RAM (NVLINK possible) • 7 Baremetal machines with 8x RTX 2080Ti 11GB RAM Insufficient cooling: Automatic server shutdown possibly without warning Openstack – what it is

Most used open source IaaS platform o based on python, sql, messaging, http REST APIs, … o started by NASA & Rackspace, currently involved Companies: Red Hat/Ubuntu/Mirantis/... o commercially used (Telekom, …): Projects, Quotas o 2 major releases a year

Multiple Services/Components o required: Keystone(Identity), Nova(Compute), Neutron(Network), Glance(Image) o optional: Horizon(Dashboard), Cinder(Blockstorage), Heat(Orchestration), Magnum(Container Orchestration), Barbican(Key Management), Swift(Objectstorage), …

Some cons … o complex installation & configuration o sometimes confusing documentation o updates & long term support (-->Sardina) Openstack – How it works Openstack – ML-Cloud Storage - Quobyte

LDAP

Load - balancers

Switches Openstack – how to use/access it

• WebGUI: https://mlcloud.uni-tuebingen.de

• Commandline tools: (apt|yum) install python-openstackclient openstack help

• python modules (python-openstacksdk)

• or the http API itself https://mlcloud.uni-tuebingen.de/identity/v3/auth/tokens POST { "auth": { "identity": { "methods": [ "password" ], "password": { "user": { "name": "admin", "domain": { "name": "Default" }, "password": "devstacker" } } } } } General Basics - SSH

• secure shell • connecting to remote machine • Being able to work on remote machine as you were sitting right in front of it • Linux: $ ssh • Windows: use e.g Putty Client Generating ssh keys (Linux)

• If you are using a Linux machine, open a terminal and enter the following: • $ ssh-keygen -t rsa -b 2048 • This creates a new ssh key. When you're prompted to "Enter a file in which to save the key," press Enter. • > Enter a file in which to save the key (/home/you/.ssh/id_rsa): [ Press Enter ] • At the next prompt, type a secure passphrase. • > Enter passphrase (empty for no passphrase): [Type a passphrase] • > Enter same passphrase again: [Type passphrase again] • Now you have created a public ssh key which can be found in /home/you/.ssh/id_rsa.pub • The public key is pasted into Openstack. Generating ssh keys (Windows)

• If using Windows, use puttygen.exe to generate ssh keys. • Select "RSA" as type of key and enter "2048" or more for number of bits. • Click on generate. Generating ssh keys (Windows)

• Enter a secure passphrase. • Click on "Save public key" to save the key on your disk. • Click on "Save private key" to save it in .ppk format on your disk. • You will use these keys with Putty to open a ssh session. • Now copy the generated public key and paste it into Openstack. General Basics - SSH

You are using Windows: You are using Linux or MacOS:

1. Start Putty # generate key 2. Go to Session and type Hostname ssh-keygen -f cloud-key 3. Get IP from Openstack Dashboard 4. You can save a session and use it again # login ssh -i cloud-key centos@ 5. Click Open to start ssh session 6. Type in the password for your ssh keypair when prompted Putty session

[email protected] General Basics - port tunneling (OpenSSH)

Route

Network A Network B some host/jumphost • IP = 134.2.168.180 • Ssh connect via [email protected] VM

Your workstation Machine/VM you want to login to --> localhost with Port 8444 open • Not in your network • IP = 192.168.212.46 • Connection to Website over HTTPS (Port 443)

ssh –L PORT_LOCALHOST:IP_YOU_WANT_TO_BE_CONNECTED_WITH:PORT_ON_THAT_MACHINE jumphost • ssh –L 8444:192.168.212.46:443 [email protected] • Open a browser & enter: https://localhost:8444 General Basics - port tunneling (Putty)

1. Start Putty. 2. Go to Session and type Hostname/IP of the jump host. 3. You can save a session and use it again later. 4. Go to Connection > SSH > Tunnels 5. Enter a port of your choice in Source Port 6. Enter the destination address (the machine accessible from the jump host but not your local machine) along with the port on the destination server. 7. Click Add to add it to the list of tunnels. 8. Click Open to start ssh tunnel. 9. Type in the password for your ssh keypair when prompted. General Basics - Shell

man manual page for whoami which user do I work as right now cd change directory ls list what is in present directory pwd print present working directory vim OR nano enter files and be able to edit them less enter file and just read cat concatenate files and print on the standard output sudo 'superuser do' for getting root privileges mv File or directory move cp file or directory copy rm remove a file mkdir make a new directory mount / umount mount/unmount a filesystem found on a device chmod / chown modify permissions/ownership of files or directories find find files in a directory hierarchy ip [a|r] print network adresse OR routes ... and many more Responsibilites

Users deploying VMs must take care of them:

VM owners must harden their VMs to avoid/prevent abuse.

No password or ssh-key sharing!

Errors, flaws or loopholes must be reported.

All illegal activities will lead to legal measures. General Basics – DNS/TLS

Domain Name Service

• Translates IP addresses to domain names and vice versa • Not 1 DNS but many hierarchically ordered: *.mlcloud.uni-tuebingen.de controlled by ZDV networkers (DFN: no wildcards) • domain records only make real sense with TLS certificates

Transport Layer Security

• Secures network traffic • Use TLS 1.2 or 1.3: previous versions are unsafe/deprecated • TLS certificates used for HTTPS and other protocols(not ssh) • Provides authentification (and starts encryption) • Certificate are associated with one or more domains (incl. wildcards) • requireDNS entries • DFN > lets-encrypt > own CA > self-signed > no encrytion General Basics - Security

SSH Keys • Use ssh keys with password, encrytion type = rsa and encryption depth ≥ 2048bit ssh-keygen -t rsa -b 4096 • Keep your private key secret! Do not share it with other people!

Networks & Firewalls • Use whitelistingin every context with firewalls (security groups) --> block everything and allow only connections from and to Ports / IP-address-ranges you need

Downloads • Downloads only from reliable sites, whenever possible use https • If available: check (GPG) signatures after downloadingrepos, software, etc.

Regularly perform security updates Stop and deactivate services and containers you no longer need Openstack Basics Projects

• Permissions • Users/Members • Ownership & Quotas

Flavors Images Hardware setup of your VM Software setup of your VM

• defines ressources --> • Linux distributionswe support: • CPUs • CentOS • GPUs (names) • Ubuntu LTS • RAM • we can create additional flavors on • Your own images demand. • Public/Private/Shared Openstack Basics - Networks local_network 192.168.0.0/24 We use ... • a physical 40G network: Openstack, Storage • VLANs for securing the physical network by dividing it into separate virtual networks. bmprov Openstack Networks 192.168.212.0/22 • VXLAN: self-service, storage limitations, no BM • FLAT/VLAN: provider - can be set up for groups and sensitive projects individually, fast qbstorage local (not routed) IPv4 networks 192.168.128.0/22 • 10.0.0.0/8: 10.0.0.0 – 10.255.255.255 • 172.16.0.0/12: 172.16.0.0 – 172.31.255.255 • 192.168.0.0/16: 192.168.0.0 – 192.168.255.255 osprovider public IPv4 network: osprovider 134.2.168.0/24 Openstack Basics - Access

Floating IPs: "Direct" Access Jumphost: "Indirect" Access

• IP of specific network: public/bmprov • Specific VM with a public IP for access

• Quota: Each project has a limited amount • May have the only purpose of getting access to other VMs or baremetal machines you cannot • Can be allocated to a project reach directly for security or other reasons

• Can be associated and disassociated to VMs: • Different access methods: The VM doesnt know about it 1. ssh to Jumphost via Floating IP, then ssh to internal machine • Used for direct access from remote host / 2. port forwarding different network 3. proxy internal ports onto public ip of the jumphost • Not working for baremetal machines Openstack Basics - Security Groups

• Using security groups you restrict access to your VMs • A security group consists of multiple network access rules • A simplified firewall for VMs: filter source/protocols/ports

• When you launch an instance, you can assign one or more security groups to it. • If you do not specify security groups, new instances are automatically assigned to the default security group.

You are in charge to secure your VMs!

• Create new security groups. • Add or delete rules to existing security groups (be aware of other machines) • Whitelisting: Only allow what you really need, block the rest. Project > Network > Security Groups Security Groups

Protocol: e.g. TCP for SSH, ICMP for ping or UDP.

Destination port on virtual machine: Define a port range. To open a single port only, enter the same value twice. ICMP does not support ports; instead, you enter values to define the codes and types of ICMP traffic to be allowed.

Source of traffic: Enable traffic to instances from • Networks (IP Ranges) • Members of other security groups: Separate VMs within the same network Security Groups

• Security groups are project specific • The default security group contains ssh&icmp and is managed by admins

• VMs can use multiple security groups. It can cause problems if the rules of different security groups overlap: e.g. if outgoing IPv4 connections are allowed in multiple used security groups.

• We therefore recommended to use one base security group like default and additional extending security groups that add functionality: e.g. +http, https which would only whitelist tcp port 80 and 443. • Be descriptive, distinguish extending security groups from base groups e.g. add + to the name VM Basics

Unified Identity Management • LDAP credentials for • Nextcloud • Gitlab • Openstack

• Use your LDAP credentials (initially set new password via Nextcloud) • Openstack: use Domain "ldap" Switch to English Adding ssh keys to Openstack

• Go to Project > Compute > Key Pairs • Click on "Import Public Key". • Either paste the public key manually or load it from the public key file you created. • /home/you/.ssh/id_rsa.pub VM Basics

• A VM behaves like a remote Linux machine • It is accessible through its public IP and ssh using keys VM Basics – how to create a VM