Operating System Installation for Cloud

Home , Aptitude (software), OpenSUSE, Rolling release

Masaryk University Faculty of Informatics

Bachelor’s Thesis

Martin Lacko

Brno, Fall 2017

Masaryk University Faculty of Informatics

Operating System Installation for Cloud

Bachelor’s Thesis

Martin Lacko

Brno, Fall 2017

This is where a copy of the official signed thesis assignment and a copy ofthe Statement of an Author is located in the printed version of the document.

Declaration

Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source.

Martin Lacko

Advisor: RNDr. Jan Kasprzak, Ph.D.

Acknowledgements

Foremost, special thanks go to RNDr. Jan Kasprzak, PhD. for his advice and guidance, during the writing of this thesis; and also my manager Stanislav Kozina for random tips and tricks. Lastly, I would like to thank my family and friends for their support.

iii Abstract

This thesis aims to explore methods for virtual machine image creation for Faculty of Informatics private IaaS cloud server. The thesis describes suitability of installed, upgraded and cloud-ready images for Stratus.FI server, focusing on the process of unattended installation, as well as preparing resources for various automated installation software.

iv Keywords operating system, installation, cloud, OpenNebula, Stratus.FI

Contents

Introduction 1

1 Stratus.FI 3 1.1 Disk Images ...... 3 1.2 Templates ...... 4 1.3 Virtual Networks ...... 5 1.4 Contextualization ...... 6 1.5 Virtual Machine Template creation ...... 7 1.5.1 Template requirements ...... 7 1.5.2 Installation in Sunstone ...... 8 1.5.3 Upgrading existing templates ...... 10 1.5.4 Local Installation ...... 11 1.5.5 Cloud-based images ...... 11

2 Linux 13 2.1 Fedora ...... 13 2.1.1 Installation ...... 13 2.1.2 Upgrade ...... 14 2.1.3 Cloud images ...... 14 2.1.4 CentOS ...... 15 2.2 Debian ...... 16 2.2.1 Installation ...... 16 2.2.2 Upgrade ...... 18 2.2.3 Cloud images ...... 18 2.2.4 Ubuntu ...... 19 2.2.5 Linux Mint ...... 20 2.3 OpenSUSE ...... 20 2.3.1 Installation ...... 21 2.3.2 Upgrade ...... 21 2.4 ArchLinux ...... 22 2.4.1 Installation ...... 22 2.4.2 Cloud images ...... 24 2.5 Gentoo ...... 24

3 Microsoft Windows 27 3.1 Installation ...... 28

vii 3.2 Upgrade ...... 29

4 FreeBSD 31 4.1 Installation ...... 31 4.2 Upgrade ...... 32

5 Conclusion 33 5.1 Future work ...... 33

Bibliography 35

viii List of Figures

1.1 Non-persistent image lifetime 5 1.2 Debian virtual machine template 6 2.1 Fedora upgrade script 14 2.2 Modifying packages of Linux image 15 2.3 Debian partitioning recipe 18 2.4 Debian upgrade commands 18 2.5 OpenSUSE consensus for the system update 22 2.6 Setting up static IP addresses 23 2.7 Gentoo one-context installation 25

Introduction

Cloud computing, or cloud, is a model for sharing a pool of computing resources, which allows for the fast deployment of client services with minimal effort (document NIST SP 800-145 [1] which specifies cloud computing was published in the September 2011). Among the most significant benefits of cloud computing in comparison to dedicated servers are on-demand deployment, allowing a consumer to acquire resources without the necessity for human inter- action; and that, resources can be elastically provisioned and released, even automatically in some cases. From the business point of view, cloud services can be easily moni- tored for automatic control and resource optimisation, as well as usage monitoring and reporting for any monetisation models.

Cloud computing can be divided into three primary service models according to the scope of the offered resource environment:

∙ Infrastructure as a service (IaaS) is the most customizable of the three and is able to deploy and run any arbitrary software or operating systems. IaaS sheds the requirement for infrastructure management from the customer and provides computing, storage and/or networking resources, usually in the form of virtual machines.

∙ Platform as a service (PaaS) can most easily be described as a cloud-based middleware for developers and can easily deploy consumer applications created using tools supported by the provider. The underlying infrastructure cannot be modified or controlled by the customer, but rather the emphasis is placed on customisation of the application-hosted environment.

∙ Software as a service (SaaS), or software on-demand, is a provider’s application running on a cloud infrastructure. The application is accessible usually through a thin client interface, such as a web browser. The customer cannot manage the infrastructure or any aspects of the application environment, just a limited set of application configuration settings.

1 Cloud computing can also be differentiated according to access to the services. In short, a private cloud is an infrastructure for the exclusive use of a single organisation; and the public cloud is for open use by the general public, with additional ‘hybrid’ types, somewhere in between or representing a combination of both. Only the infrastructure as a service model allows for the installation of a custom operating system on the provided resources, which is the primary goal of this thesis. There are, naturally, many different open- source IaaS platforms (such as OpenStack, CloudStack, Eucalyptus and others), for private cloud computing infrastructures. OpenNebula is used as a faculty private cloud computing cluster – Stratus.FI.

2 1 Stratus.FI

As described on its technical page1, the primary function of Stratus.FI is to provide a quick testing or deployment environment for applications and also to allow students to try out different operating systems. OpenNebula offers a comprehensive user interface in the formof the Sunstone [2, chapter Sunstone Setup], which provides an all-in- one solution for most of the users and is the primary interface of the Stratus.FI server. Because of this, the focus will be on utilising this interface as much as possible. The Sunstone has two views: the more straightforward cloud view and an advanced user view. The view mode can be changed in the user panel (top right corner of the Sunstone interface) and all further work described in this thesis will be in reference to the Sunstone user view.

1.1 Disk Images

Images in OpenNebula are designed to hold operating systems or data used by virtual machine instances or shared by users. There are six types of images: ∙ OS: A bootable disk that contains an operating system. Every template must define at least one image of this type. ∙ CDROM: A read-only data format. In terms of this thesis, CDROM images are used as operating system installation media. ∙ DATABLOCK: Data storage. These can be created from virtual machine data or can serve as an empty disk. ∙ CONTEXT: Plain files available on the contextualisation disk, ready to be used as additional configuration files from the cluster. ∙ KERNEL and RAMDISK are specialised plain text files used by virtual machines for custom booting. In this thesis, neither of these two types is used in any scenario.

1. https://www.fi.muni.cz/tech/unix/stratus.html.cs

3 1. Stratus.FI

KERNEL, RAMDISK and CONTEXT types cannot be used as virtual machine disks, and are listed in the Sunstone interface under files tab. While OS, DATABLOCK and CDROM are created or uploaded through the storage tab or cloned from an existing image.

OpenNebula allows for the specification of an image persistence, which must be manually managed during template creation. Operat- ing system installation requires a persistent image which is used to hold an installed operating system, while published templates must use non-persistent disks so that they can be used by multiple users. The differences between persistent and non-persistent images lie in the way they handle changes and in the disk instantiation rules. A persistent disk stores every change for its whole lifetime, but can be used only in one virtual machine instance at a time, whereas non- persistent disks store changes for an instance lifetime only and can be used by multiple virtual machines at once, allowing for the expansion of disk capacity during instantiation and enabling the use of the thin provisioning mechanism. Thin provisioning is a disk storage optimisation method that gives the appearance of more space than currently available, optimizing the efficiency of cloud storage area usage. The whole lifecycle of a non-persistent image is shown in figure 1.1 (copied from documentation page[2, chapter Virtual Machine Im- ages]).

1.2 Templates

Templates describe how the virtual machines should be created during instantiation. As detailed in the official documentation[2, chapter Virtual Machine Templates], a configuration should contain at least a name, capacity, disks and networking settings. OpenNebula templates are in the simple name=value format, although XML format is also supported. Templates similar to one described in figure 1.2 will be used primarily in the virtual machines created for this thesis, should any substantial difference be required for a specific system this will be noted in the text.

4 1. Stratus.FI

Figure 1.1: Non-persistent image lifetime

1.3 Virtual Networks

A virtual machine cannot be connected directly to a physical network. Instead, the host should be connected to one or more Virtual Networks that serve as mappers for physical network, with fine-grained control over the connected virtual machines. The Virtual Network consists of: underlying physical infrastructure for support with the network driver; logical address space and guest configuration attributes.

5 1. Stratus.FI CONTEXT = [ CRYPTED_PASSWORD = "$USER[CRYPTED_PASSWORD]", NETWORK = "YES", SSH_PUBLIC_KEY = "$USER[SSH_PUBLIC_KEY]" ] CPU = "0.4" DESCRIPTION = "Debian 9.3.0 x86_64, 4 GB" DISK = [ IMAGE = "Debian 9.3.0 OS", IMAGE_UNAME = "xlacko2" ] DISK = [ SIZE = "2048", TYPE = "fs" ] GRAPHICS = [ LISTEN = "0.0.0.0", TYPE = "VNC" ] HYPERVISOR = "kvm" INPUTS_ORDER = "" LOGO = "images/logos/debian.png" MEMORY = "2048" NIC = [ NETWORK = "503-usrpriv", NETWORK_UNAME = "oneadmin" ] OS = [ BOOT = "disk0" ] VCPU = "1" Figure 1.2: Debian virtual machine template

In Stratus.FI, the 508-usrpriv Virtual Network is dedicated to students.

1.4 Contextualization

Contextualization in OpenNebula refers to the process and effect of creating a user-specific environment (context) inside of an instantiated virtual machine. The most prominent features of contextualization are the SSH authentication setup, network and custom environment configuration and initialization scripts and custom file provision.

6 1. Stratus.FI

Regarding faculty students, the required settings are: 1. CRYPTED_PASSWORD attribute, used as the root password for *nix systems. 2. WIN_USER attribute, user name for new users in Windows systems. 3. WIN_PASSWORD attribute, password for the Windows user. 4. Public SSH key, used to access the virtual machines remotely. This value should be set in the “Auth” tab of the user settings in the Sunstone interface. OpenNebula builds an image containing the context for each virtual machine (which has enabled contextualization) and makes this image available via the IDE driver as a virtual CD-ROM, labeled CONTEXT, ready to be mounted in the operating system. The VM configuration, on the other hand, must be undertaken on the guest machine, for which purpose there are official one-context packages available2 for the various operating systems, which will be discussed in system installation chapters, later on.

1.5 Virtual Machine Template creation

The thesis will focus on clean installation through the Sunstone interface. There are three alternative methods which will also be described in this section, as they may be more suitable for certain operating systems.

1.5.1 Template requirements Cloud system creation differs from a bare-metal installation asthe different use of a virtual machine and the cloud cluster settings mustbe kept in mind. Physical servers commonly run a plethora of applications for various reasons, as well as multiple services. That is usually not the case with cloud virtual machines, which are expected to run only a single application or are used for a sole purpose. Virtual machine

2. https://github.com/OpenNebula/addon-context-linux

7 1. Stratus.FI installation should reflect this difference to ensure efficient useof cluster resources. To address a virtual machines’ single purpose, it is a good start with minimal system installation. One of the unique points of OpenNebula contextualization is the way it handles swap space. Every machine with defined swap in the template will have a separate volatile disk mounted for this purpose; volatile in this context means that there are no guarantees regarding disk persistence after the host is shut down, only that there will be such disk available after boot. The contextualization package handles all problems and the requirements in terms of swap space, but smart installation softwares recommend (or worse, enforce) separate swap partitioning during the installation. The contextualization package handles all OpenNebula-specific contextualization settings, and thus it is clear that the new system should include it, in a version later than v5.0.3, if possible. An additional recommendation is to add the Faculty of Informatics SSL certificate3 and remove all extra settings or artefacts (if possible) after installation, such as commonly created user accounts or any unneces- sary additional systems.

1.5.2 Installation in Sunstone Clean install through the Sunstone should be preferred method for template creation, for most of the Stratus.FI users, because it is the most convenient, as it does not require any additional software and fully utilises the thin image provisioning of OpenNebula. The whole process is, for the most part, straightforward: 1. Obtain the operating system installation media and upload it to Stratus.FI through the Storage > Images page as a CDROM image. 2. Create a suitable image. At the time of writing this thesis it is not possible to create empty OS from the creation Sunstone page, although this problem is easily fixed by initially creating an empty DATABLOCK image and then changing it to OS type. For installation, the disk must be marked as persistent in the creation page; otherwise, the whole process will be lost. Fine

3. https://www.fi.muni.cz/tech/unix/fadmin-cert.html.en

8 1. Stratus.FI

tuning OS image advanced settings is a way of improving the efficient usage of Stratus.FI resources. The BUS specifies the disk interface, and it is recommended to use one of the virtio drivers[2, chapter KVM Driver]: older virtio-blk or newer virtio-scsi driver, named Virtio and SCSI in the BUS selection menu respectively. virtio-scsi is preferable for everyday usage, as it allows discard of unused disk blocks, potentially reducing the size of the disk image (using tools similiar to the fstrim is best practice in this scenario). Most operating systems define their minimal required disk space, such as 20 GiB for Windows 10 64-bit version, or around 4 GiB for most of the minimal Linux installations. 3. Define the basic template. Most values in the general sections are self-explanatory. In the storage section, add the CDROM image as Disk 0, OS as Disk 1 and optionally add swap as Disk 2 by marking it volatile and selecting its type to swap and specify its size. It is necessary to setup boot from the CDROM (or Disk 0) in the OS Booting section. And connect to the 503-userpriv network in the Network panel. 4. Instantiate the template and complete the installation. The main drawback of the Sunstone installation is that it does not support any contextualization during installation or any automatic network setup. Therefore a connection must be defined manually. Virtual machine specific addresses are in the Network panel details, while global settings, such as network masks, default gateways or DNS servers, are in the Virtual Network page. The current settings of 503-usrpriv are: ∙ DNS: 147.251.48.14 and 147.251.48.41 ∙ Default gateway: 172.26.0.1 ∙ Network mask: 255.255.0.0, or \16 in CIDR notation Because the OS has been marked as persistent, it should now contain the freshly installed operating system. If any additional changes are necessary, it is at this point that they should be made. If the template is ready for publishing, terminate the virtual machine.

9 1. Stratus.FI

5. In Storage > Images select OS image and change it to to non- persistent state so that multiple users can use it at once. 6. Clean template up by removing the CDROM disk and setup a boot to the OS disk. 7. Optionally customise the template context and share the CDROM image by giving use permission to everyone, in its storage settings. As part of this thesis, a set of configuration files and scripts were created for automated installation and are maintained in the faculty Git- lab repository4 (this thesis refers to commit tagged as bachelor’s_thesis), as well as included in the attachment of this thesis. It is worth noting that this process can be performed from the Open- Nebula command line interface, but at the time of writing this thesis, Stratus.FI does not support a CLI connection for technical reasons.

1.5.3 Upgrading existing templates Upgrade of an existing system makes sense for rolling distributions of Linux, such as Gentoo or ArchLinux, as they promote this method themselves. Fixed release distributions can also be created using this process, but massive changes (such as swapping initialization systems) might leave some residual artefacts in the upgraded version. Most notable benefits of upgrading are that it is much faster and easier because most of the work has already been done during creation: 1. Copy the template of the desired virtual machine, clone the main OS disk and set it as persistent. In the cloned template change the boot disk for the new one. 2. Instantiate the cloned virtual machine and upgrade system. This step depends on the distribution and type of system running on the host and will be described in the individual chapters. 3. Change the disk back to the non-persistent state and modify the name and description of the template.

4. https://gitlab.fi.muni.cz/xlacko2/stratus.fi-image-creation

10 1. Stratus.FI

1.5.4 Local Installation The main idea is to run the installation on a local virtual machine, as described in this thesis, and to upload the finished disk. The local hypervisors ability to take care of the network settings and usually lower installation time are its main advantages over using the Sun- stone. However, misusing this method might leave an image in an inconsistent state after virtual machine initialization and also requires a user to have installed virtualization software locally. The biggest challenge is to correctly check the format of the prepared image, as an incorrect driver might halt the virtual machine, requiring manual termination.

1.5.5 Cloud-based images Cloud-based images are out-of-the-box installations ready for virtual machine usage and are usually directly available with regular installation media as an alternative solution. They are prepared for a few cloud vendors, but usually without images prepared for the OpenNebula. Virtual machine images can be modified and adapted for use with a different vendor, like conversion of an OpenStack-ready images for the OpenNebula. For all virtual machine modifications, I used the virt-sysprep tool from the libguestfs library.

2 Linux

Linux is a family of open-source Unix-like operating systems, based on the Linux kernel. According to W3Techs statistics1, 66.9% of websites known to them use a Unix system, of which 55.3% are Linux systems.

2.1 Fedora

Fedora is RPM-based distribution developed by the Fedora Project and sponsored by Red Hat. Fedora comes with stable release and has a development cycle of approximately 6 months. The latest version, Fedora 27, was released on the 14th November 2017, with an expected release of Fedora 28 on the 1st May 2018. Fedora offers multiple types of installation with various flavours although Fedora Server provides the best minimal installation for virtual machine templates.

2.1.1 Installation As with all Red Hat operating systems, Fedora comes with Anaconda as the primary installation software, with built-in automatic installation in the form of Kickstart. The Kickstart configuration file, or a kickstart for short, contains set of pre-defined answers for the questions asked during installation and can include a set of custom scripts to be executed after installation. These files consist of a mandatory Command section, which describes the installation and installation environment, and optional sections describing the installed packages and custom scripts2. Ana- conda automatically generates a kickstart for the freshly installed system which is found at /root/anaconda-ks.cfg. An example of a Stratus.FI-optimised kickstart is available in this thesis’ repository, in folder Fedora named fedora-ks.cfg.

1. https://w3techs.com/technologies/history_overview/operating_ system 2. http://pykickstart.readthedocs.io/en/latest/kickstart-docs.html

13 2. Linux #!/bin/sh if [ -z "$1" ]; then echo "Specify upgrade version" exit 1 fi dnf upgrade -y && dnf install dnf-plugin-system-upgrade -y && dnf system-upgrade download --refresh --releasever=$1 -y && dnf system-upgrade reboot

Figure 2.1: Fedora upgrade script

2.1.2 Upgrade

Fedora introduced the "Dandified Package Tool", or dnf, for the Fedora 18 release as its package manager, replacing "Yellowdog Modifier, Up- dated" (yum), which is still used by the CentOS and Red Hat Enterprise Linux. From a regular user’s point of view, there is no difference between the two other than in the names. In common with every operating system upgrade, to perform an update of all packages first constitutes best practice. As dnf is modular software, and does not include system upgrade functionality by default, it must be downloaded first. The figure 2.1 contains simple script automatizing the whole process. Upgrading Fedora is fast and straightforward, but it is ultimately an unreliable process for creating virtual machine images. Problems with custom repositories and packages are common in every release, and as OpenNebula contextualization is not included in the officially supported repositories, it or its dependencies might become damaged during the process.

2.1.3 Cloud images

Fedora also ship their custom cloud-based images. Most suitable of these is a qcow2 image for OpenStack. From the description, this image is adapted for an OpenStack, and as a result it contains the package

14 2. Linux

cloud-init, general contextualisation package, which is not suitable for the Stratus.FI server. This package must be removed and replace with the OpenNebula one-context package. Simply booting into the image and replacing the package will not help, as these images are designed to expect predefined OpenStack key-pairs, which define the password and SSH-key. The best wayto overcome this problem is by using the virt-sysprep tool. Packages can be removed and added without booting the image. An example showing the replacement of cloud-init with one-context is shown in figure 2.2. $ virt-sysprep --uninstall cloud-init \ > --install "https://github.com/OpenNebula/" \ > "addon-context-linux/releases/download/v5.4.1/" \ > "one-context-5.4.1-1.el7.noarch.rpm"

Figure 2.2: Modifying packages of Linux image

Now the image is ready to upload to Stratus.FI. An important step is to setup the mapping driver to qcow2 in the advanced settings of the Sunstone interface.

2.1.4 CentOS CentOS stands for Community ENTerprise Operating System and is the community edition of Red Hat Enterprise Linux stripped of branding and support. CentOS ships multiple types of installation media, from which minimal and network installations are the most suitable for the Sun- stone installation. Red Hat product uniformity allows the kickstart used for Fedora installation to be used with CentOS 7 (and with the Red Hat Enterprise Linux 7), with an exception of repository settings. Generic cloud images, as well as Docker, Amazon and Vagrant images, are available for the CentOS 7. Generic cloud image, rebranded OpenStack images from Fedora, requires same treatment for Stratus.FI usage. As stated, Red Hat Enterprise Linux is a similar operating system, barred behind subscriptions and requiring customer accounts, as a result of which no further information will be given about this system.

15 2. Linux 2.2 Debian

Debian is a free operating system maintained by the Debian Project association. Debian systems allow the user to choose from different kernels for their system (at the time of writing, Debian supports Linux and FreeBSD kernels, with work in progress for additional kernels, primarily the Hurd). Debian focuses on the two primary keys of stability and security which are achieved through a simple release cycle system. During development, all new packages are included, whether or not they are ready, and are checked them for security and stability flaws, when this is complete, all packages are frozen into “stable” release. This method leads to stable system but commonly with outdated packages. There are two additional development branches: “unstable” and “testing”, which do not suffer from the problem of old packages, they do not however guarantee stability and thus are less suitable as server systems.

2.2.1 Installation

Debian offers multiple packages for automatic deployment, although I will focus mainly on the installation using Debian Installer. The Debian Installer is the official and preferred software for installing the operating system, supporting a variety of installation methods dependent on the specific computer architecture. Automated installation is supported via preconfigured files (or preseeds), which are poorly documented in their specification articles3.

Preseeding an installation is a way to provide answers to some of the questions asked during installation. Similarly to Fedora’s Kick- start, preseed offers automated generation of a preconfigured file from the answers selected when running the Debian system through a debconf-set-selection tool4. Supported methods of preseeding are:

3. https://wiki.debian.org/DebianInstaller/Preseed 4. https://www.debian.org/releases/stretch/amd64/apbs03.html.en

16 2. Linux

1. Adding the preseed file to the installer disk. This technique requires modification to the provided installation disk described in the article5.

2. Autoloading the preseed from a web server via DHCP, which allows for a fully automated installation, as demonstrated and documented6, but requires control over DHCP server on a network.

3. Downloading the preseed file from the web, which is mypre- ferred option.

Automated installation is performed by selecting this option in advanced settings of the Debian Installer boot screen. This process requires access to the internet, which must be set up manually before providing a link to the preseed file. The remainder of the installation is fully automatic.

Preseed used for Debian installation is part of the thesis repository, found in a Debian folder, named preseed-debian.cfg. This file is, for the most part, self-explanatory, excepting the custom partitioning recipe. As mentioned in the preseed file commentary of partitioning section, a predefined atomic recipe will not install on smaller disks and also creates a swap partition by default. Because of this, the preseed file defines an atomic_without_swap recipe. This recipe is in PartMan format, or as stated on its wikitech page7: “incomprehensible automatic partitioning language”.

Figure 2.3 shows the recipe used. First, the partman-basicfilesystems command disables any annoying warnings about missing swap; the rest is recipe itself. The format description is in the PartManAuto help file8, 780 50 -1 xfs specifies that the minimal disk space required is 780 MiB as specified by Debian. Partition priority is unimportant, as

5. https://wiki.debian.org/DebianInstaller/Preseed/EditIso 6. http://hands.com/d-i/ 7. https://wikitech.wikimedia.org/wiki/PartMan 8. https://wikitech.wikimedia.org/wiki/PartMan/Auto

17 2. Linux d-i partman-basicfilesystems/no_swap boolean false d-i partman-auto/expert_recipe string \ atomic_without_swap :: \ 780 50 -1 xfs \ $primary{ } \ $bootable{ } \ method{ format } \ format{ } \ use_filesystem{ } \ filesystem{ xfs } \ mountpoint{ / } \ . Figure 2.3: Debian partitioning recipe there is only one partition, with an unlimited maximum size expressed by -1 and a parted specified file system.

2.2.2 Upgrade Debian uses DEB-based package management with a basic dpkg library, APT tools and recommended aptitude software. The dpkg library is not suitable for upgrade, and aptitude is recommended for upgrading systems older than Debian Jessie. For later versions, APT is a recommended tool for system upgrade, and will be used for the purposes of this thesis.

$ apt-get update $ apt-get upgrade $ apt-get dist-upgrade

Figure 2.4: Debian upgrade commands

2.2.3 Cloud images Debian has prepared cloud-init OpenStack images in qcow2 and raw format for the x86_64, where the amd64.qcow2 image is the most suitable for Stratus.FI server.

18 2. Linux

Again, cloud-init must be replaced with the one-context package, which is available from the official repositories under the name opennebula-context. Unfortunately, this package is available in outdated version and is not supported by Stratus.FI server.

2.2.4 Ubuntu Ubuntu is an official Debian derivative (“Debian ‘derivative’ is acollo- quial term used to refer to distributions which are based on Debian in various ways”9). Ubuntu choses a more user-friendly operation with more convenient features, such as a home-brewed desktop environment: Unity (which will be replaced in the next release by the GNOME), or the popular Ubuntu Software Center. However, Ubuntu and Debian are not fully binary compatible, even though they have a common package format. Ubuntu gets most of its packages from the Debian unstable branch, and many more third-party repositories are made for Ubuntu than De- bian. This fixes the problem of sometimes heavily outdated packages at cost of stability. A serious problem for Ubuntu however is its installation software, or better the modifications they made to Debian Installer. First and foremost, workstation versions of Ubuntu come with a nice graphical interface, called Ubiquity for Debian Installer, that ignores preseed files, making automatic installation of this version impossible. Thankfully, server versions of Ubuntu do not included the graphical interface, although automatic installation is still not fully possible without installation disk modification. The problem lies in the modified order of the Debian Installer, that requires a specific installation configuration before networking, so that even when the preseed fileis provided, automatic installation begins after the few first dialogues for keyboard settings, localization, networking and host-name specification. It is worth mentioning, that the preseed file will override these settings for the installed system. Preseed files are provided as boot option url=, triggered by pressing tab key during boot screen selection.

9. https://wiki.debian.org/Derivatives

19 2. Linux

I do not recommend Ubuntu automatic installation as the primary image creation method, even though the modified Debian preseed is part of the repository, named preseed-ubuntu.cfg in the Ubuntu folder. First of all, Ubuntu comes with the stubborn idea that the user should not have access to the root account, instead, a separate admin- istrative account should be used, which with incorrect preseed might leave the installed system in inaccessible state. For students requiring Ubuntu up-to-date images, manual installation might be easier for Stratus.FI users.

2.2.5 Linux Mint

This distribution is built on the Ubuntu, but it takes extra steps to focus on new users. It includes an abundance of everyday use software such as Flash player or codecs, but removes options for experienced users such as availability of server versions or official cloud-based images. In fact, official Linux Mint installation does not allow minimal oreven installation without graphical desktop environment. Any attempts at stripping or modification of this system would lead to a butchered Ubuntu installation. Linux Mint is not a suitable system for a cloud environment.

2.3 OpenSUSE

OpenSUSE is another RPM-base sponsored by SUSE Linux LLC. Open- SUSE promotes Tumbleweed rolling release version and Leap stable release version at the same time, as well as ready to use cloud images for Microsoft Azure, Amazon Elastic Cloud Compute, Google Compute Engine and OpenStack. In contrast to Debian, OpenSUSE takes the opposite direction with their thoroughly extensive, pitch-perfect and example ridden documentation.

20 2. Linux

2.3.1 Installation YaST (Yet another Setup Tool), as stated on the OpenSUSE landing page10, is “the best/only comprehensive Linux system configuration & installation tool” and has built-in unattended installation AutoYaST software. Installation definition in AutoYaST is provided through the control file, which in contrast to previous automatic installations, doesnot give pre-defined answers to the questions asked during installation but instead employs a new, full-scale system definition. These control files are XML documents able to define every aspect ofthenew system, from global environment setting, X11 configuration to custom questions during installation and have their own preinstall/post- partitioning/chroot/post-install or initialization scripts provided directly in the control file. AutoYaST is a viable solution for both the rolling and stable release versions of OpenSUSE. After booting from the installation disk, customize the boot options to point to the configuration file, in format: autoyast=. OpenSUSE developed a Configuration Management System which is a recommended graphical tool for the fast creation of these files. Unfortunately, this software requires a pre-installed graphical Open- SUSE system, and offers an obfuscated interface, mostly suitable for basic installations. Creating custom configuration files by hand is, in my opinion, faster and more comfortable. Example of such control file, optimised for Stratus.FI is in folder OpenSUSE named autoinst.xml.

2.3.2 Upgrade OpenSUSE comes with the custom Zypper package management tool, based on the libzypp software management engine. Upgrading stable Leap or older versions of OpenSUSE is possible in offline mode through upgrade media or image, and online mode through the Zypper. During the upgrade, it is important to be aware of a few caveats, such as the fact that the upgrade can be done only gradually from succeeding releases; and that updates of older 32-bit systems are not possible, as Leap is 64-bit only.

10. https://www.opensuse.org/

21 2. Linux

Upgrade itself is heavily conditional and is described in its documentation on page11. Overall, upgrading the stable version of Open- SUSE is not worth the effort. For Tumbleweed rolling release, upgrading is much more efficient and meaningful than its stable counterpart, as it offers a reliable and quick environment for updates. Zypper offers two means of updating, conservative update, which will not update packages from different repositories (or vendors) and might not tidy up old repositories; and the more drastic dist-upgrade, which will deliberately switch to the newest vendor or remove custom packages completely, but sanitize the system afterwards. The best way to undertake an OpenSUSE upgrade is to use the middle ground with dist-upgrade with forbidden vendor changes, as shown in figure 2.5. $ zypper dist-upgrade --no-allow-vendor-change

Figure 2.5: OpenSUSE consensus for the system update

2.4 ArchLinux

ArchLinux is an independently developed distribution aimed at experienced Linux users and operating on the rolling release system, with strong emphasis on system customisability for the user. Currently, ArchLinux is phasing out older i686 architecture support and purging all packages for this architecture.

2.4.1 Installation ArchLinux, by default, does not offer any installation software or automated installation tools. On the contrary, ArchLinux expects users to do the whole installation by hand. On the positive side, ArchLinux installation is intended only for new installations; the existing system can be easily updated with the single command pacman -Syu, which installs all updates with automatic confirmation. Another valid option is to use the options -Syyu, which will update all packages, even if they appear up-to-date.

11. https://en.opensuse.org/SDB:System_upgrade

22 2. Linux

This notion makes ArchLinux unsuitable for automatic installation, but there are a few solutions. One of them is Anarchy-Linux (formerly known as arch-anywhere), which is ArchLinux-based distribution enriched for an installer tool in order to mainstream the whole process but without any automated installation solution, which allows for the creation of the first image by hand with the remainder left for the update. The second one is to employ one of user-created installation bash scripts, containing the process. The best known of these is ArchLinux Ultimate Install (AUI), available on the Github12. AUI comprises two sets of scripts: FIFO, which installs the system base; and LILO, which sets up the system environment. Unfortunately, FIFO script cannot be automated at all and LILO script, while able to predefine configuration, installs a heavyweight graphical environment. As my final solution, I opted for custom scripts based on an article of Van Nguyen13.

The scripts were customised for better control and pruned of unnec- essary configurations, and can be found in an ArchLinux folder ofthe repository, as install.tar tarball. Installation consists of install.sh and chroot.sh scripts. To begin installation of new system form installation media, firstly, configure the IP addresses with the ip command, as shown in figure 2.6, then download scripts tarball and extract it, lastly invoke install.sh with an name of the destination disk, for example: ./install.sh sda. $ ip link set ens3 up $ ip addr add 172.26.X.X/16 dev ens3 $ ip route add default via 172.26.0.1 $ echo "nameserver 147.251.48.41" >> /etc/resolv.conf Figure 2.6: Setting up static IP addresses

ArchLinux is not an officially supported system for the OpenNeb- ula contextualization package, but one-context is present in the Arch User Repository (AUR), but only in the old v4.14.0 version, which

12. https://github.com/helmuthdu/aui 13. https://shirotech.com/linux/how-to-automate-arch-linux-installation

23 2. Linux does not work on the Stratus.FI server. Instead, automated installation contains custom systemd script, that is capable of setting up the root password and configuring the network automatically (sometimes it fails to setup network properly, in such case invoking the context command from terminal should fix the problem).

2.4.2 Cloud images ArchLinux images are a work-in-progress with a few experimental images ready to download. This thesis will not deal with any of these images as they are subject to change.

2.5 Gentoo

Gentoo, even more than ArchLinux, is Linux distribution aimed at experienced Linux users and is built around the Portage software. Portage is the software distribution system used for update and package management as well as (partially) for system installation. Gentoo does not belong to either rolling or stable release distribution, but rather to a completely different operating system archetype as a source-based system, rather than all previously mentioned binary- based systems. This means that installing a package does not require downloading of a pre-built binary blob, but rather downloading of source code and building this locally before installation. This model gives Gentoo another level of software optimization. Gentoo installation is based on a stage tarball. Stage tarball is a compressed part of the system: the later the stage, the less actual installation is needed but fewer customisations are then available. There are four stages, numbered one to four, and the Gentoo Handbook [3] describes Gentoo installation based around stage 3 tarball. This stage contains an almost-functional system, missing only the kernel and the bootloader. For the curious, I modified the ArchLinux install script to partially automatize the process to the point that only kernel configuration needs manual handling. Static IP addresses are configured same way as in ArchLinux 2.6. There is also a possibility to reduce involvement in the script even further by setting up the CONFIG variable to localyesconfig value,

24 2. Linux which will automatically copy all loaded kernel modules from LiveCD (but still needs a few extra inputs). For actual cloud image creation, Gentoo advises use of the stage 4 tarball for an image creation and even provides a script to do so14. The only addition to the Stratus.FI server is the unofficial one-context package on the disk. The process of doing so is shown in figure 2.7 (extracted from the install scripts).

wget -P /root "https://github.com/jirutka/one-context"\ "/archive/master.zip" unzip /root/master.zip /bin/bash/ /root/one-context-master/install rc-update add vmcontext boot

Figure 2.7: Gentoo one-context installation

Updating Gentoo with the Portage is straightforward and split into two parts: emerge update && emerge upgrade.

14. https://github.com/prometheanfire/gentoo-cloud-prep/blob/master/ 03-prep-that-image.sh

3 Microsoft Windows

Microsoft Windows is a series of graphical, multi-tasking operating systems developed by Microsoft. The first version of Windows, Mi- crosoft Windows 1, was released on 20th November, 1985. Windows initially did not contain a multi-user environment; this was changed with Windows NT with the first multi-user system Microsoft Windows NT 3.1. In contrast to the Linux systems, Microsoft Windows is not an open-source nor free operating system, but students and employees of the Faculty of Informatics of Masaryk University, have access to free copies of Microsoft software through the Imagine server1. Modern Microsoft Windows comes in multiple workstation versions as well as separate server versions (with different product la- belling). For cloud images, server versions are more suitable, but still with harsh downsides for actual use: ∙ Worse minimal requirements. The newest Windows Server 2016 asks for 10 GiB minimum free disk space just to begin an installation. ∙ Required product activation must be handled separately during installation and after instantiation of a virtual machine. ∙ Security Identifiers (SID) are unique values to identify auser. After each user login, the system retrieves the SID from a local database and places it into the access token of that user. This SID is used to identify the user in all subsequent interactions with Windows security. If not handled properly, Security Identifiers will not allow correct image contextualisation. Except for minimal requirements, other problems can be solved with the system preparation, or Sysprep, tool, which captures customised Windows images for further use. The most prominent features of Sysprep are: removing system- specific data from installation (such as Security Identifiers), resetting

1. https://e5.onthehub.com/WebStore/ProductsByMajorVersionList.aspx? ws=7616c804-8b6f-e011-971f-0030487d8897

27 3. Microsoft Windows the Windows product activation and system hardware generalisation to enable system migration to multiple virtual machines.

3.1 Installation

Microsoft Windows comes with a home-brewed Windows Setup. The Windows Setup supports network installation, but only with an existing source server, working DNS and DHCP connections, which are not available on the Stratus.FI server. Because of this, full installation media is required.

For automated install, Windows Setup supports automated installation through the XML answers file. In comparison to the OpenSUSE configuration files, Windows Setup files are compact and intuitive to use. Sample templates are available for download from the Mi- crosoft websites2. Microsoft also offers a useless graphical tool (for Microsoft Windows systems only) to generate these files: Microsoft SIM, standing for System Image Manager, contained in the Windows ADK (Windows Assessment and Deployment Kit). The example in this chapter will show how to create the image for Microsoft Windows Server 2016 with the lightly modified x64 BIOS example file that can be found in the thesis repository under the name Autounnatend.xml. Firstly, OEM (original equipment manufacturer) information is removed from the answer file. Secondly, a question about product key handling arises:

∙ Removing the product key allows for the reuse of the answer file, but requires manual handling, such as product selection or activation key insertion, during installation.

∙ Present product key allows Windows Setup to determine the specific product to install and also allows fully automatic installation. But product key and product mismatch will cause installation failure, and most importantly, it requires the private activation key to be published. Thankfully, most products

2. https://docs.microsoft.com/en-us/previous-versions/windows/ it-pro/windows-8.1-and-8/dn621892(v=win.10)

28 3. Microsoft Windows

have "trial" activation keys publicly available on the Microsoft websites.

I aim to deploy the system automatically and then boot into specialised audit mode (loading audit mode is possible through the answer file or with the Sysprep tool), that is made for image modification, to install the one-context software for Windows available on the Github3. Answer files can be provided from multiple sources:

∙ Explicit file location, supplied in the Setup command line option. Requiring modification of boot options of disk bootloader. ∙ One of the implicit file locations, on the installation disk. ∙ File supplied on independent removable media.

The last method allows for abuse of the OpenNebula CONTEXT images described in section 1.1. As these images are contained in the root directory of context images, they are certainly fit for this job. The name of the file must be either Autounattend.xml or Unattend.xml. Finally, after installation is completed and one-context installed, the Sysprep is run with the clean-up action set to Out-of-Box Experi- ence and the shutdown option set to shutdown, to remove the SIDs and installation product key. Modern Microsoft Windows system comes with a pre-configured automatic disk optimisation, which contains trimming of unused blocks from a drive. This configuration do the same job, needed from the fstrim on the Linux systems.

3.2 Upgrade

Upgrading the Windows images for purposes of image creation is unduly, as upgrade itself is heavily dependent on the system, from which fresh installation is only common method. Additionally, there are no benefits to be gained from upgrading Windows, as the process requires massive resources, such as the installation disk or a data download of similar size; and time to complete the upgrade.

3. https://github.com/OpenNebula/addon-context-windows

4 FreeBSD

BSD stands for "Berkeley Software Distribution", named after the distribution of source code from the University of California, Berkeley. BSD operating systems are open-source derivatives of AT&T’s Research UNIX operating system. This thesis focus primarily on the FreeBSD system as it is currently the most used BSD operating system by the BSDstats project1.

4.1 Installation

FreeBSD, from version 9.0-RELEASE, comes with text-based bsdinstall software as its primary installation method, replacing the older sysinstall. The bsdinstall software comes with a simple scripting option for automatic installation. The format of this file is a short preamble, defin- ing the environment variables, followed by a shell script, a complete definition is well described in the bsdinstall manual page2. The only way of providing the scripting file is by adding it to the installation media at the /etc/installerconfig location, for which, disk modification is necessary. I will describe an example method of doing so on a Linux system.

I choose the uncompressed DVD image from available disks for installation. Disk images are in iso9660, which is, by-design, a read- only file system format. Thus disk extraction is required. The standard Linux approach is to mount the disk and copy its content locally. It might be worth noting that bsdtar, included in the libarchive Linux library, can extract the contents of the iso9660.

Now, the installation script and other customisation should be copied to the local folder. FreeBSD has an unofficial port for one-context, available on the Github3, and it is a good idea to include it on the new installation media as it allows complete unattended installation. Also, fstab must point to the installation CD-ROM, instead of the original

1. http://bsdstats.org/ 2. https://www.freebsd.org/cgi/man.cgi?bsdinstall(8) 3. https://github.com/slob/freebsd-one-context 31 4. FreeBSD disk, as it is available only with specific iso9660 headers. Lastly, new iso disk should be created from the local folder. On Linux systems, the mkisofs command should be available to do so. The whole process, in form of an automated script,is in the repository of this thesis, contained in the FreeBSD folder.

4.2 Upgrade

FreeBSD advises, in contrast to a Linux system, to firstly undertake a proper system upgrade and to update packages thereafter. Unfor- tunately, this process cannot be automatized easily as restart during upgrade is necessary:

1. Performing the upgrade with $ freebsd-update -r command, which expects the release name of the new system. FreeBSD convention for release names is 11.0-RELEASE.

2. Restart the host.

3. Update the system packages: $ freebsd-update install

32 5 Conclusion

The goal of this thesis was to test and examine different operating systems and their installation methods, and to objectively determine the best way of creating virtual machine images from the Stratus.FI user (students, professors and faculty employees) point of view. Emphasis was put on providing at least one automated installation method, with preference given to those requiring an installation seeding file, without resorting to the use of any third-party tools, with the exception of FreeBSD, as described in section 4. Usually, all of these files are simple or well documented, and because of this areonly outlined briefly with the exception of a few non-trivial parts. Each of the used seeding files are included as part of this thesis’ repository and the attachment of this thesis. Unfortunately, in the current faculty environment, only fully automatic installation, without the need to modify the installation media, was achieved by the Microsoft Windows operating system. Furthermore, I also included an alternative means of creating images, mainly describing the updating processes and processes for modification of cloud-based images or other pre-prepared solutions, and discussed their availability in comparison to automatic installation. Customisation of images was done with the libguestfs library built on libvirt virtualisation API, to show that it is an invaluable tool for creating, managing and customizing virtual machines locally, or for preparing them for the Stratus.FI server. As proof of concept, operating system templates were created for Debian, OpenSUSE, Arch- Linux and FreeBSD, which are available on the Stratus.FI server.

5.1 Future work

The next step in improving the quality of life for the Stratus.FI user would be to set up a DHCP server to tackle the biggest obstacle in the way of fully automatic installation. Or to deploy enterprise solutions for the cloud, such as configuration management tools like Puppet or PXE.

33 5. Conclusion

This thesis promotes the official one-context package as a go- to tool for contextualisation, and advise to replace any other contextualisation solution with one-context. As shown, only Fedora- based, Debian-based, OpenSUSE and Microsoft Windows systems are officially supported by the OpenNebula project, maintainer ofthe one-context package. Leaving the rest with, out-dated or barely working unofficial ports. Future work should focus on the improvement of the one-context package or an alternate solution like cloud-init for OpenNebula package, to tackle the problem with limited one-context support. Additional future goal would be further automatization of image creation, with scripts based on OpenNebula API for the periodic and unattended image creation and update of existing images.

34 Bibliography

1. GRANCE, Peter Mell , Timothy. The NIST Definition of Cloud Computing [online] [visited on 2017-12-05]. Available from: http://nvlpubs. nist . gov / nistpubs / Legacy / SP / nistspecialpublication800 - 145.pdf. 2. OpenNebula 5.4 Documentation [online]. OpenNebula.org, 2002–2017 [visited on 2017-11-13]. Available from: https://docs.opennebula. org/5.4/index.html. 3. Complete Handbook - Gentoo Wiki [online]. Gentoo Foundation, Inc., 2001–2017 [visited on 2017-10-09]. Available from: https://wiki. gentoo.org/wiki/Complete_Handbook. 4. Guest Configuration [online]. OpenNebula Systems, 2015–2017 [visited on 2017-11-13]. Available from: https://docs.vonecloud.today/ 2.0/guest_configuration/. 5. PartMan [online]. Wikimedia Foundation, 2017 [visited on 2017-11-19]. Available from: https://wikitech.wikimedia.org/wiki/PartMan. 6. LEHEY, Greg. Explaining BSD [online]. The FreeBSD Project, 1995– 2017 [visited on 2017-12-01]. Available from: https://www.freebsd. org/doc/en_US.ISO8859-1/articles/explaining-bsd/article. html. 7. Fedora Documentation [online]. Red Hat, Inc. et al., 2017 [visited on 2017-10-05]. Available from: https://docs.fedoraproject.org/. 8. Debian – Documentation [online]. SPI et al., 1997–2017 [visited on 2017-10-05]. Available from: https://www.debian.org/doc/. 9. Official Ubuntu Documentation [online]. Canonical Ltd., 2017 [visited on 2017-10-06]. Available from: https://help.ubuntu.com/. 10. Official User Guide [online]. Linux Mint, 2006–2017 [visited on 2017-10-06]. Available from: https : / / linuxmint . com / documentation/user-guide/Cinnamon/english_18.0.pdf. 11. doc.opensuse.org - Documentation Guides & Manuals [online]. SUSE LLC., 2010–2017 [visited on 2017-10-07]. Available from: https : //doc.opensuse.org/.

35 BIBLIOGRAPHY

12. ArchWiki [online]. The ArchLinux Team [visited on 2017-10-08]. Avail- able from: https://wiki.archlinux.org/. 13. Unattended Windows Setup Reference [online]. Microsoft Windows [visited on 2017-10-11]. Available from: https://docs.microsoft.com/ en-us/windows-hardware/customize/desktop/unattend/. 14. FreeBSD Documentation [online]. The FreeBSD Project, 1995–2017 [visited on 2017-10-11]. Available from: https://www.freebsd.org/ docs.html.