Enhancing Openstack Clouds Using P2P Technologies

Master of Science in Telecommunication Systems October 2017

Enhancing OpenStack clouds using P2P technologies

Robin Philip Joseph

Faculty of Computing Blekinge Institute of Technology SE371 79 Karlskrona, Sweden This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulllment of the requirements for the degree of Master of Science in Telecommunication Systems. The thesis is equivalent to 20 weeks of full time studies.

Contact Information: Author: Robin Philip Joseph E-mail: [email protected]

University advisor: Dr. Kurt Tutschku Department of Computer Science (DIDD)

Faculty of Computing Internet : www.bth.se Blekinge Institute of Technology Phone : +46 455 38 50 00 SE371 79 Karlskrona, Sweden Fax : +46 455 38 50 57 Abstract

It was known for a long time that OpenStack has issues with scalability. Peer-to-Peer systems, on the other hand, have proven to scale well without signicant reduction of performance. The objectives of this thesis are to study the challenges associated with P2P-enhanced clouds and present solutions for overcoming them. As a case study, we take the architecture of the P2P-enhanced OpenStack implemented at Ericsson that uses the CYCLON P2P protocol. We study the OpenStack architecture and P2P technologies and nally propose solutions and provide possibilities in addressing the challenges that are faced by P2P-enhanced Open- Stack clouds. We emphasize mainly on a decentralized identity service and management of Virtual machine images. This work also investigates the characterization of P2P architectures for their use in P2P-enhanced OpenStack clouds. The results section shows that the proposed solution enables the existing P2P system to scale beyond what was originally possible. We also show that the P2P-enhanced system performs better than the standard OpenStack.

Keywords: Cloud Computing, OpenStack, P2P, Scalability

i Acknowledgements

Firstly, I would like to thank my family for bringing me this far in life. I am immensely grateful to my professor at BTH, Dr. Kurt Tutchku and my supervisor at Ericsson, Dr. Fetahi Wuhib. This work would not be possible without their guidance, insights and their patience. I am thankful to Dr. João Monteiro and Vinay Yadav from Ericsson Cloud Research, for providing support and boosting my morale. I am also grateful to Ericsson for providing resources and experiences. I would also like to express my unending gratitude to my friends and colleagues for supporting me and encouraging me in my work.

ii Contents

Abstract i

Acknowledgements ii

1 Introduction 1 1.1 Concepts ...... 1 1.1.1 OpenStack ...... 1 1.1.2 Peer-to-Peer Technologies ...... 2 1.1.3 P2P-enhanced OpenStack clouds ...... 3 1.2 Problem description ...... 3 1.3 Motivation ...... 4 1.4 Research Questions ...... 4

2 Conceptual Background 5 2.1 Summary of the existing P2P-enhanced OpenStack ...... 5 2.2 Keystone and Federated Identity ...... 7 2.2.1 Keystone ...... 7 2.2.2 Federated Identity ...... 8 2.2.3 External Identity Providers (IDP) ...... 9 2.2.4 Keystone as an IDP ...... 9 2.2.5 Mapping ...... 10 2.3 Container Technology ...... 10 2.3.1 LXD ...... 10 2.3.2 Docker ...... 10

3 Related Work 12

4 Method 14 4.1 Overview of the applied methods ...... 14 4.2 Analysis of Basic Functional Requirements in OpenStack . . . . . 14 4.2.1 Identity service ...... 15 4.2.2 Image management ...... 16 4.2.3 Scheduling of Virtual Resources ...... 16 4.2.4 Networking Service ...... 17

iii 4.3 Functional Analysis of capabilities of Peer-to-Peer technologies for OpenStack ...... 17 4.4 Analysis of Numerical Scalability and Performance of Features of P2P Architectures ...... 20 4.5 Inference Phase ...... 22 4.5.1 Keystone solution decision ...... 22 4.5.2 Image management solution decisions ...... 23 4.6 Implementation and Verication ...... 24 4.6.1 Image management solution ...... 24 4.6.2 Decentralized Identity Service solution ...... 24

5 Results 28 5.1 Decentralized Identity Solution ...... 28 5.2 Peer-to-Peer System ...... 29

6 Analysis and Discussion 32 6.1 How can Peer-to-Peer architectures be characterized for their suitability in P2P-based OpenStack clouds? ...... 32 6.2 How can P2P technology be used for decentralized identity service in OpenStack clouds and which P2P-technology should be applied for high scalability of this service? ...... 33

7 Conclusions and Future Work 34

References 35

iv List of Figures

1.1 OpenStack Services Outline ...... 1

2.1 Architecture of the existing implementation of P2P-enhanced Open- Stack ...... 6 2.2 Architecture of the P2P-OpenStack agent ...... 6 2.3 Keystone Backends ...... 8 2.4 Mapping of remote user to local user ...... 11

4.1 Outline of phases and methods employed in the thesis ...... 15 4.2 Flow of image between OpenStack clouds ...... 16 4.3 Functional design space of P2P service overlays ...... 18 4.4 View of the Additional Dimension of the Functional Design space 19 4.5 Architecture of the Identity service solution ...... 25 4.6 Flow of Requests between Agents for Tokens ...... 26

5.1 Average Response times for 64 requests ...... 28 5.2 Average area under cputime curve for 64 requests ...... 29 5.3 Averages of Response times versus Number of requests and number of SPs ...... 30 5.4 VM startup time and fail rate of the standard and P2P systems with increasing system size and load ...... 31 5.5 VM failure rate of the standard and P2P systems (size 32) with increasing concurrent VM requests ...... 31

v Chapter 1 Introduction

1.1 Concepts

Peer-to-Peer technologies oer a highly scalable and eective alternative to the server-client model. These technologies, when incorporated into clouds, can provide enhanced versions of clouds that are ecient, scalable and overcome the limitations of the server-client paradigm.

1.1.1 OpenStack OpenStack is an open-source Cloud operating system which provisions virtual resources and services. The core functionality of OpenStack is delivered through the services that it oers. The OpenStack services among others include the compute service, identity management, networking and image management. Fig 1.1.1 taken from the ocial OpenStack documentation [10] presents the Open- Stack services and their primary functions.

Each service of OpenStack provides a separate functionality and is exposed to the end-user via REST APIs. Below is a brief outline of the OpenStack services.

Figure 1.1: OpenStack Services Outline

1 Chapter 1. Introduction 2

Nova (Compute service) This service provides the management of virtual machines through an abstraction layer for compute drivers that interfaces with supported hypervisors. Multiple hypervisors are supported by OpenStack including Kernel-based Virtual Machine (KVM), Xen, Hyper-V and Linux Containers (LXC).

Neutron (Networking Service) Previously named Quantum, this service provides various networking functions like creating and deleting networks, subnets. It also provides additional functions for management of IP address, DNS, DHCP, load balancing etc.,

Keystone (Identity Service) Authentication and authorization throughout the OpenStack infrastructure are provided by Keystone. This service is pluggable and integral in the secure com- munication of the dierent services of OpenStack.

Glance (Image Service) Glance Image service manages the discovery, registration and the distribution of virtual machine images (VMI) to the OpenStack compute nodes.

Horizon (Dashboard Service) This service provides a web-based interface for managing, monitoring and provisioning of the OpenStack resources.

1.1.2 Peer-to-Peer Technologies Generally, P2P systems are highly scalable, highly distributed and self-organizing, which implement P2P services and functions to perform tasks in an altruistic sense. In other words, each peer in the system is involved in transactions that do not directly benet itself but benets the system on the whole. Every peer in a strictly P2P system is equal in terms of functionality and must be able to take the role of both the server as well as the client, simultaneously. A P2P protocol provides the function to join and leave the P2P overlay as well as to insert, delete, discover and retrieve the resources and also service-specic functions that are highly distributed. The term Peer-to-Peer (P2P) is often perceived as "not client/server" and has utility in systems, protocols, mechanisms, architectures, services and applications [34]. In modern applications, the border between P2P and client-server paradigms are not very clear. While some architectures can be considered to fully comply Chapter 1. Introduction 3 with the P2P paradigm, others can be in between client-server and P2P depending on the denition of P2P being considered. Thus, we rst dene what we mean by P2P in our paper. We borrow this denition from [7]. "A system is considered to be P2P if the elements that form the system share their resources in order to render the service that the system is designed to provide. The elements in the system both provide services to other elements and request services from other elements." In practical applications, a system can be considered Peer-to-Peer even if few of its elements do not strictly follow this criterion. This denition is within the context of a single service and hence in complex systems which oer multiple services, some of the services can be P2P while others can be client-server. P2P services are designed to overcome the limitations of the client-server model in terms of fault tolerance, scalability and self-organization. Thus, as the number of nodes increases, the performance of the P2P systems will not degrade as much as that of the client-server systems.

1.1.3 P2P-enhanced OpenStack clouds Multiple OpenStack clouds can be associated together to form a P2P system and to co-operate and function as a single cloud. The performance of this P2P- enhanced OpenStack cloud was shown to be better than the standard OpenStack cloud when we considered the scale of the compute nodes [12]. This abstraction of several clouds as one cloud overcomes the limitations of the underlying elements of the individual compute nodes. Thus, it is of interest to further develop P2P-enhanced OpenStack clouds and study the suitability of P2P technologies in enhancing the other services of OpenStack in terms of scalability. In this paper, we take the denition of scalability from [23]. Thus, Scalability can be dened as the adaptability to changes in the peer-to-peer system size, load extent and nature of the load. [34] states that scalability describes the numerical relationship between an inuencing factor that changes numerically over a large range of values and a performance metric of a P2P service overlay.

1.2 Problem description

It is known that OpenStack is not very good at scaling. Thus, there is an interest in the research community to develop this aspect of OpenStack clouds by employing technologies that are known to be scalable such as Peer-to-Peer technologies. There is also a gap in the research area when it comes to implementing P2P technologies in the OpenStack clouds, investigating the limitations of these P2P-enhanced clouds and eectively overcoming them. Chapter 1. Introduction 4 1.3 Motivation

This thesis aims to provide a solution for scaling OpenStack cloud instances on the cloud level. We aim to investigate the suitability of P2P technologies in enhancing the OpenStack services and eectively improving the scalability and performance of these clouds. We use the case study of the P2P-enhanced OpenStack cloud set up at Ericsson to investigate the limitations and bottlenecks of the architecture and to evaluate the use of P2P technologies and functions in overcoming these limitations. The primary focus is on improving the identity and authorization management of the P2P system. This thesis paper is structured as follows. Chapter 2 outlines the important concepts helpful in understanding this thesis paper. Chapter 3 presents the related work done in the eld of Peer-to-Peer technologies and OpenStack scalability. Chapter 4 describes in detail, the method along with the implementation and evaluation of the solutions. Chapter 5 and 6 provide the results of the evaluation along with analysis and discussions. Lastly, chapter 7 provides the conclusion and future work of the thesis work.

1.4 Research Questions

From descriptions of the motivation and the problem statement along with an investigation on the P2P-enhanced Cloud taken as our case study, we have two research questions that demand an investigation.

1. How can Peer-to-Peer architectures be characterized for their suitability in P2P-based OpenStack clouds?

2. How can P2P technology be used for decentralized identity service in Open- Stack clouds and which P2P technology should be applied for high scalability of this service? Chapter 2 Conceptual Background

This chapter presents a summary of a P2P-enhanced OpenStack cloud that we use as a case study. It also briey describes the concepts and technologies that were used in the thesis.

2.1 Summary of the existing P2P-enhanced Open- Stack

The paper by Xin Han [12] pointed out the limitations in scalability of the Open- Stack compute nodes per controller node. Instead of looking at the lower level issues of the individual services, the author proposed a new architecture of Open- Stack, as a solution which abstracts several OpenStack clouds to behave as a single P2P-enhanced cloud and thus improves the scalability of OpenStack clouds. The solution involved the design of an OpenStack agent which acted as a mediator between the OpenStack API and the user. The agents collectively form a Peer-to-Peer overlay network. The system is a hybrid P2P system, as illustrated in Figure 2.1 taken from the same paper, where all the clouds share a common identity service provided by one centralized Keystone node. The protocol chosen was the CYCLON protocol [35] because of its low diameter and ability to deal with massive node failures and high churn rates. Each peer keeps track of a subset of the peers in its neighbor list and a shu ing algorithm is used to keep updating and exchanging the list of neighbors. The architecture of the agent includes an agent Database, HTTP server, a CYCLON component, four proxies (for four services of OpenStack Nova, Neutron, Glance and Keystone) and a scheduler component. Figure 2.2, also taken from the paper, outlines the architecture of the agent which includes the proxies for each OpenStack service. The proxies act as brokers and maps agents' endpoints (URLs) to OpenStack components' endpoints. The database runs MySQL and stores the users' allocation info. The HTTP server listens on a port and mediates between the user and agent. The CYCLON component is the implementation of CYCLON API and maintains a list of aliated neighbors and exchanges its list of neighbors with other agents periodically. Lastly, the scheduler selects an

5 Chapter 2. Conceptual Background 6

Figure 2.1: Architecture of the existing implementation of P2P-enhanced Open- Stack

OpenStack cloud instance to create the requested resource based on lters and weighers. Filters and weighers are used to select the suitable cloud from the overlay network. As a proof of concept, Create, Read and Delete (CRD) operations of Open- Stack are partially supported. The evaluation was done through two scenarios using a benchmarking tool called Rally, and it involved comparing the bench- marks of a Standard OpenStack cloud with the Peer-to-Peer solution that was implemented. The rst scenario tests the concurrency of the requests and how the increase in concurrent requests aect the controller response times. The second scenario evaluates the eect of increasing the number of controller nodes on the response times. The results show that without considering the eect of the

Figure 2.2: Architecture of the P2P-OpenStack agent Chapter 2. Conceptual Background 7 centralized Keystone node, using Peer-to-Peer to abstract multiple clouds as one cloud makes it scale better compared to a conventional OpenStack cloud.

2.2 Keystone and Federated Identity

2.2.1 Keystone The Keystone service provides the basic identity and authorization functions in OpenStack clouds. The concepts in Keystone focus on authorization, access management, and discovery. Keystone manages access and authorization by creating an RBAC (Role Based Access Control) policy which is enforced on each public API endpoint. Figure 2.3 taken from the book by Martinelli et al. [20] gives an illustration of the most common Keystone back-end functions and the technologies used to provide them. The components in green viz., Assignment, Resource and Catalog are usually SQL (Structured Query Language) based, the red identity component indicate usually LDAP (Lightweight Directory Access Protocol) or SQL, the blue Token indicates SQL or Memcache and lastly, the gray component is the Policy and served from a le.

SQL backend When Keystone acts as an identity provider, the credentials and information such as name, password and descriptions can be stored in databases like MySQL, PostgreSQL and DB2. The benets of using these databases are that they can be easily set up and user management can be done through OpenStack APIs. However, Keystone provides weak password support and no password rotation or recovery. Most enterprises have their own LDAP server that they want to use.

LDAP backend The benet of using LDAP backend is that Keystone is no longer the identity provider and the security of the identity service will depend on the enterprise LDAP service. Keystone still needs to authorize the users and thus users need to be created in Keystone as well, which may not be feasible in hybrid clouds with hundreds of hosts. Service accounts for each OpenStack service also need to be stored somewhere and enterprises may not want it to be in LDAP back-ends.

Federated Identity backend In the case of Federation, OpenStack leverages the existing infrastructure to authenticate users and retrieve info about users. There is a further separation between Keystone and handling identity information along with possibilities for single sign-on service and hybrid clouds. The actual passwords don't go through Chapter 2. Conceptual Background 8

Figure 2.3: Keystone Backends

Keystone and identity providers handle authentication completely. The short- coming of this approach is in the complexity of setting up identity sources. The following section will give a more detailed description of the federated identity in OpenStack.

2.2.2 Federated Identity OpenStack provides an extension to its identity service, Keystone, which allows a user associated with one cloud to access resources of another trusted cloud. This extension is called 'Federated identity' and is supported in OpenStack versions starting from Kilo. Through this extension, authentication and authorization functions of the identity service are separated and assigned to separate entities. These entities are the Identity Provider (IDP) which manages credentials and the service provider (SP) which authorizes and grants access to resources. Federation involves some common steps to implement, irrespective of the identity provider used. They are generalized as follows.

Conguring Apache to use federation capable modules. This includes setting up the Apache modules for the corresponding Federation protocol used (SAML or OpenID connect) and also conguring the IDP service to be properly discovered by the SP.

Conguring Keystone to support federation. This includes creating the OpenStack projects, domains, groups, roles, mappings etc., as well as conguring the SP and IDP for use with their respective clouds. There are two ways in which OpenStack consumes federated identity, using external identity providers and using Keystone as the identity provider. Chapter 2. Conceptual Background 9

2.2.3 External Identity Providers (IDP) OpenStack supports external identity providers and broadly two kinds of identity providers viz., Security Assertion Markup Language (SAML) based and OpenID Connect (OIDC) based identity providers [8]. These external identity providers currently operate when Keystone is run under the Apache web server. The two Apache modules mod_shib and mod_auth_openidc are used as clients to the External IDPs. These modules act as an interface between Keystone service and the IDP service.

SAML The Security Assertion Markup Language (SAML) was built on x.509 by dening authentication and attribute assertions in XML rather than in ASN.1. Shibbo- leth, an implementation of the SAML standard was developed as part of the "In- ternet2 Middleware Initiative" to address resource sharing among organizations that use dierent authentication and authorization methods. The implementation of Shibboleth in an Apache module mod_shib interfaces with Keystone and the identity provider.

OpenID connect OpenID Connect is an authentication layer on top of OAuth version 2.0 (Open Authorization), an open standard for token-based authorization and authentication for internet users to authorize websites and access their information on other websites without exchanging passwords. OpenID Connect was specically designed for federated identity management and SSO (Single SignOn) [8]. The users' identity is encoded in a secure JSON Web Token (JWT), which is portable and supports a range of signature and encryption algorithms. The Apache module mod_auth_openidc interfaces with Keystone and OpenID connect identity providers in OpenStack clouds.

2.2.4 Keystone as an IDP OpenStack also supports Keystone service itself being an identity provider, instead of depending on external identity providers. Thus, the credentials of one Keystone can be used to log into multiple OpenStack clouds that have their own Keystone services for authorization. The users log in through the IDP Keystone and receive a signed SAML assertion which they use to authenticate into the SP Keystones. This ow uses the SAML protocol outlined in section 2.2.3. Chapter 2. Conceptual Background 10

2.2.5 Mapping Keystone Federated identity is made possible through the use of a concept known as 'Mapping'. As the name suggests, there is a mapping of remote users with that of local users or groups. The Authorising Keystone service can then make policy decisions based on Role Based Access Control (RBAC). For this purpose, Open- Stack creates temporary 'ethereal' users which have control over the resources, these users are removed once their tasks are complete. Projects can also be provisioned at the time of authentication. The Figure 2.4 gives an illustration of the mapping document which is given as an argument to the mapping function of Keystone. In the gure, the users described by the regular expression "*@stu- dent.com" are mapped to a local group given by the group id. Thus the remote users can have the same access permissions as the local group.

2.3 Container Technology

Compared to Virtual Machines, containers are faster to start and stop and also use lesser resources for the same tasks. They do not need to boot an entire machine and only require a small part of the Operating System to run applications. This enables users to run a higher number of applications per physical machine and also multiple instances of the same application without dependency issues.

2.3.1 LXD LXD is a container management system that handles creation, conguration and networking in LXC containers. It is built on top of LXC containers which is an OS-level virtualization for running containers using two kernel features namely Cgroups and Namespaces. These features provide isolation of resources and manage access to them. The benet of LXD compared to other container technologies like Docker is its closeness in features to hypervisors like Xen and KVM. Thus it makes sense to use LXD instead of Docker containers to deploy software like OpenStack which is an incorporation of multiple applications and processes.

2.3.2 Docker Docker combines the isolation features of the kernel like Cgroups and Namespaces with a union-capable le system to start multiple containers that are streamlined to run one process or app and scale up and down as required. Docker containers also provide faster deployment and easier management of processes. Thus Docker is best suited for applications like MySQL, Apache web servers etc. Chapter 2. Conceptual Background 11

Figure 2.4: Mapping of remote user to local user Chapter 3 Related Work

Distributed Cloud computing and federated clouds have been a topic of research interest since the invention of virtualization technology. Khethavath et al. [17] combine the concept of distributed hash tables and an auction-based game model to address the issue of resource discovery and resource allocation in a distributed cloud. This paper deals only with a theoretical approach with some simulation. Cloud computing and P2P technologies were used by Mayer et al. [21] to provide a Platform-as-a-Service that emphasize on self-adaptability and reliability. Buyya et al. [6], Rochwerger et al.[30] and Munoz et al. [22] on the other hand, independently propose their architectures to manage federated clouds. However, they do not cover the scaling of these collaborative clouds and the architectures do not solve the complete decentralized management of the cloud services. Apart from the general approaches to federation in clouds, Decentralized Authentication mechanisms and algorithms are also investigated extensively in the research community. Yu et al. [37] explore the concept of trust in P2P networks and propose a distributed reputation mechanism to detect unwanted and malicious peers. Although the concepts and techniques are novel and eective against malicious peers, the authentication and identity management itself are not considered. Neuman et al. [24] described one of the earliest authentication systems and protocols, Kerberos, upon which the OpenStack Keystone service was based. Kerberos was implemented alongside Keystone by Ayed et al. [2] which also use access control lists along with authorization tickets to implement a generic access control system for clouds. The simulations are done in CloudSim software and also implemented in OpenStack with Kerberos integration. Al- though Kerberos included the concept of authenticating multiple Kerberos servers in a decentralized manner, it could not account for the issues of discovery and interoperability in hybrid clouds. Pustchi et al. [28] give a thorough and detailed elaboration of federated authorization and also propose a multi-cloud trust model at cloud, domain and project levels. Sitaram et al. [32] present a method of federating clouds that involves extending the OpenStack API to allow users to be authenticated against multiple clouds, including AWS clouds. This method has been superseded by the

12 Chapter 3. Related Work 13

OpenStack federated identity which is built into the newer versions of OpenStack. Chadwick et al. [8] explain the implementation and function of the federated identity extension of OpenStack that allows users of one cloud to get authenticated and use resources of other trusted clouds. This technology has been incorporated into OpenStack since the Kilo version release. Management and Scheduling of Virtual resources like VM and VMIs in large-scale virtual environments is an np-complete problem that is still a topic of interest and relevance for researchers of current times. Víctor Méndez et al. [22] describe the use of an image contextualization manager in their federated cloud architecture Rafhyc. In the paper, they utilize a golden boot image and necessary information for dierent contextualization methods. This method may be suitable for scientic applications and PaaS, but in the case of P2P-enhanced OpenStack clouds, it would not be feasible to include contextualization for every use case. Razhavi et al. [29] leverage the ability of hypervisors to start a VM with only a tiny part of the image locally stored. The remaining parts are downloaded on request. Caches are used to store the chunks of the image that are required to boot a VM. Wan et al. [36], Peng et al.[27], Lin et al. [19] and Kochut et al. [18] propose independent but similar solutions that use deduplication technology. The image is split into chunks and duplicate data blocks are avoided by using ngerprinting technology. This helps save storage space and reduces the load on the network. However, deduplication of image data in decentralized clouds with independent storage servers has not yet been studied. The works by Chen et al. [9] and Huang et al. [16] utilize the multi-source download capabilities of the BitTorrent protocol to deliver VM images. The VMs that boot from a single image will share the resources among themselves and reduce the network load and time in fetching images from the storage devices. However, these works do not deal with the complete decentralization of sourcing the images and is only ecient when several VMs start from the same images. The method of using P2P for provisioning images was also analyzed by [13] and an algorithm was proposed for server bandwidth provision for ecient distribution of chunks in the P2P model. In summary, the work presented in this paper builds on previous research to explore how Peer-to-Peer and distributed technologies can be used to address the issues of OpenStack scalability. While the earlier works focus on distributed and hybrid clouds and few works relate to using some P2P technologies for certain cloud functionalities, we study the possibilities of combining the two and evaluate the suitability of Peer-to-Peer technologies in OpenStack clouds. Chapter 4 Method

4.1 Overview of the applied methods

The methods followed in this thesis work are listed below and will be described in detail in the subsequent sections.

Analysis of the Basic Functional Requirements in OpenStack clouds.

Functional Analysis of the capabilities of available Peer-to-Peer technologies for network control applications and for OpenStack clouds.

Analysis of Numerical Scalability and Performance of Features of Peer-to- Peer architectures.

Inference Phase

Verication Phase: Implementation and Enhancement of P2P technologies in OpenStack clouds

Figure 4.1 illustrates the methods and phases employed in the thesis work. The rst three methods constituted the accumulation of data, processing it and analyzing the information. Then we performed inference and decided on what to implement. The nal stage was the implementation of the solution and testing it against the predicted behavior in a realistic scenario.

4.2 Analysis of Basic Functional Requirements in OpenStack

The thesis conducted an analysis of the services of OpenStack clouds and their requirements. The previous work on P2P-OpenStack clouds was studied and the technical scope and performance requirements were dened. The OpenStack architecture was analyzed by [12] and it was discovered that the scalability of the compute service, Nova, in terms of the number of compute nodes was limited.

14 Chapter 4. Method 15

Figure 4.1: Outline of phases and methods employed in the thesis

This was because of the limited number of socket descriptors in RabbitMQ message broker service and also scalability issues with certain components, which led to the inception of the P2P-enhanced OpenStack solution described by the paper. The paper also mentions the lack of a decentralized identity service in the solution presented. Thus after an analysis of the P2P-enhanced OpenStack solution, a brief outline of the requirements in OpenStack clouds is presented for improving their scalability. The requirements, based on the analysis done on OpenStack clouds and the P2P-enhanced OpenStack clouds described in Chapter 2 are given below.

4.2.1 Identity service The current P2P-enhanced OpenStack, as illustrated by Figure 2.1, uses a centralized Keystone node to manage authentication and authorization of all the clouds in the P2P network. This arrangement makes the single node vulnerable to security and performance issues since it can act as a bottleneck or single point of failure. Additionally, since this system involves the use of multiple clouds (spread across devices) that act as one P2P system, the resources of one user may also be spread across multiple clouds. This centralized Keystone service becomes a bottleneck for a system size exceeding 16 clouds. The challenge then becomes to authenticate one user across all the clouds (which accommodate the user's resources). It has to be noted that a P2P system may not have a global view of the system and also that the system should be able to authenticate a user belonging to any cloud dynamically [8]. Chapter 4. Method 16

Figure 4.2: Flow of image between OpenStack clouds

4.2.2 Image management The current implementation performed the distribution and management of Vir- tual machine images (VMI) among the dierent OpenStack clouds in the P2P overlay, by sending the entire binary image le from the source cloud to the target cloud. This implies that the image is rst copied from the glance storage back-end to the target glance service, which copies it to its back-end. When a request is sent to the target cloud to boot a VM using the image le, the compute node again copies the le from the glance back-end to its local le system. As it happens to be a P2P system, if such a request is received by a peer that is neither the source nor the target for the image le, as illustrated by Figure 4.2, it may need to download the entire source image le to its local storage and then upload the le to the target cloud. Since the image les are usually in the order of gigabytes, this process is inecient with respect to both, the network utilization and the le I/O.

4.2.3 Scheduling of Virtual Resources According to many research works including [26] , nding optimal solutions to allocate VMs to physical resources is an NP-complete problem. Scheduling of Virtual Machines becomes an even greater hurdle when it comes to P2P systems in which a scheduler does not have a global view of the system. The scheduling of VMs may depend on the policies that are employed in the organization as well as the requirements of the users themselves. The policies of the organization could be to balance the load on the entire system and reduce the utilization of the physical devices to accommodate more users, whereas the users may prefer to have their own VMs on the same network or in the vicinity. Hence there is a requirement to evaluate the scheduling algorithms and heuristics and choose the most optimal one for the right scenario and design a solution that dynamically chooses the algorithm or heuristic and allocates the VMs to the physical resources. Chapter 4. Method 17

4.2.4 Networking Service In the P2P-enhanced cloud environment, the users may require their VMs to be able to communicate with each other. Hence, if there are multiple VMs instantiated on dierent resources, there will be the need to create virtual overlay networks. If the VMs are booted in the same physical network, then their latency will be minimum, compared to those created in clouds outside the physical network, thus there will be a requirement to create and manage overlay virtual networks and subnets such that the end user only sees the networks that he is associated with and is not aected by the complexity of the underlying connections. It is to be noted that creation and deletion of VMs are tasks that occur very frequently in the cloud.

4.3 Functional Analysis of capabilities of Peer-to- Peer technologies for OpenStack

The functional design space spans the possibilities space from which the architecture of a specic P2P service overlay for a given control task can be chosen. In the case of OpenStack clouds, the functional design space would span the possibilities of all the functions of P2P systems that a given OpenStack service functionality can be extended with. In this paper, we have analyzed the functional components of dierent P2P systems and the functional requirements of OpenStack clouds with respect to scalability and performance and tried to classify the P2P functions based on the possibility and benets of their implementation in OpenStack clouds. Figure 4.3 was taken from the work described in [34]. The paper reports an initial work on investigating the functional possibilities of P2P architectures and additionally, the emphasis was on the two basic P2P functions, "resource mediation" and "resource access control and exchange". The virtualization of the P2P functions was also of interest, which had values ranging from 'centralized' to 'decentralized'. Figure 4.3 illustrates the two-dimensional design space and also a classication of the basic P2P architectures. The architecture of the P2P-enhanced OpenStack can be taken from the functional design space by selecting the most suitable P2P architecture for the particular application of OpenStack. An enhancement of individual services of Open- Stack may not require the incorporation of the entire architecture and thus an additional dimension can be added to the functional design space to include the individual functional components that can be taken from the P2P architecture. Figure 4.4 illustrates the additional dimension added to the original functional design space. For representational purposes, we show only a few P2P architectures and only two dimensions of the Functional design space. This design space consists of the feature set that serves the purpose of each of the P2P architectures. Chapter 4. Method 18

Figure 4.3: Functional design space of P2P service overlays

Each block in the gure represents one feature of a Particular P2P architecture. Thus, depending on the use-case, a feature can be taken from the feature set and incorporated into the OpenStack Services. A functional analysis was done on some of the popular P2P architectures and the functionalities are presented in brief as follows.

Chord Chord is a structured P2P system and uses Distributed Hash Tables in elegantly structuring the overlay topology. The randomness of peer-ID assignment implies that the successor peers may be diverse in ownership, jurisdiction and geography. This may lead to high overhead in the underlying physical network.

Pastry The Pastry protocol is a structured P2P overlay that takes into account the local- ity of the peers. Using a heuristic approach, it leverages two criteria, numerical closeness and local proximity simultaneously. Pastry networks are vulnerable to internal attacks like the Sybil attacks.

Kademlia Kademlia makes use of the shortest unique prex of a peer's identier. Every message is used to update the bucket of a Kademlia peer and hence it does not provide a periodic overhead to the underlying network. Kademlia also leverages Chapter 4. Method 19

Figure 4.4: View of the Additional Dimension of the Functional Design space parallel querying which eciently reduces the time taken to return a query response. The performance statistics of this protocol is well known for a large number of users.

CAN Content Addressable Network (CAN) maps the resources to individual points in a virtual coordinate system. Multiple dimensions and realities can be chosen which enables to add specic eciency requirements to the application. Its distinct feature among others is the mechanism of taking over the zone of another peer that had recently exited the system. The benets of CAN include Data availability, fault tolerance along with an ability to increase routing eciency by increasing the dimensions of the coordinate space. The complexity of the CAN architecture is one of its disadvantages.

Gnutella The Gnutella protocol has the advantage of serverless operations along with sim- plicity. The peers also do not require any additional synchronization when a node leaves the overlay. The disadvantages of Gnutella is in its use of ooding which generates high network trac [34]. Thus it does not scale well for large networks. The limited search implies that a lookup may not be exhaustive. Additionally, the randomness of the overlay leads to ineciency in operations. [23]

Edonkey Edonkey has a simple and centralized resource mediation architecture with a low number of server entities and thus each operation has a low number of hops. Chapter 4. Method 20

There is more control over the availability of the resources but as there are a low number of index servers, it is more prone to shutdowns and server overloads. Edonkey assumes that every peer is equal and altruistic and thus is limited in its adoption of peers that don't share resources ie., free riders.

BitTorrent BitTorrent is a content distribution overlay which focuses on quick replication of a single le by leveraging a sophisticated incentive based Multi-Source Download mechanism and Piece selection strategies. The centralized tracker provides more control over the resources and hence can help in maintaining control and to censor over the ow of information. The tracker can also become a single point of failure and can make the overlay vulnerable against attacks. A distributed tracker-less protocol has also been proposed.

CYCLON CYCLON overlays try to fulll the requirement of low diameter, are resilient to massive node failure and deal with high churn rates, while retaining a random graph property [35]. Killing half of the nodes of the network does not threaten the connectivity of the nodes.

Yappers Yappers uses a combination of Gnutella and DHT and can group nodes according to some attribute they have in common [11]. It can perform a lookup on every node as well as limit which nodes to search relative to the type of information searched.

4.4 Analysis of Numerical Scalability and Perfor- mance of Features of P2P Architectures

The thesis work involved a comprehensive literature study and analysis on the methods for evaluating the scalability and performance of Peer-to-Peer architectures. The main research was done on the unique and eective ways of evaluating the scalability of popular P2P architectures. The numerical scalability of P2P architectures has been evaluated by dierent methods and using dierent measures based on the type of the P2P architecture and the function that it fullls in the system. [33] introduces the concept of Stochastic scalability. Traditionally scalability deals with the size of the system but it would also be benecial to evaluate the mid-term and short-term stochastic behavior of the system to get a better idea of how the system would scale and Chapter 4. Method 21 perform. Many random variables like inter-arrival time, mean online time, query rate and churn rates are some of the measures of stochastic scalability of a system. In the case of Chord based P2P overlays which implement DHT algorithms and form a ring topology, the connectivity of the peers with respect to other peers is considered as important as the functional scalability of the system. Two important concepts have been introduced by [3] for evaluating the stability of Chord based platforms. Global disconnection i.e., the case where one specic peer loses all its connections to the overlay and Local disconnection i.e., where at least one peer in the overlay loses all its connections to the overlay. As the ring becomes larger, the probability of local disconnection may reduce but the probability of global disconnection will be higher. Thus evaluating these two probabilities can help evaluate the scalability of the system itself. Physical link delays also aect the performance of searches in a P2P overlay network. [4] evaluates the impact of network delay in the search times in DHT., to prove the scalability in very large chord rings and to guarantee QoS demands. It was found that the search delay in Chord based P2P rapidly increases in smaller ring sizes but stays moderate for very large populations. This is because, once the size of the population grows by the next power of 2, the nger table of the peers grows by one entry and thus an additional peer can be reached by only one hop. Thus, mean search delays slightly decreases as the population increases from 2i − 1 to 2i ∀i ∈ {0, 1, 2...}. [5] evaluates the scalability of a Kademlia based distributed network management tool by evaluating the search times for the peers with respect to the increasing size of the overlay. The overlay size was not the only criteria for evaluating the scalability of the system. The correctness of the system does not depend on the size of the system and thus, the average number of known neighbors were calculated with respect to the churn times of the P2P systems. In the case of Pastry based overlay described in [15], the Cumulative Distri- bution Function (CDF) was calculated with respect to the search times of the dierent scenarios. The probability of successful search with respect to the hops was also analyzed. Since the overlay network of Pastry does not consider the physical network layout, longer routes are taken as often as shorter routes with respect to the underlay and this could be a problem for some applications of P2P and thus an evaluation of the search times with respect to the underlying scale of the scenario was also considered. The paper also found that the leaf set of size 4 was a sucient balance in the trade-o between the improvements and the overhead introduced into the system. [14] and [25] evaluate the performance of an Edonkey based P2P overlay. Among others, they implemented a cache server which caches and serves the most popular les to eciently reduce the network usage. The performance of the caching peer was evaluated by analyzing the Complementary Cumulative Distribution Function (CCDF) with respect to the download time of a 3 MB le for the dierent applications of cache peers and also for the dierent churn times Chapter 4. Method 22 of peers.

4.5 Inference Phase

The thesis applied an inference phase where the functional and numerical scalability analysis of P2P systems were matched with the analysis of the basic requirements in OpenStack and P2P-enhanced OpenStack clouds, with the core objective of improving the scalability of P2P-enhanced OpenStack clouds. The following subsections will outline the decisions involved in implementing the solution.

4.5.1 Keystone solution decision From section 4.2.1, we found two primary requirements in case of Keystone service in P2P-enhanced OpenStack clouds, listed below.

1. Ability to authenticate a user against multiple clouds

2. Overcome the limitations of a centralized Keystone service

Thus, following the methodology, we matched these requirements with the capabilities of P2P technologies and designed a solution that is scalable and improves on the previous implementation.

Peer-to-Peer based solution The OpenStack identity service broadly provides two types of functions, Authenti- cation and Authorization. Keystone authenticates users by providing credentials when they register themselves to OpenStack services. Whenever a function call arises, the user authorization is veried by lookup of databases. If the verication is successful, a token is provided which includes a list of projects and roles assigned to the user and can be used for subsequent function calls, instead of re-authenticating. These two functionalities correspond with the two functionalities of P2P Architectures namely, resource mediation and resource-access-and- exchange [34]. Here, the resource is analogous to the user identity itself. The problems of identity management in the current P2P-enhanced OpenStack implementation viz., Storing Identities of all the clouds in the P2P overlay can be externalized to another entity which can reduce the load on the Keystone service. In this way, every cloud can be provisioned with its own Keystone service which can authorize users generated by the external entity without having to store every user's data in its database. BitTorrent, which has centralized resource mediation and decentralized resource access and exchange, solves this issue. The BitTorrent trackers provide the peers with information about other peers and resources, in Chapter 4. Method 23 the same way, the external identity provider provides the identities and each peer in the OpenStack P2P network authorizes these identities and grants access to its resources. The concept of Federated identity described in Chapter 2 enables the resource mediation function of Keystone to be externalized. From the possible solutions, Keystone-to-Keystone Federation was chosen as opposed to an external identity provider. Using the services of external IDPs introduces additional complex components to maintain and also security issues which are best avoided by making use of OpenStack components. Thus Keystone-to-Keystone was the most preferred option in designing the solution.

4.5.2 Image management solution decisions P2P algorithms have been implemented by the research community to speed up the provisioning of resources. [13] describes two models of P2P image management specically using BitTorrent-based implementations, but on images that are sourced from a centralized data center. When the number of VMs instantiated from a single image is not very large, these algorithms are ineective in reducing network load as explained by [27], which also proposes a chunk level distribution algorithm that leverages the ability of a VM to boot with only a small part of the image le. Thus, a small portion can be fetched rst and the rest can be fetched on demand. [16] and [18] both emphasize the caching of similar chunks of the images and use multipoint collaboration to minimize latency in fetching images. The technique used is to divide the image le into equal-sized or variable sized chunks and depending on the similarity of the VMIs in the host, locally reconstruct these chunks and only retrieve the dissimilar chunks. This method reduces the amount of information passing through the network apart from reducing the time taken to provision a VM. [29] describes a similar approach of caching the popular chunks and discusses the factors for VM instance slow-down namely, placement of the image on the physical node and transfer of the image to the compute node. The same issues occur in the case of P2P OpenStack clouds, the dierence being that the placement and transfer of images will be on a broader level i.e., clouds as opposed to nodes. [31] concentrated on eective replication of images to make them highly available and looks into optimizing the number and uniformity of placement of VMIs.

What is chosen and why? Even though there were several ways of improving the management of Virtual machine images, the conceptualized solution was similar to the implementation of [29] and the caching peer of [25] but instead of caching the chunks of the images, we leverage the caching of the whole binary image le. We designed a simple HTTP server which makes the image available through a URL and we also leverage the compute node of the OpenStack cloud as a cache server. In the Chapter 4. Method 24 previous implementation (Chapter 2), when an agent sends a request to boot a VM on a cloud which does not have the required image le, the whole image le is uploaded to the target cloud as described by Figure 4.2. The current solution does not upload the image le but makes the source image le available to the target cloud through a URL generated by the HTTP server itself. This URL is used to register an image in the Glance service of the target cloud and upon starting a VM, the respective compute node of the target cloud downloads the image from the URL and stores it in its local storage. Subsequent boots will cause the image to be directly used by the compute node and thus avoids unnecessary load on the network, without installing additional components to the OpenStack cloud.

4.6 Implementation and Verication

The implementation part of this thesis was carried out by extending the existing solution described by Chapter 2. Two functional requirements of OpenStack clouds mentioned in Section 4.2 were addressed in the case of implementation and evaluation. The implementation and evaluation of each functional requirement were done as follows.

4.6.1 Image management solution The image management issue was addressed by caching the binary image les using OpenStack-based components. In the P2P-enhanced OpenStack agent, an additional simple component was added. This component, not only creates an image entry in the glance registry of the target cloud but also serves the binary images via a URL available through Glance API. The component creates a symbolic link to the image on the local directory and then using REST API, it makes the image available to the OpenStack agent. Since this solution solved the issue that reduces the number of copies of the images made in the complete P2P system, actual tests were not considered to be necessary and the time spent in writing tests and verifying the reduction in the number of copies of images would not do justice to the decentralized Identity service solution and hence testing of the image management solution was not given priority.

4.6.2 Decentralized Identity Service solution The issue with the identity service was addressed by using the Federated identity extension of OpenStack, specically, we used the Keystone-to-keystone (K2K) Federation, described in detail in section 2.2.2, where one Keystone acts as an Identity Provider and all other Keystones in the system become the service Chapter 4. Method 25 providers. This solution eectively reduces the load on the centralized Key- stone service by separating the Authentication and authorization functions of Keystone. Thus the Keystone which acts as the Identity provider can, in future works, become decentralized itself. But in this thesis work, Keystone-to-Keystone Federation was used and implemented as follows based on [8]. The implemented Keystone should, in theory, be able to allow users of more number of clouds to get authorized to the system resources.

Figure 4.5: Architecture of the Identity service solution

Figure 4.5 represents the architecture implemented by this thesis work. The centralized component is basically a Keystone service that has been congured as the Identity Provider (IDP) and individual clouds of the entire system are provisioned with their own Keystone services which are congured as Service Providers (SP). The conguration is as described in section 2.2.2.

IDP The IDP is where users and identities are stored and veried upon request. It mainly consists of a Keystone service which is congured with SSL and provisioned to manage SAML assertions using Apache and Python modules.

SP The service provider is where the accesses to the cloud resources are dened and authorization is performed. The ow of requests between two agents with respect to authorization tokens is described in Figure 4.6. When an agent requests for Chapter 4. Method 26

Figure 4.6: Flow of Requests between Agents for Tokens an object like VM or image to be created on a particular cloud, the SP will verify with the IDP whether the user is valid and by the use of mapping and Role Based Access Control (RBAC), the user is authenticated. This process involves an exchange of SAML assertion which is basically an XML document that describes the roles and information about the user. Subsequently, a cookie is generated which is then used to retrieve the token from the target cloud. Once a token is received, an ephemeral user is created at the target cloud and the standard OpenStack ow is followed as if the cloud is local to the requesting user. Additionally, The tokens that were retrieved by a cloud from every other cloud is cached to be reused until they expire. The testing and evaluation of this work include the testing of the scalability and performance of the identity solution as a system and also the testing of the scalability of the clouds after incorporating this identity solution into the P2P system that is already existing, as described in chapter 2.

Experimental Setup for Identity management The Experiment for evaluating the scalability of the Keystone-to-Keystone Fed- eration setup involved the use of 5 servers provided through the tool MAAS (Metal-as-a-service). The specications of each server are:

24 Core processor

24 GB RAM

Ubuntu 16.04 (Xenial) OS

224.6 GB Storage

One Keystone IDP (Identity Provider) was setup and congured in a Docker container on one of the servers and the other four were used to run the SPs (Service Provider) along with one common OpenStack service, Glance. The test involved a maximum of 32 SPs and to avoid any resource deciency, only 8 SPs were setup in each server with each SP inside its own Docker container. The metrics involved are the CPU-time, response time and the number of SPs. CPU-time, provided by the Linux kernel, indicates the total time spent by the CPU in execution. The response time is the time taken to perform an Chapter 4. Method 27

OpenStack service request, specically, 'glance image-list'. Python scripts were written to send the requests to the target number of clouds and with a set number of repetitions and concurrency. The tests were performed for 64 requests and with a logarithmic increase in the number of SPs. The area under the CPU-time graph was calculated (the cumulative time spent by the CPU in performing all the requests). Thus the expected result for a scalable system would be that this cumulative execution time does not increase for an increasing number of Service providers. The tests were performed with 10 iterations and the values were plotted and presented in section 5.

Experimental Setup for P2P system To evaluate the decentralized identity solution as a part of the whole P2P system, the components of the decentralized solution were incorporated into the P2P system and tests were performed to compare this P2P-enhanced OpenStack cloud with a standard OpenStack cloud. Since the previous solution could only accommodate 16 clouds and was limited by the centralized Keystone component, the current implementation was evaluated for up to 32 clouds. The performance of the P2P system is evaluated under dierent scenarios, changing the size of the system as well as load on the system. The results of tests on the previous version of this system was documented in [12] which involved the use of VMs. In the current version we have set up the system in LXD containers and the results are documented. The OpenStack services corresponding to the controller node, compute node and the controller with agent services were deployed in a separate container. The resources were restricted to the containers to get an understanding of the system performance at its limit (2 CPU cores and 6 GB of RAM for the controller, 2 CPU cores and 4 GB of RAM for compute nodes). A separate container is assigned to deploy the Keystone IDP. To perform the tests, scripts were written which send requests to the systems, similar to users of a real cloud. The script attempts to boot a number of VM instances concurrently and keeps track of the boot time and success rate of the boots. The performance of the system is compared with that of an equivalent Open- Stack deployed as a standard set up. The resources of this standard OpenStack system are also constrained (4 CPU cores and 8GB of RAM for the controller, 2 CPU cores and 4GB of RAM for the compute). The P2P and the standard system are considered equivalent when they both have the same number of compute nodes. The test scripts in the case of the standard OpenStack sends all the requests concurrently to the controller node. Chapter 5 Results

The results section contains two subsections, section 5.1 gives the evaluation results of the identity solution set up as an independent system and section 5.2 gives the evaluation results of the P2P system that has been enhanced with the identity solution.

5.1 Decentralized Identity Solution

The following gures 5.1 and 5.2 represent the change in response times for 64 requests with respect to the change in the number of SPs in the identity solution set up as a separate system. The requests in this scenario is that of listing images (image-list) using the OpenStack Glance service. The axes of the following graphs are logarithmic to give us a proper perception of the changing parameters with respect to the logarithmic increase in the scale of the system. Figure 5.1 represents the results of the tests performed for 64 requests. The results show that the curve for the response times reduces when the system scales to 8 and then remains more or less constant for increasing scale of the system. Figure 5.2 represents the results of the tests performed for 64 requests. The results show that the curve for the average area under the CPU-time graph reduces when the system scales to 8 and then remains more or less constant for increasing scale of the system. To evaluate the stochastic scalability of the system, the tests were also run while changing number of requests (logarithmic increments from 1 to 10000 requests) along with a changing number of service providers and the results are represented in Figure 5.3. The results show that the response times are consis- tent and relatively lower for a higher number of SPs.

5.2 Peer-to-Peer System

We also ran experiments to understand how the P2P system incorporated with the identity solution performs when it scales, and how its performance compares to an equivalent standard OpenStack deployment. In this experiment, we start

28 Chapter 5. Results 29

Figure 5.1: Average Response times for 64 requests with a deployment of OpenStack with one cloud instance and congure a Python test script with one user. This script simulates a user and attempts to start 4 VMs concurrently and measures how long it takes, on average, to boot up a VM and also what fraction of the attempts failed. This is repeated 15 times. We then double the system size (and the number of users) and redo the experiment. This is continued until the system has 32 instances with 32 users generating load concurrently. The entire set of experiments are repeated with the equivalent standard OpenStack setup whereby the number of compute nodes is increased with the number of users generating the load. The results are provided in Figure 5.4. The following are the graphs showing a comparison of the performance of the P2P system (which incorporates the Keystone solution) and the standard OpenStack. The test scripts are run that boot 4 VMs concurrently and measure how long it takes to boot up a VM and also the percentage of VMs that failed. We then double the system size and redo the experiment until there are 32 users performing the boot requests concurrently. This test is similarly performed in the Standard OpenStack and the results are presented. The Figure 5.4 describes the change in average VM boot time and percentage of failed VMs with the increasing size of the system. The curves for the P2P system improves from system sizes larger than 8. At the size of 32, the boot times for the standard OpenStack is 70% more than that of the P2P system and the failure percentage is considerably larger compared to the P2P system. We Chapter 5. Results 30

Figure 5.2: Average area under cputime curve for 64 requests also notice that the performance of the P2P system does not show deterioration compared to the Standard system even at a system size greater than 16. We also performed additional tests by xing the system size at 32 and increasing the concurrency of the requests. Figure 5.5 shows that the Standard system has much greater fail rate and boot time compared to the P2P system for increasing concurrency of VM boot requests. Chapter 5. Results 31

Figure 5.3: Averages of Response times versus Number of requests and number of SPs

Figure 5.4: VM startup time and fail rate of the standard and P2P systems with increasing system size and load Chapter 5. Results 32

Figure 5.5: VM failure rate of the standard and P2P systems (size 32) with increasing concurrent VM requests Chapter 6 Analysis and Discussion

6.1 How can Peer-to-Peer architectures be characterized for their suitability in P2P-based Open- Stack clouds?

P2P architectures can be characterized by their unique methods and techniques implemented to support their scalability. From this thesis work and the existing solution that this work tries to improve on, it is clear that P2P architectures can be used in two ways in OpenStack based clouds.

Creating the overlay network of the OpenStack clouds

Enhancing services of the OpenStack clouds

OpenStack is a large suite of services that each provide a dierent functionality but collectively provides the cloud computing platform to the end users. The controller node is the backbone on which OpenStack functions and so an eective means of leveraging the strengths of multiple OpenStack clouds is to incorporate the P2P membership protocols into the controller nodes. From this thesis work, including the results in section 5, we have shown that the separation of resource mediation and resource-access-and-exchange as done in the BitTorrent protocol was eective in overcoming the issue of the centralized identity in the P2P system. We also proposed a solution to the image management issue by designing an HTTP server for images that also acts as a client to the Glance server. Thus the individual services of OpenStack can be enhanced by incorporating the unique strategies and functionalities that P2P architectures use to achieve high scalability. The failure rate in section 5.2 in the case of the Standard System is asserted to be because of the limitation of the single centralized controller in handling the larger requests and in the case of the P2P system, because of the limitations with the Python thread and multiprocessing libraries.

33 Chapter 6. Analysis and Discussion 34 6.2 How can P2P technology be used for decentralized identity service in OpenStack clouds and which P2P-technology should be applied for high scalability of this service?

Keystone as described in section 2.2.1 provides two functions, Authentication and Authorization. We have shown that separating the two functionalities is advantageous in enabling the authentication and authorization between clouds. This can be done by reconguring the Keystone service of each cloud to utilize only the authorization functionality while the authentication functionality can be externalized to another identity provider. The centralized component will be the authentication component and this provides the identities to the clouds. The decentralized component is the Key- stone service providers at each cloud that authorizes users of the system against the authorized resources of the system. The results of this thesis work show that the performance of this identity solution does not reduce for increasing scale of the system both as an independent service and also as part of the overall P2P-enhanced OpenStack cloud. On the contrary, it has been shown that for increasing scale of the system, the performance of the identity solution as an independent system improves as the scale increases. And more importantly, it has been shown that it has been possible to scale the system beyond 16 clouds and overcome the limitation of the centralized Keystone in the P2P system of the previous work. Chapter 7 Conclusions and Future Work

This thesis work analyzed the scalability issues in P2P-enhanced OpenStack clouds and also the functionalities and architectures of P2P systems. Solutions to the issues of the centralized identity and image management were proposed and evaluated. It was shown that Federated identity management component of OpenStack was eective in improving the identity service in P2P-enhanced OpenStack clouds. As future work, further research is essential to nd solutions to the other limitations of the P2P-enhanced OpenStack clouds, especially the networking between VMs of dierent clouds. Another direction to further this work is to nd approaches to completely decentralize the Identity Provider Keystone. It would also be interesting to see how functionalities of more P2P architectures can be used to enhance the increasing number of services of OpenStack.

35 References

[1] Luc Onana Alima, Ali Ghodsi, and Seif Haridi. A Framework for Struc- tured Peer-to-Peer Overlay Networks. In Global Computing, pages 223249. Springer, Berlin, Heidelberg, March 2004.

[2] Hella Kael-Ben Ayed and Bilel Zaghdoudi. A generic Kerberos-based access control system for the cloud. Annals of Telecommunications, 71(9-10):555 567, October 2016.

[3] A. Binzenhofer, D. Staehle, and R. Henjes. On the stability of chord-based P2p systems. In GLOBECOM '05. IEEE Global Telecommunications Con- ference, 2005., volume 2, pages 5 pp., November 2005.

[4] Andreas Binzenhöfer, Phuoc Tran-gia, Andreas Binzenhöfer, and Phuoc Tran-gia. Delay Analysis of a Chord-based Peer-to-Peer File-Sharing System. In in ATNAC 2004, 2004.

[5] Andreas Binzenhöfer, Kurt Tutschku, Björn auf dem Graben, Markus Fiedler, and Patrik Arlos. A P2p-Based Framework for Distributed Net- work Management. In Wireless Systems and Network Architectures in Next Generation Internet, pages 198210. Springer, Berlin, Heidelberg, July 2005.

[6] Rajkumar Buyya, Rajiv Ranjan, and Rodrigo N. Calheiros. InterCloud: Utility-Oriented Federation of Cloud Computing Environments for Scaling of Application Services. In Algorithms and Architectures for Parallel Pro- cessing, pages 1331. Springer, Berlin, Heidelberg, May 2010.

[7] Gonzalo Camarillo. Peer-to-Peer (P2P) Architecture: Denition, Tax- onomies, Examples, and Applicability. RFC 5694, November 2009.

[8] David W. Chadwick, Kristy Siu, Craig Lee, Yann Fouillat, and Damien Ger- monville. Adding Federated Identity Management to OpenStack. Journal of Grid Computing, 12(1):327, March 2014.

[9] Z. Chen, Y. Zhao, X. Miao, Y. Chen, and Q. Wang. Rapid Provisioning of Cloud Infrastructure Leveraging Peer-to-Peer Networks. In 2009 29th IEEE International Conference on Distributed Computing Systems Work- shops, pages 324329, June 2009.

36 References 37

[10] OpenStack Foundation. Home OpenStack Open Source Cloud Computing Software. https://www.openstack.org. Accessed: 2017-03-01.

[11] Prasanna Ganesan, Q. Sun, and H. Garcia-Molina. YAPPERS: a peer- to-peer lookup service over arbitrary topology. In IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Commu- nications Societies (IEEE Cat. No.03CH37428), volume 2, pages 12501260 vol.2, March 2003.

[12] Xin Han. Scaling OpenStack Clouds Using Peer-to-peer Technologies. Mas- ter's thesis, Chalmers University of Technology, Gothenburg, Sweden, 2017.

[13] Jian He, Yi Liang, Yonggang Wen, Di Wu, and Yupeng Zeng. On P2p Mechanisms for VM Image Distribution in Cloud Data Centers: Modeling, Analysis and Improvement. In Proceedings of the 2012 IEEE 4th Interna- tional Conference on Cloud Computing Technology and Science (CloudCom), CLOUDCOM '12, pages 5057, Washington, DC, USA, 2012. IEEE Com- puter Society.

[14] T. Hopfeld, K. Tutschku, F. U. Andersen, H. de Meer, and J. O. Oberen- der. Simulative performance evaluation of a mobile peer-to-peer le-sharing system. In Next Generation Internet Networks, 2005, pages 281287, April 2005.

[15] T. Hossfeld, S. Oechsner, and K. Tutschku. Evaluation of a pastry-based P2p overlay for supporting vertical handover. In IEEE Wireless Communications and Networking Conference, 2006. WCNC 2006., volume 4, pages 22912296, April 2006.

[16] B. Huang, R. Lin, K. Peng, H. Zou, and F. Yang. Minimizing Latency in Fetching Virtual Machine Images Based on Multi-point Collaborative Ap- proach. In 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, pages 262267, August 2013.

[17] P. Khethavath, J. Thomas, E. Chan-Tin, and H. Liu. Introducing a Dis- tributed Cloud Architecture with Ecient Resource Discovery and Optimal Resource Allocation. In 2013 IEEE Ninth World Congress on Services, pages 386392, June 2013.

[18] A. Kochut and A. Karve. Leveraging local image redundancy for ecient virtual machine provisioning. In 2012 IEEE Network Operations and Man- agement Symposium, pages 179187, April 2012.

[19] C. Lin, Q. Cao, H. Zhang, G. Huang, and C. Xie. SDVC: A Scalable Dedu- plication Cluster for Virtual Machine Images in Cloud. In 2014 9th IEEE References 38

International Conference on Networking, Architecture, and Storage, pages 8892, August 2014. [20] Steve Martinelli, Henry Nash, and Brad Topol. Identity, Authentication, and Access Management in OpenStack: Implementing and Deploying Keystone. O'Reilly Media, Inc., 1st edition, 2015. [21] P. Mayer, A. Klarl, R. Hennicker, M. Puviani, F. Tiezzi, R. Pugliese, J. Keznikl, and T. Bure. The Autonomic Cloud: A Vision of Voluntary, Peer-2-Peer Cloud Computing. In 2013 IEEE 7th International Conference on Self-Adaptation and Self-Organizing Systems Workshops, pages 8994, September 2013. [22] Víctor Méndez Muñoz, Adrian Casajús Ramo, Víctor Fernández Albor, Ri- cardo Graciani Diaz, and Gonzalo Merino Arévalo. Rafhyc: an Architecture for Constructing Resilient Services on Federated Hybrid Clouds. Journal of Grid Computing, 11(4):753770, December 2013. [23] Jari Mäntylä. Scalability of Peer-to-peer systems. Technical report, Helsinki University of Technology, 4 2005. [24] B. C. Neuman and T. Ts'o. Kerberos: an authentication service for computer networks. IEEE Communications Magazine, 32(9):3338, September 1994. [25] Jens O. Oberender, Frank-Uwe Andersen, Hermann de Meer, Ivan Dedinski, Tobias Hoÿfeld, Cornelia Kappler, Andreas Mäder, and Kurt Tutschku. En- abling Mobile Peer-to-Peer Networking. In Wireless Systems and Mobility in Next Generation Internet, pages 219234. Springer, Berlin, Heidelberg, June 2004. [26] Elina Pacini, Cristian Mateos, and Carlos García Garino. Multi-objective Swarm Intelligence schedulers for online scientic Clouds. Computing, 98(5):495522, May 2016. [27] C. Peng, M. Kim, Z. Zhang, and H. Lei. VDN: Virtual machine image distribution network for cloud data centers. In 2012 Proceedings IEEE INFOCOM, pages 181189, March 2012. [28] Navid Pustchi, Ram Krishnan, and Ravi Sandhu. Authorization Federation in IaaS Multi Cloud. In Proceedings of the 3rd International Workshop on Security in Cloud Computing, SCC '15, pages 6371, New York, NY, USA, 2015. ACM. [29] K. Razavi and T. Kielmann. Scalable virtual machine deployment using VM image caches. In 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC), pages 112, November 2013. References 39

[30] B. Rochwerger, D. Breitgand, A. Epstein, D. Hadas, I. Loy, K. Nagin, J. Tordsson, C. Ragusa, M. Villari, S. Clayman, E. Levy, A. Maraschini, P. Massonet, H. Muñoz, and G. Tofetti. Reservoir - When One Cloud Is Not Enough. Computer, 44(3):4451, March 2011.

[31] D. Shen, F. Dong, J. Zhang, and J. Luo. Cost-Eective Virtual Machine Image Replication Management for Cloud Data Centers. In 2014 IEEE Intl Conf on High Performance Computing and Communications, 2014 IEEE 6th Intl Symp on Cyberspace Safety and Security, 2014 IEEE 11th Intl Conf on Embedded Software and Syst (HPCC,CSS,ICESS), pages 229236, August 2014.

[32] D. Sitaram, H. L. Phalachandra, A. Vishwanath, P. Ramesh, M. Prashanth, A. G. Joshi, A. R. Desai, Harikrishna Prabhu C. R, Prafulla, Shwetha R, and Yashaswini A. Keystone federated security. In 8th International Con- ference for Internet Technology and Secured Transactions (ICITST-2013), pages 659664, December 2013.

[33] Phuoc Tran-Gia and Andreas Binzenhöfer. On the stochastic scalability of information sharing platforms. Technical Report 364, University of Wuerzburg, 6 2005.

[34] Kurt Tutschku. Peer-to-Peer Service Overlays Capitalizing on P2P Technology for the Design of the Future Internet. other, University of Würzburg,Institute for Computer Science, Department of Distributed Sys- tems (Informatik III), June 2008.

[35] Spyros Voulgaris, Daniela Gavidia, and Maarten van Steen. CYCLON: Inex- pensive Membership Management for Unstructured P2p Overlays. Journal of Network and Systems Management, 13(2):197217, June 2005.

[36] J. Wan, S. Han, J. Zhang, B. Zhu, and L. Zhou. An Image Management System Implemented on Open-Source Cloud Platform. In 2013 IEEE Inter- national Symposium on Parallel Distributed Processing, Workshops and Phd Forum, pages 20642070, May 2013.

[37] Bin Yu, M. P. Singh, and K. Sycara. Developing trust in large-scale peer-to- peer systems. In IEEE First Symposium onMulti-Agent Security and Sur- vivability, 2004, pages 110, August 2004.