Departamento de Eletrónica, Universidade de Aveiro Telecomunicações e Informática 2014 Eduardo Miguel Um toolkit web para integração de serviços cloud Coutinho Gomes de Pinho A web toolkit for cloud service integration Departamento de Eletrónica, Universidade de Aveiro Telecomunicações e Informática 2014 Eduardo Miguel Um toolkit web para integração de serviços cloud Coutinho Gomes de Pinho A web toolkit for cloud service integration Dissertação apresentada à Universidade de Aveiro para cumprimento dos requisitos necessários à obtenção do grau de Mestre em Engenharia de Com- putadores e Telemática, realizada sob a orientação científica do Doutor Carlos Manuel Azevedo Costa, Professor auxiliar do Departamento de Eletrónica, Telecomunicações e Informática da Universidade de Aveiro. o júri / the jury presidente / president Prof. Doutor António Manuel Melo de Sousa Pereira Professor catedrático da Universidade de Aveiro vogais / examiners committee Prof. Doutor Rui Pedro Sanches de Castro Lopes Professor coordenador da Escola Superior de Tecnologia e Gestão do Instituto Politécnico de Bragança Prof. Doutor Carlos Manuel Azevedo Costa Professor auxiliar da Universidade de Aveiro (orientador) agradecimentos / Agradecimentos a família, colegas e amigos, pelo constante apoio. acknowledgements Palavras Chave computação na cloud, interoperabilidade entre clouds, serviços web, engen- haria de software, aplicações web. Resumo A utilização do paradigma de computação na cloud está hoje generalizada em diferentes áreas da sociedade. No entanto, a utilização de recursos forne- cidos por múltiplos fornecedores de serviços tem um conjunto de problemas associados à normalização e interoperabilidade destes serviços. Os esforços para ultrapassar tal problema têm passado pela criação de especificações abertas e frameworks de integração. Contudo, o desenvolvimento de aplica- ções web levanta outras questões no que diz respeito ao acesso e gestão de recursos cloud por parte da lógica da aplicação executada no lado do cli- ente. Esta dissertação propõe e desenvolve uma plataforma extensível para a integração de serviços cloud, desenhada para satisfazer os requisitos e pa- drões comuns das aplicações web. A plataforma inclui políticas de controlo de acesso e mecanismos para a partilha, delegação e replicação de recursos. Finalmente, apresentam-se testes de desempenho da solução implementada, seguindo-se uma análise e discussão dos resultados obtidos. Keywords cloud computing, cloud interoperability, web services, software engineering, web applications. Abstract The latest trends on cloud and multi-cloud computing are well established in our society. However, the lack of interoperability raised a few issues that have been tackled with open standards and integration frameworks. Still, web ap- plication development adds a few more issues when accessing and managing cloud resources in the application’s logic. This thesis describes an extensi- ble platform architecture for portable cloud service integration, designed to satisfy requirements and usage patterns of web applications. Moreover, it im- plements access control policies and mechanisms for cloud resource sharing, delegation and replication. Finally, the thesis presents performance tests of the solution, along with an analysis and discussion of the results obtained. Contents Contents i List of Figures v List of Tables vii Glossary ix 1 Introduction 1 1.1 Overview1 1.2 Proposal2 1.3 Thesis Structure3 2 State of the Art 5 2.1 Web Services5 2.2 Cloud Computing6 2.3 Cloud Services7 2.3.1 Compute8 2.3.2 Data Storage9 2.3.3 Message Queues 10 2.3.4 Notification 10 2.4 Cloud Computing Interoperability and Sky Computing 10 2.4.1 Cloud Interoperability Standardization 11 2.4.2 Solutions for Cloud Service Integration 13 2.5 Service Delivery Cloud Platform 15 2.5.1 Storage Abstraction 17 2.5.2 Database Abstraction 18 2.5.3 Notification Abstraction 18 2.6 mOSAIC Project 18 i 2.6.1 Architecture 18 2.6.2 Components Description 20 2.6.3 API design 22 3 Platform Design 25 3.1 A cloud service platform oriented to web applications 25 3.2 Platform Requirements 26 3.3 The Architecture 27 3.4 Cloud Controller model 28 3.5 Cloud resource model and manipulation 30 3.6 Agents, Authentication and Access Control Policies 32 3.6.1 Resource Delegation 34 3.7 The API 37 3.8 The Plugin support 37 3.9 The Services 38 4 Platform Development 43 4.1 Pre-Development Considerations 43 4.1.1 mOSAIC Considerations 44 4.2 Data Models and Protocols 46 4.3 Cloud Controller 49 4.3.1 Cloud Controller Web Services 49 4.4 Remote Connector SDK 53 4.4.1 Internal Architecture 54 4.5 Cloud Provider Access Drivers 55 4.6 SDCP Runtime 57 4.7 Resource Type: Simple Database 59 4.7.1 AWS SimpleDB 62 4.7.2 Client Module 62 5 Results and Discussion 63 5.1 Benchmarking 63 5.1.1 Environment Conditions and Sequence of Operations 63 5.1.2 Results 64 5.1.3 Observations 66 5.2 Discussion topics 67 5.2.1 mOSAIC Integration 67 5.2.2 Security 67 5.2.3 Benefits 68 5.2.4 Drawbacks 69 ii 6 Conclusion 71 6.1 Future Work 72 References 73 iii List of Figures 2.1 Service Delivery Cloud Platform legacy architecture 16 2.2 Main sub-systems of the mOSAIC architecture 20 2.3 API layers of mOSAIC 24 3.1 SDCP Concept Diagram 28 3.2 SDCP Cloud Controller Model 29 3.3 SDCP Remote Connector model, containing an example of dynamic remote connector access information. 31 3.4 SDCP’s cloud resource hierarchy model 32 3.5 SDCP Agent Authentication and Access Control Diagram 33 3.6 Resource delegation in SDCP, client-oriented approach. 36 3.7 Resource delegation in SDCP, server-oriented approach. 36 3.8 Filestore resource model. 39 3.9 Simple database resource model. 40 3.10 Notification service resource model. 41 4.1 Remote connector internal structure 54 4.2 Internal structure of a mOSAIC connector-driver pair implementation 56 5.1 A comparison of performance between the use of SDCP for simple database operations and direct SimpleDB access. 66 v List of Tables 4.1 A simple comparison of web server applications and technologies. 44 4.2 JSON document definitions in SDCP. 48 4.3 Main SDCP cloud controller web service. 51 4.4 Administration SDCP cloud controller web service. 51 4.5 Cloud access SDCP cloud controller web service. 51 4.6 Internal SDCP cloud controller web service. 52 4.7 Client Runtime API description. 58 4.8 SDCP Simple Database remote connector web service description. 61 5.1 The benchmarking results containing average and maximum execution times of each operation type, in milliseconds. 65 vii Glossary ACL Access Control List JSON JavaScript Object Notation AMQP Advanced Message Queuing Protocol NaaS Notification as a Service API Application Programming Interface NIST National Institute of Standards and AWS Amazon Web Services Technology CaaS Communication as a Service OCCI Open Cloud Computing Interface CAMP Cloud Application Management for OS Operating System Platforms PaaS Platform as a Service CCIF Cloud Computing Interoperability QoS Quality of Service Forum REST Representational State Transfer CDMI Cloud Data Management Interface RPC Remote Procedure Call CFP Call For Proposals SaaS Software as a Service CORS Cross Origin Resource Sharing SDCP Service Delivery Cloud Platform CSB Cloud Service Broker SDK Software Development Kit CSV Comma Separated Values SLA Service-Level Agreement DaaS Data-Storage as a Service SOA Service Oriented Architecture DBMS Database Management System DICOM Digital Imaging and Communications SOAP Simple Object Access Protocol in Medicine SQL Structured Query Language FTP File Transfer Protocol TLS Transport Layer Security GAE Google App Engine URI Universal Resource Identifier GICTF Global Inter-Cloud Technology Forum URL Universal Resource Locator HTTP Hyper-Text Transfer Protocol UUID Universally Unique IDentifier HTTPS HTTP Secure VM Virtual Machine IaaS Infrastructure as a Service WSDL Web Service Definition Language IDL Interface Description Language XML Extended Markup Language IP Internet Protocol XMPP Extensible Messaging and Presence JPA Java Persistence API Protocol ix chapter 1 Introduction 1.1 overview Web applications can deliver a program to the user without a previous installation process on the client machine, while still being able to delegate part of the application’s logic (mostly the user interface) to the local machine, regardless of its operating system. The modern HTML5 standard [1], while still under development, has already increased the potential of many web applications with capabilities such as new local storage mechanisms and JavaScript objects. These additional elements significantly reduce the need for third-party dependencies, like for instance, Adobe Flash. Furthermore, from the fact that web applications lift many resource requirements from the local machine, the development of cross-platform applications for mobile devices has become feasible and worthwhile. While the evolution of web applications has been a reality, it has also brought new challenges. There are many critical applications on the web that need to assure reliability and scalability in order to support their business models correctly, with a minimal chance of failure. The cloud computing concept emerged to become a rather acceptable solution to the deployment of web services, bringing several advantages like, for instance, elasticity, scalability, robustness, flexibility and fault tolerance. The consequence is that many enterprise services are handled by cloud platforms. Amazon Web Services (AWS) [2], Microsoft Windows Azure [3], Rackspace Online [4], Heroku [5] and PubNub [6] are only a few of the cloud providers that are publicly available for service provisioning to their customers. They supply a distinct range of service types, including but not limited to, storage, computing, notification and database services. The outsourcing of applications, including web applications, became prevalent for more demanding services. One of the main drawbacks of these cloud 1 providers resides at their interfaces, which are not interoperable. Thus, it leads to additional effort when a service needs to be migrated or augmented to another provider.