A Java Middleware for High Performance Computing (HPC) and Internet of Things (Iot)
Total Page:16
File Type:pdf, Size:1020Kb
UNIVERSIDADE FEDERAL DE OURO PRETO A Java middleware for High Performance Computing (HPC) and Internet of Things (IoT) Leonardo de Souza Cimino Universidade Federal de Ouro Preto Orientador: Joubert de Castro Lima Disserta¸c~aosubmetida ao Instituto de Ci^encias Exatas e Biol´ogicasda Universidade Federal de Ouro Preto para obten¸c~aodo t´ıtulode Mestre em Ci^enciada Computa¸c~ao Ouro Preto, Setembro de 2017 C490j Cimino, Leonardo de Souza. A Java middleware for High Performance Computing (HPC) and Internet of Things (IoT) [manuscrito] / Leonardo de Souza Cimino. - 2017. 81f.: il.: color; grafs; tabs. Orientador: Prof. Dr. Joubert de Castro Lima. Dissertação (Mestrado) - Universidade Federal de Ouro Preto. Instituto de Ciências Exatas e Biológicas. Departamento de Computação. Programa de Pós- Graduação em Ciência da Computação. Área de Concentração: Ciência da Computação. 1. Internet . 2. Computação de alto desempenho. 3. Middleware. I. Lima, Joubert de Castro. II. Universidade Federal de Ouro Preto. III. Titulo. CDU: 004.738.5 Catalogação: www.sisbin.ufop.br A Java middleware for High Performance Computing (HPC) and Internet of Things (IoT) Leonardo de Souza Cimino Universidade Federal de Ouro Preto Orientador: Joubert de Castro Lima ii Um middleware Java para Computa¸c~aode Alta Performance (HPC) e Internet das Coisas (IoT) Resumo Middlewares est~aoem todo lugar e provavelmente permanecer~aoem todo lugar por muito tempo para ajudar a reduzir a complexidade no desenvolvimento de aplica¸c~oes. Existem aplica¸c~oesque necessitam da integra¸c~aoentre as tecnologias Internet das Coisas (IoT) e Computa¸c~aode Alto Desempenho (HPC), entretanto, n~aoexiste nenhuma al- ternativa de middleware que aborda os problemas de ambas ao mesmo tempo. Em rela¸c~aoa HPC, os middlewares apresentados n~aoabordam solu¸c~oespara servi¸cosde sensoriamento, atua¸c~aoe sensibilidade de contexto. Em rela¸c~aoa IoT, os middlewares apresentados n~aoabordam solu¸c~oespara simplificar processamento paralelo/distribu´ıdo de prop´ositogeral e servi¸cosde armazenamento. Para reduzir a lacuna entre as tecnolo- gias HPC e IoT, inicialmente apresentamos uma revis~aosobre a integra¸c~aoentre IoT e HPC. Posteriormente, descrevemos um middleware IoT/HPC chamado JavaC´a&L´a (JCL) com uma API capaz de programar diferentes categorias de dispositivos e que suporta tr^esmodelos de programa¸c~ao. JCL ´eum middleware j´aexistente e desen- volvido originalmente para HPC. Neste trabalho, realizamos diversas melhorias para torn´a-louma solu¸c~aoque suporta o desenvolvimento de aplica¸c~oesintegradas IoT/HPC. As principais contribui¸c~oesdeste trabalho s~ao:i) uma revis~aoda literatura e uma an´alise comparativa entre os principais middlewares IoT e HPC; ii) a defini¸c~aode uma lista de requisitos que um middleware IoT/HPC deve suportar baseado na literatura; iii) um middleware que atende a maioria dos requisitos apresentados anteriormente, atenunando a lacuna existente na literatura de middlewares. Os resultados mostraram que o JCL ´eescal´avel, possui instala¸c~aosimplificada, aborda a maioria dos requisitos IoT/HPC e iii torna simples o desenvolvimento de aplica¸c~oesIoT/HPC port´aveis e distribu´ıdaspara as plataformas Android e Java, sendo capaz de rodar em clusters compostos por dispositivos Java, Android e Arduino. iv A Java middleware for High Performance Computing (HPC) and Internet of Things (IoT) Abstract Middleware systems are everywhere and most likely will remain everywhere for a long time to help reduce the complexity of application development. There are real user-applications (UAs) that need the integration between Internet of Things (IoT) and High Performance Computing (HPC), however there is no middleware alternative that addresses issues of both IoT and HPC technologies at the same time. In terms of HPC, the presented middleware solutions did not address solutions for sensing, actuating and context awareness services. In terms of IoT, the presented middleware solutions did not address solutions to simplify general-purpose parallel/distributed processing and storage services. To reduce the gap between HPC and IoT technologies, we first present a review about IoT and HPC integration. After that, we describe an IoT/HPC middleware so- lution, called JavaC´a&L´a(JCL), with an API capable of programming different device categories and supporting three programming models. JCL is an already existing mid- dleware solution originally designed for HPC. In this work, we made many improvements to turn it a middleware that supports the development of IoT/HPC integrated applica- tions. The main contributions of this work are: i) a literature review and comparative analysis of the main IoT and HPC middleware solutions; ii) the definition of a list of requirements based on the literature that an integrated IoT/HPC middleware solution should support; iii) a middleware that attends most of the requirements that an inte- grated solution should have, attenuating the existing gap in the middleware literature. The results demonstrated that JCL scales well, has a simple deployment, addresses most of the fundamental IoT/HPC requirements and it is very simple to develop portable and v integrated IoT/HPC distributed applications for Android and Java platforms that run over clusters composed of Java, Android and Arduino devices. vi Declara¸c~ao Esta disserta¸c~ao´eresultado de meu pr´opriotrabalho, exceto onde refer^enciaexpl´ıcita´e feita ao trabalho de outros, e n~aofoi submetida para outra qualifica¸c~aonesta nem em outra universidade. Leonardo de Souza Cimino vii Agradecimentos Primeiramente a Deus por mais um objetivo alcan¸cado; Ao meu pai, minha m~aee meu irm~ao por estarem sempre presentes e me darem for¸casnos momentos dif´ıceis; A` minha namorada por ser meu porto seguro, por todo apoio e compreens~aodurante esse per´ıodo; Ao professor Joubert por todo conhecimento transmitido e pela dedica¸c~aoaplicada a este trabalho; A toda equipe do laborat´orioHPC por toda ajuda; A todos aqueles que me ajudaram de alguma forma durante essa trajet´oria. Contents List of Figures xi List of Tables xiii 1 Introduction 1 1.1 Goal . .3 1.1.1 Specific Goals . .3 1.2 Contributions and Results . .3 1.3 Work organization . .3 2 IoT and HPC integration: revision and perspectives 5 2.1 Abstract . .5 2.2 Introduction . .5 2.3 IoT/HPC middleware requirements . .7 2.3.1 Security and Privacy . .7 2.3.2 Context Awareness . .8 2.3.3 Different Programming Models . .9 2.3.4 Distributed Storage . .9 2.3.5 Heterogeneity . 10 2.3.6 Simple Deployment . 11 viii CONTENTS ix 2.3.7 Fault Tolerance . 11 2.3.8 General-purpose Computing . 12 2.3.9 Code Refactoring . 12 2.3.10 Task Scheduler . 13 2.3.11 Super-peer Support . 13 2.3.12 Collaborative Development . 14 2.3.13 Performance and Scalability . 14 2.3.14 Sensing Analytics . 15 2.4 IoT/HPC integration: discussion and perspectives . 15 2.4.1 IoT Middlewares . 15 2.4.2 CoT Middlewares . 17 2.4.3 HPC Middlewares . 19 2.4.4 Perspectives . 21 2.5 Conclusion . 22 3 A middleware solution for integrating and exploring IoT and HPC capabilities 24 3.1 Abstract . 24 3.2 Introduction . 25 3.3 Related Work . 27 3.4 JCL middleware proposal . 29 3.4.1 Simple Deployment . 30 3.4.2 Security and Privacy . 33 3.4.3 Heterogeneity . 34 3.4.4 Context Awareness . 36 x CONTENTS 3.4.5 Code Refactoring . 37 3.4.6 Processing and Storage Scheduling . 38 3.4.7 Different Programming Models . 39 3.4.8 Super-peer support . 42 3.4.9 Collaborative Development . 43 3.4.10 Distributed Storage . 44 3.4.11 Performance and Scalability . 46 3.4.12 General-purpose computing . 52 3.4.13 Comparative analysis . 52 3.4.14 Developing an IoT/HPC application in JCL . 54 3.5 Conclusion . 55 4 Conclusion 57 Bibliography 59 List of Figures 3.1 JCL Services . 31 3.2 JCL multi-computer deployment view . 31 3.3 JCL multi-computer deployment view . 32 3.4 JCL multi-computer deployment view . 33 3.5 Runtime to register an application, requiring code conversions - Java to Android . 35 3.6 Runtime to register an application, requiring code conversions - Android toJava..................................... 35 3.7 Business logic - Hello World . 37 3.8 Distribution logic - Hello World . 38 3.9 Distribution logic - Complex Hello World . 39 3.10 1 to 16 concurrent UA getSensingDataNow() calls . 41 3.11 1 to 250 concurrent UA getSensingDataNow() calls . 41 3.12 Logical network partition using JCL-Super-Peer components . 42 3.13 JCL-Super-Peer managing different number of JCL-Hosts . 43 3.14 Overhead when the number of JCL-Super-Peers increased . 43 3.15 Runtime to store sensing data when the number of sensors varied . 45 3.16 Runtime to store sensing data when the interval time varied . 45 3.17 Runtime to store sensing data when the number of devices increased . 46 xi xii LIST OF FIGURES 3.18 JCL-Host CPU usage to publish sensing data periodically from several sensors . 48 3.19 JCL-Host CPU usage to publish sensing data according to different intervals 48 3.20 Arduino memory usage to publish sensing data periodically from several sensors . 49 3.21 JCL-Host memory usage to publish sensing data periodically from several sensors . 49 3.22 JCL-Host memory consumption to publish sensing data according to dif- ferent intervals . 49 3.23 JCL-Host CPU usage versus Zanzito when sensors increased . 53 3.24 JCL-Host memory consumption versus Zanzito when sensors increased . 53 3.25 JCL-Host battery usage versus Zanzito when sensors increased . 54 3.26 High-level vision of the Moving Object Observer application . 55 3.27 Moving Object Observer IoT Application .