Contributions to Large-scale Distributed Systems The infrastructure Viewpoint - Toward Fog and Edge Computing as The Next Utility Computing Paradigm? Adrien Lebre To cite this version: Adrien Lebre. Contributions to Large-scale Distributed Systems The infrastructure Viewpoint - To- ward Fog and Edge Computing as The Next Utility Computing Paradigm?. Distributed, Parallel, and Cluster Computing [cs.DC]. Université de Nantes, 2017. tel-01664343 HAL Id: tel-01664343 https://tel.archives-ouvertes.fr/tel-01664343 Submitted on 14 Dec 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Université Bretagne Loire Habilitation à Diriger des recherches Presentée par Adrien Lebre École Doctorale Sciences et technologies de l’information, et mathématiques Discipline : Informatique et applications, section CNU 27 Unité de recherche : IMT Atlantique, Inria, LS2N Soutenue le 1er Sept. 2017 Contributions to Large-scale Distributed Systems The infrastructure Viewpoint Toward Fog and Edge Computing as The Next Utility Computing Paradigm? Contributions aux systèmes distribués à large échelle la vision infrastructure Vers le Fog et l’Edge computing comme nouveau modèle d’informatique utilitaire ? M. Claude Jard, Président Professor in Computer Science at University of Nantes Head of Nantes Digital Science Laboratory (LS2N). M. Erik Elmroth, Rapporteur Professor in Computing Science at Umea University, Sweden Head of Umea University research on distributed systems. M. Manish Parashar, Rapporteur Distinguished Professor of Computer Science at Rutgers University, USA Director of the Rutgers Discovery Informatics Institute (RDI2). M. Pierre Sens, Rapporteur Professor at the University of Pierre et Marie Curie (Paris 6), France Head of the REGAL Resarch Group. M. Frédéric Desprez, Examinateur Senior Researcher at Inria Deputy Scientific Director at Inria It does not matter how slowly you go as long as you do not stop. — Confucius Acknowledgments Because I see now how overloaded can be the schedule of senior researchers, I would like to start these acknowledgments by expressing my gratitude to all the members of the committee for accepting to evaluate m work despite their respective obligations. It is a real honor for me to have such a prestigious committee. This document covers most of the activities I made since the defense of my Phd in 2006. During these ten years, I had the opportunities to work with many French as well as Foreign colleagues. While these are acknowledged throughout the document, I would like, here, to warmly thank all of them for the collaborations we had and that have made the researcher I am today. I should highlight that I am much indebted to • Dr. Christine Morin for showing me what a community can accomplish ; • Prof. Mario Südholt for the freedom and trust he has given me in each action I have wanted to achieve since my arrival in his team ; • Dr. Frédéric Desprez for his confidence and guidance he has never stopped to give me during this period. Special thanks go to the Phd candidates, Post-docs and Engineers I had the chance to work with. In addition to performing the real work that makes research progress- ing, they have been key persons that through their competences and efforts have kept my enthusiasm and my energy to conduct research activities intact. The contribu- tionstothefieldofdistributedcomputingIam gladtopresentinthisdocumentcould not have been achieved without them. Finally, my most affectionate thanks go to: • My Friends who are throught our weekend outings and meals the best way for me to “recharge my batteries” ; • My parents-in-law who indirectly have participated to the work presented in this manuscript, in particular by taking care of my kids each time it has been necessary ; • My parents who after 39 years still continue to impress me for the devotion they have for their five children ; • My brothers, sister(s) (in-law) and their kids with whom I should have spent much more times than what I have done ; • Mychildren,Elie-LouandJonaswhoare,bytheirdailysmiles,anendlesssource of happiness ; • And, my lovely wife, Dorothée, who is, in good times and bad, the most won- derful person I have met. iii Table of Contents I Introduction 1 1 Introduction 3 1.1 GeneralContext.............................. 3 1.2 ScientificContributions . 4 1.2.1 DynamicschedulingofVMs . 5 1.2.2 VirtualizationandSimulationtoolkits . 6 1.2.3 BeyondtheClouds........................ 7 1.3 LookingBacktotheFuture . 7 1.4 OrganizationoftheDocument . 9 II Dynamic Virtual Machine Scheduling 10 2 VM Cluster-Wide Context Switch 13 2.1 ChallengeDescription . 13 2.2 OurProposal:AGenericVMsContextSwitch . 15 2.2.1 Fundamentals........................... 15 2.2.2 ArchitectureOverview. 16 2.3 Proof-of-Concept&Validations . 20 2.3.1 Proof-of-Concept . .. .. .. .. .. .. .. .. .. .. .. 20 2.3.2 Experiments ........................... 20 2.4 RelatedWork ............................... 22 2.5 Summary.................................. 23 3 Distributed Virtual Machine Scheduler 25 3.1 Addressing Scalability, Reactivity and Fault-tolerant Aspects . 25 3.1.1 ChallengeDescription . 25 3.1.2 OurProposal:DVMS ...................... 27 3.1.3 Proof-Of-Concept&Validation. 30 3.1.4 Summary ............................. 33 3.2 LocalityAware-SchedulingStrategy. 34 3.2.1 ChallengeDescription . 34 3.2.2 OurProposal:ALocality-awareOverlayNetwork . 35 3.2.3 Proof-of-Concept&Validation . 38 3.2.4 Summary ............................. 40 3.3 RelatedWork ............................... 41 4 Conclusion & Open Research Issues 43 III Virtualization and Simulation toolkits 46 5 Adding virtualization capabilities to SimGrid 49 5.1 ChallengeDescription . 49 5.2 SimGridOverview ............................ 50 5.3 Ourproposal:SimgridVM. 51 5.3.1 AddingaVMWorkstationModel . 52 5.3.2 AddingaLiveMigrationModel. 54 5.3.3 SimGridVMAPI(CandJava). 58 5.4 Validation ................................. 59 5.4.1 ExperimentalConditions . 61 5.4.2 EvaluationoftheVMWorkstationModel . 62 5.4.3 LivemigrationswithvariousCPUlevels. 63 5.4.4 LiveMigrationsunderCPUContention . 64 5.4.5 LiveMigrationsunderNetworkContention . 65 v vi TableofContents 5.5 RelatedWork ............................... 66 5.6 Summary.................................. 67 6 Virtual Machine Placement Simulator 69 6.1 ChallengeDescription . 69 6.2 OurProposal:VMPlaceS. 70 6.3 DynamicVMPPAlgorithms . 72 6.4 Experiments................................ 74 6.4.1 AccuracyEvaluation . 74 6.4.2 AnalysisofEntropy,SnoozeandDVMS . 75 6.5 RelatedWork ............................... 77 6.6 Summary.................................. 78 7 Conclusion & Open Research Issues 81 IV Beyond the Clouds: Recent, Ongoing and Future Work 85 8 The DISCOVERY Initiative 87 8.1 Context&Motivations . 87 8.2 FromCloudtoFog/EdgeComputingFacilities . 89 8.3 RevisingOpenStackThroughaBottom/UpApproach . 91 8.3.1 DesignConsiderations. 91 8.3.2 TheChoiceofOpenStack . 92 8.4 InnovationOpportunities . 94 8.5 Summary.................................. 96 9 Revising OpenStack to Operate Fog/Edge Computing Infrastruc- tures: a First Proof-Of-Concept 99 9.1 ChallengeDescription . 99 9.2 RevisingOpenStack............................ 100 9.2.1 DistributingtheAMPQBus. 101 9.2.2 DistributingtheDatabases . 101 9.2.3 TheNovaPOC:FromMySQLtoRedis . 102 9.3 ExperimentalValidation . 103 9.3.1 ImpactofRedisw.r.tMySQL . 104 9.3.2 Multi-siteScenarios . 106 9.3.3 CompatibilitywithAdvancedFeatures. 107 9.4 Summary.................................. 108 10 Conducting Scientific Evaluations of OpenStack 109 10.1 ChallengeDescription . 109 10.2 Background ................................ 110 10.2.1 DeployingOpenStackandcontrollingExperiments . 110 10.2.2 EnosDefaultBenchmarks. 110 10.2.3 MonitoringandGatheringMetrics. 111 10.3 EnOS.................................... 111 10.3.1 EnOS Description Language for Flexible Topologies . 111 10.3.2 EnOSWorkflow ......................... 112 10.4 Experiments................................ 113 10.5 RelatedWork ............................... 115 10.6 Summary.................................. 116 11 Ongoing and Future Work: A Software Stack for Fog/Edge In- frastructures 119 Bibliography 125 V Appendix 135 A Résumé long en francais 137 a.1 Contextegénérale............................. 137 a.2 ContributionsScientifiques . 138 a.2.1 Placementdynamiquedemachinesvirtuelles . 138 TableofContents vii a.2.2 Virtualisationetbibliothèquesdesimulation . 140 a.2.3 Audelàdel’informatiqueennuage . 142 a.3 Unerecherchedirigéeparl’expérimentation . 143 B Detailled Curriculum Vitae/List of Publications/Invited Talks 145 Part I INTRODUCTION This habilitation thesis presents an overview of the major research activities I conducted as part of the ASCOLA Research Group of the Ecole des Mines de Nantes (now IMT Atlantique), where I have held an Associate Professor position since October 2008. ASCOLA is a joint team oftheAutomation,ProductionandComputerScienceDepartmentofIMTAtlantique,theInria research center in Rennes, and the “Laboratoire des Sciences du Numérique de Nantes” (LS2N,