Tommi Mikkonen Adnan Ashraf (Editors) Developing Cloud Software
Total Page:16
File Type:pdf, Size:1020Kb
Ivan Porres | Tommi Mikkonen Adnan Ashraf (Editors) Developing Cloud Software Algorithms, Applications, and Tools Turku Centre for Computer Science TUCS General Publication No 60, October 2013 Developing Cloud Software Algorithms, Applications, and Tools Edited by Ivan Porres Tommi Mikkonen Adnan Ashraf TUCS General Publication No 60, October 2013 ISBN 978-952-12-2952-7 ISSN 1239-1905 ii Preface Executed during years 2010-2013, Cloud Software is a Finnish research pro- gram, whose goal has been to significantly improve the competitive position of Finnish software intensive industry in global markets in the field of cloud computing. More than 30 Finnish IT companies and research organizations have participated in the program, and the output of the program has been considerable both in terms of scientific publications as well as industry trans- formation towards using cloud computing techniques. This book contains selected contributions by the Cloud Software Pro- gram partners focusing on innovative algorithms, applications, and tools to develop new services and applications to be deployed in public and private cloud infrastructures. These are needed to fully utilize the concept comput- ing as a utility as promised by cloud computing. In addition to the actual contributions, the first chapter of the book contains an introduction to cloud computing from the point of view of software development. For the rest of the book, each chapter tackles a concrete problem relevant to software de- velopment for cloud infrastructures that originated in the program company partners. We want to thank all the participants in the Cloud Software program for their e↵orts, shared experiences, and insights during these four years that have been materialized in the contents of the book. Specially, we want to thank Dr. Janne J¨arvinen, focus area director, Professor Veikko Sepp¨anen, academic coordinator, Dr. Tua Huomo, program coordinator, and Dr. Pauli Kuosmanen, Digile CTO, for leading the program. In addition, we wish to thank Hans Ahnlund and Mika Laurila (ECE Ltd.), Marc Englund (Vaadin Ltd.), Sami Helin (Steeri Oy), Jarno Kallio (Packet Video Finland), Risto Laurikainen (CSC), and Heikki Nousiainen (F-Secure) for performing reviews for the di↵erent chapters that constitute the book. Finally, we also wish to thank all the contributors of this book for their dedicated e↵orts both in the program itself as well as in reporting the results. Ivan Porres, Tommi Mikkonen, Adnan Ashraf October 2013 iii iv Acknowledgments This work was supported by TEKES, the Finnish Funding Agency for Tech- nology and Innovation, as part of the Cloud Software program of DIGILE, the Finnish Strategic Centre for Science, Technology and Innovation in the field of ICT and digital business. v vi Contents 1 Introduction to Cloud Computing Technologies 1 1.1 Technology Drivers and Adoption Problems . 2 1.2 Background Technologies . 7 1.2.1 Cloud Hardware Virtualization Technologies . 7 1.2.2 Sandboxing ........................ 9 1.2.3 VirtualizedNetworkBlockStorage . 10 1.2.4 NetworkVirtualization . 11 1.3 Scalable Cloud Technologies . 12 1.3.1 ScalableDatastore . 13 1.3.2 ScalableFileStorage . 17 1.3.3 ScalableBatchProcessing . 18 1.3.4 Approaches to Fault Tolerance in Cloud Infrastructure 20 1.3.5 Latency and Clouds . 21 1.4 CloudfromApplicationProgrammerView . 23 1.4.1 Infrastructure as a Service . 23 1.4.2 PlatformasaService. 27 1.4.3 SoftwareasaService . 31 1.4.4 Discussion......................... 36 1.5 Conclusions . 36 2 The Social Devices Platform: An Infrastructure for Social Devices in a Mobile Cloud 43 2.1 Introduction . 44 2.2 Social Devices . 45 2.2.1 Motivation and Background . 45 2.2.2 RunningExample. 47 2.2.3 Requirements and Characteristics . 49 2.3 The Social Devices Platform . 52 2.3.1 OverallArchitecture . 53 2.3.2 ProximityServer: Managing Proximity Information . 54 2.3.3 StateServer: Managing Device States and Properties . 54 vii 2.3.4 Controller: Triggering and Scheduling of Actions . 55 2.3.5 CaaS:ConfigurationofActions . 56 2.3.6 Orchestrator: ExecutionofActions . 57 2.3.7 SocialDevicesClient: The Client for Social Devices . 58 2.4 Related Work . 59 2.5 Discussion and Results . 62 2.6 Conclusions . 66 3 Prediction-Based Virtual Machine Provisioning and Admis- sion Control for Multi-tier Web Applications 71 3.1 Introduction . 72 3.2 Related Work . 75 3.2.1 VMProvisioningApproaches . 77 3.2.2 AdmissionControlApproaches . 78 3.3 Architecture............................ 80 3.3.1 Load Balancer . 80 3.3.2 GlobalController . 82 3.3.3 AdmissionController . 83 3.3.4 CloudProvisioner. 84 3.3.5 Entertainment Server . 84 3.3.6 ApplicationServer . 84 3.3.7 ApplicationRepository . 85 3.4 Algorithms............................. 86 3.4.1 Load Prediction . 89 3.4.2 The Server Tier . 90 3.4.3 TheApplicationTier . 92 3.4.4 AdmissionControl . 93 3.5 ExperimentalEvaluation . 94 3.5.1 VMProvisioningExperiments . 95 3.5.2 AdmissionControlExperiments . 100 3.6 Conclusions . 106 4 Proactive Virtual Machine Allocation for Video Transcoding in the Cloud 113 4.1 Introduction . 114 4.2 VideoBitStreamStructure . 116 4.3 SystemArchitecture . 118 4.4 VideoSegmentation. 120 4.5 ProactiveVMAllocationAlgorithms . 121 4.5.1 VMAllocationAlgorithm . 122 4.5.2 VMDeallocationAlgorithm . 124 viii 4.6 Load Prediction . 126 4.7 MPI-Based Distributed Video Transcoder. 127 4.8 SimulationResultsofVMAllocation . 128 4.8.1 Experimental Design and Setup . 128 4.8.2 ResultsandAnalysis . 131 4.9 Evaluation of MPI-Based Distributed Video Transcoder . 132 4.9.1 Experimental Design and Setup . 133 4.9.2 ResultsandAnalysis . 134 4.10 Related Work . 136 4.11 Conclusion . 138 5 Hadoop in Large Scale Data Analytics for Bioinformatics 145 5.1 Introduction . 146 5.2 MapReduce . 149 5.2.1 Execution Model . 150 5.2.2 DistributedFileSystem . 154 5.3 Hadoop .............................. 156 5.3.1 Pig . 158 5.3.2 Hive ............................ 159 5.3.3 HBase ........................... 162 5.4 HadoopinBioinformatics . 166 5.4.1 Hadoop-BAM . 167 5.4.2 SeqPig........................... 176 5.5 Closing Remarks . 176 6 Performance Testing in the Cloud Using MBPeT 191 6.1 Introduction . 191 6.2 Related Work . 193 6.3 The Performance Testing Process . 195 6.3.1 Model Creation . 195 6.3.2 Model Validation . 195 6.3.3 Test Setup . 195 6.3.4 Load Generation . 196 6.3.5 Monitoring ........................ 196 6.3.6 Test Reporting . 196 6.4 MBPeTToolArchitecture . 197 6.4.1 TheMasterNode ..................... 197 6.4.2 TheSlaveNode...................... 202 6.4.3 GraphicalUserInterface . 204 6.5 Model Creation . 204 6.5.1 Workload Characterization . 205 ix 6.5.2 WorkloadModelingUsingPTA . 206 6.6 PerformanceTestingProcess. 207 6.6.1 Test Setup . 207 6.6.2 Load Generation . 209 6.6.3 Test Reporting . 211 6.7 Experiments . 211 6.7.1 YAAS ........................... 211 6.7.2 TestArchitecture . 212 6.7.3 Load Models . 212 6.7.4 Experiment 1 . 215 6.7.5 Experiment 2 . 219 6.8 Conclusions . 222 7 Cloud Communication Service 227 7.1 Introduction . 228 7.2 Business Scenarios . 228 7.3 Past work . 230 7.3.1 Framework . 231 7.3.2 Quality of an Ontology . 232 7.3.3 Cloud Communication System as a Big Data Problem . 233 7.4 HumanInteraction ........................ 237 7.4.1 Information Filtering and Recommender Systems . 238 7.4.2 Content-based Information Filtering . 238 7.4.3 CollaborativeFiltering . 239 7.4.4 Knowledge Based Recommendation . 239 7.5 Implementation and User Interfaces of the Cloud Communi- cation System . 240 7.6 Conclusion . 245 8 HTML5 in Mobile Devices – Drivers and Restraints 249 8.1 Introduction . 249 8.2 Theoretical Background . 252 8.2.1 TechnologyEvolution. 252 8.2.2 Research Framework . 253 8.3 Technology Overview . 254 8.3.1 HTML5 .......................... 254 8.3.2 FirefoxOS......................... 256 8.4 Analysis .............................. 257 8.4.1 AddedValue ....................... 257 8.4.2 Ease of Experimentation . 259 8.4.3 Complementary Technologies . 260 x 8.4.4 IncumbentRole...................... 260 8.4.5 Technological Performance . 262 8.5 Summary of Results and Discussion . 263 8.5.1 General Results . 263 8.5.2 Practical Examples . 265 8.6 Conclusions . 267 9 Collaborative Coding Environment on the Web: A User Study 275 9.1 Introduction . 275 9.2 Collaborative Development: The CoRED Approach . 277 9.3 Experiment . 280 9.3.1 Research Questions . 280 9.3.2 Proof-of-concept Session . 281 9.3.3 Experiment Setup . 282 9.3.4 Surveys .......................... 283 9.4 Results . ..