Building blocks of scalable applications Ciprian Cr˘aciun Scientific adviser: Prof. Dr. Dana Petcu West University of Timi¸soara,Romania 9th July 2012 Abstract Today's most demanding applications are social networking or social media sites, which are built in the emerging cloud computing environment, using the newest up to date techniques and tech- nologies. Some of these applications are built on top of specialized software platforms, hosted by the cloud providers, called PaaS. Therefore the current work tries to propose such an open source PaaS solution. It starts with an analysis of the business and economic environment surrounding these applications, then presents a set of case studies for both some popular applications, but also a few promising commercial platforms. Following that it expresses the requirements distilled from what has been observed, then an overall architecture and design of the proposed platform. It concludes by describing the experiences gathered while implementing a prototype of the platform, as part of the European FP7 research project mOSAIC. Abstract Cele mai populare aplicat¸ii ale zilelor noastre sunt de regul˘acele de \social media" sau \social networking", construite ^ıngeneral ^ıncontextul domeniului emergent al \cloud computing". Multe dintre acestea sunt de fapt construite utiliz^andplatforme specializate, oferite de furnizorii de servicii \cloud", numite PaaS. ^In acest context lucrarea de fat¸˘a^ı¸sipropune s˘adescrie ¸sis˘aimplementeze o astfel de platform˘a, disponibil˘aca ¸siproiect \open source". Pentru ^ınceputlucrarea prezint˘ao scurt˘aanaliz˘aa mediului economic ce guverneaz˘aaceste aplicat¸ii, urm^andapoi prezentarea c^atorva studii de caz, at^atal unor aplicat¸ii populare, c^at¸sia c^atorva platforme comerciale. ^In contextul expus, se trece la identificarea cerint¸elor software ale platformei finale, apoi la descrierea arhitecturii ¸sia designului general al acesteia. ^In final sunt prezentate experient¸ele practice acumulate ^ıncercˆandimplementarea unui prototip al acestei platforme, activitate conex˘aproiectului european de cercetare mOSAIC. Contents 1 Introduction 1 2 IT ecosystem 5 2.1 Cloud computing . .5 2.1.1 Characterization . .5 2.1.2 Models . .7 2.2 Economical context . .7 2.3 Development methodologies . 10 2.4 Developer roles . 10 2.5 Target applications . 13 2.5.1 Generic layers . 13 2.5.2 Generic architecture . 14 2.5.3 \The right mix".................................. 16 2.6 Miscellaneous remarks . 17 3 Case studies 19 3.1 Application case studies . 19 3.1.1 Reddit . 19 3.1.2 LinkedIn . 21 3.1.3 RightScale patterns . 23 3.2 Platform case studies . 23 3.2.1 Heroku . 23 3.2.2 Cloud Foundry . 29 3.2.3 App Engine . 30 4 Design 31 4.1 Inspiration . 31 4.2 Concepts . 32 4.2.1 Developer, user, and provider . 32 4.2.2 Application . 32 4.2.3 Infrastructure . 33 4.2.4 Platform . 34 4.2.5 Component . 34 4.2.6 Resource . 35 4.2.7 Management and service interface . 36 4.2.8 Service . 36 4.2.9 Component hub . 37 4.2.10 Controller . 37 4.2.11 Node . 37 i 4.2.12 Cluster . 38 4.2.13 Tool . 38 4.2.14 Sub-system and agent . 38 4.3 Requirements . 39 4.3.1 Functional requirements . 39 4.3.2 Architectural requirements . 42 4.3.3 Non-functional requirements . 43 4.3.4 Constraints . 44 4.4 Architecture . 45 4.4.1 Layers . 45 4.4.2 Inter-sub-systems architectural style . 45 4.4.3 Intra-sub-systems architectural style . 46 4.5 Life-cycles . 47 4.5.1 Application and platform life-cycle . 47 4.5.2 Node life-cycle . 48 4.5.3 Agent, component, service and resource life-cycle . 49 4.6 Sub-systems . 51 4.6.1 Component hub . 52 4.6.2 Component harness . 52 4.6.3 Component controller . 54 4.6.4 Container controller . 55 4.6.5 Node controller . 57 4.6.6 Credentials service . 60 4.6.7 Miscellaneous services . 60 4.7 Component hub and controller . 64 4.7.1 Dynamo's model . 64 4.7.2 Component hub role . 66 4.7.3 Component controller role . 68 4.7.4 Design . 69 4.8 Credentials service . 72 4.9 Security . 74 4.9.1 Beliefs . 74 4.9.2 Assumptions . 75 4.9.3 Options . 76 4.9.4 Approach . 76 4.9.5 Challenges . 77 4.9.6 Solutions . 78 5 Implementation 83 5.1 Guidelines . 83 5.2 Backing technologies . 85 5.2.1 Programming languages . 85 5.2.2 OS, distributions, virtualization . 87 5.2.3 Load balancers, proxies . 88 5.2.4 Distributed databases . 89 5.2.5 Messaging systems . 90 5.2.6 Data formats . 90 5.3 Outcomes . 91 5.3.1 mOSAIC component controller and component hub . 91 ii 5.3.2 mOSAIC resource and components . 92 5.3.3 mOSAIC \Real time feeds"application . 93 6 Conclusions 97 6.1 Summary . 97 6.2 Open problems and future work . 98 6.3 Acknowledgements . 98 Bibliography 99 References . 99 Miscellaneous . 100 A Repositories i B Acronyms iii C Glossary v iii Chapter 1.