D1.2 Unicorn Reference Architecture Unicorn Reference Architecture Deliverable D1.2 Editors Athanasios Tryfonos Demetris Trihinas Reviewers Julia Vuong (CAS) Panos Gouvas (Ubitech) Sotiris Koussouris (Suite5) Date 29 September 2017 Classification Public 1 D1.2 Unicorn Reference Architecture Contributing Author # Version History Name Partner Description Table of Contents (ToC) and partner Demetris Trihinas UCY 1 contribution assignment. Athanasios Tryfonos UCY 2 Updated section 8 and subsections structure. Unicorn System Requirements and User Role Zacharias Georgiou UCY 3 Overview. State-of-the art for Cloud Application Design George Pallis UCY 4 and Management & Cloud Application Security and Privacy Added Reference Architecture diagrams and Fenareti Lampathaki Suite5 5 description Added Section 5 flow diagrams and updated Sotiris Koussouris Suite5 6 architecture description Updated architecture based on received Spiros Koussouris Suite5 7 feedback. Added unicorn use-cases Merged content regarding reference Manos Papoutsakis FORTH 8 architecture. Merged content regarding state- of-the-art Updated introduction and added executed Giannis Ledakis Ubitech 9 summary and conclusion Merged content for demonstrators and Panagiotis Gouvas Ubitech 10 implementation aspects Merged content regarding architecture flows Julia Vuong CAS 12 and new diagrams, updated introduction and state-of-the-art Merged updates regarding demonstrators, Spiros Alexakis CAS 13 use-cases and introduction Minor refinements on all document. Erik Robertson Redikod 14 Document finalized for internal review. 2 D1.2 Unicorn Reference Architecture 15 Reviewer comments addressed Inserted Table of Abbreviations and updated 16 Executive Summary 17 Final Version 3 D1.2 Unicorn Reference Architecture Contents Contents 4 1 Introduction 12 1.1 Document Purpose and Scope 13 1.2 Document Relationship with other Project Work Packages 13 1.3 Document Structure 14 2 State of the Art and Key Technology Axes 16 2.1 Micro-Service Application Development Paradigm 16 2.2 Cloud Application Design and Management 19 2.2.1 Cloud Application Portability, Interoperability and Management 19 2.2.2 Monitoring 20 2.2.3 Elastic Scaling 22 2.3 Cloud Application Security and Data Privacy Enforcement 23 2.3.1 Data privacy-by-design and encrypted persistency 23 2.3.2 Security and Data Restriction Policy Enforcement Mechanism 24 3.3.3 Risk and Vulnerability Assessment 25 2.4 Containerization and Cluster Orchestration 26 3 Unicorn System Requirements and User Role Overview 30 4 Unicorn Reference Architecture 33 4.1 Motivational Example 40 4.2 Cloud Application Development and Validation 41 4.3 Application Deployment 45 4.4 Monitoring and Elasticity 47 4.5 Security and Privacy Enforcement 49 5 Unicorn Use-Cases 53 6 Unicorn Demonstrators 79 6.1 Enterprise Social Data Analytics 79 6.1.1 Overview 79 6.1.2 Technical Implementation 80 6.1.3 Business and Technical Challenges 81 6.1.4 Demonstrator Relevance to Unicorn Use Cases 83 6.2 Encrypted Voice Communication Service over Programmable Infrastructure 84 6.2.1 Overview 84 6.2.2 Technical Implementation 84 6.2.3 Business and Technical Challenges 85 4 D1.2 Unicorn Reference Architecture 6.2.4 Demonstrator Relevance to Unicorn Use Cases 86 6.3 Prosocial Learning Digital Game 87 6.3.1 Overview 87 6.3.2 Technical Implementation 87 6.3.3 Business and Technical Challenges 88 6.3.4 Demonstrator Relevance to Unicorn Use Cases 89 6.4 Cyber-Forum Cloud Platform for Startups and SMEs 90 6.4.1 Overview 90 6.4.2 Technical Implementation 91 6.4.3 Business and Technical Challenges 94 6.4.4 Demonstrator Relevance to Unicorn Use Cases 95 7 Implementation Aspects of Reference Architecture 96 7.1 Version Control System 98 7.2 Continuous Integration 98 7.3 Quality Assurance 99 7.4 Release Planning and Artefact Management 99 7.5 Issue Tracking 99 8 Conclusions 101 9 References 102 5 D1.2 Unicorn Reference Architecture List of Figures Figure 1: Technologies that Unicorn relies on and will contribute to 12 Figure 2: Deliverable Relationship with other Tasks and Work Packages 14 Figure 3: Container-based Virtualization 27 Figure 4: Usage of Linux containerization toolkit by Docker 28 Figure 5: CoreOS Host and Relation to Docker Containers 28 Figure 6: Identified Unicorn Actors 31 Figure 7: Non-functional requirements 32 Figure 8: Unicorn Reference Architecture 33 Figure 9: Eclipse CHE High-Level Architecture 34 Figure 10: Unicorn Eclipse CHE Plugin Overview 36 Figure 11: High-Level Unicorn Orchestration 37 Figure 12: Tosca Topology Template 38 Figure 13: Unicorn Core Context Model Mapping 40 Figure 14: Content Streaming Cloud Application 41 Figure 15: Cloud Application Development and Validation 44 Figure 16: Application Deployment 46 Figure 17: Monitoring & Elasticity Flow 48 Figure 18: Security Enforcement 50 Figure 19: Privacy Enforcement 51 Figure 20: Unicorn Use Case UML Diagram 53 Figure 21: S5 Enterprise Data Analytics Suite*Social Architecture 80 Figure 22: CAS SmartWe and OPEN Deployment 92 Figure 23: CAS OPEN Architecture 93 Figure 24: Major releases of Unicorn Integrated Framework 97 Figure 25: Development Lifecycle 98 List of Tables Table 1: Mapping of functional requirements to user roles 31 Table 2: Policies for Content Streaming Application 41 Table 3: Define runtime policies and constraints use-case 54 Table 4: Develop Unicorn-enabled cloud application 54 Table 5: Package Unicorn-enabled cloud application 55 Table 6: Deploy Unicorn-compliant cloud application 56 Table 7: Manage the runtime lifecycle of a deployed cloud application 57 Table 8: Manage privacy preserving mechanisms into design time 58 Table 9: Manage privacy enforcement on runtime 59 Table 10: Manage security enforcement mechanisms 60 Table 11: Manage security enforcement mechanisms (enabler enforces security/privacy constraints) 61 Table 12: Monitor application behaviour and performance 62 Table 13: Adapt deployed cloud applications in real time 63 Table 14: Get real-time notifications about security incidents and QoS guarantees 64 Table 15: Perform deployment assembly validation 65 6 D1.2 Unicorn Reference Architecture Table 16: Perform security and benchmark tests 66 Table 17: Manage cloud provider credentials 67 Table 18: Search for fitting cloud provider offerings 68 Table 19: Define application placement conditions 68 Table 20: Develop code annotation libraries 70 Table 21: Develop enablers enforcing policies via code annotations 71 Table 22: Provide abstract description of programmable cloud execution environment through unified API 71 Table 23: Develop and use orchestration tools for (multi-)cloud deployments 72 Table 24: Manage programmable infrastructure, service offerings and QoS 73 Table 25: Ensure secure data migration across cloud sites and availability zones 73 Table 26: Ensure security and data privacy standards 74 Table 27: Monitor network traffic for abnormal or intrusive behaviour 75 Table 28: Manage the Unicorn core context model 76 Table 29: Manage enablers enforcing policies via code annotations 77 Table 30: Manage cloud application owners 78 Table 31: Enterprise Social Data Analytics Relevance to Use Cases 83 Table 32: ubi:phone Relevance to use cases 86 Table 33: Prosocial Learning Relevance to use cases 89 Table 34: Cyber-Forum Relevance to use cases 95 7 D1.2 Unicorn Reference Architecture Executive Summary Unicorn deliverable D1.2 – Unicorn Reference Architecture, hereafter simply referred to as D1.2, moves one step closer to the fulfillment of the vision of the project which is the development of a framework that facilitates EU- wide digital SME’s and startups to deploy cloud applications following the micro-service paradigm to multi-cloud execution environments. In Unicorn D1.1 Stakeholders Requirements Analysis [1], we analyzed the particular and demanding ICT needs of SMEs and startups by trawling leading industry studies and conducting personalized interviews with our target audience. Through this analysis we extracted the functional and non-functional requirements and user roles for the Unicorn Framework eco-system. Furthermore, we identified gaps in the industry and academia that Unicorn fills in. Based on this comprehensive analysis, we define in D1.2 the overall architecture of Unicorn and the components that comprise it, in complete alignment with the derived functional and non-functional requirements. The figure above illustrates a high-level overview of the Unicorn Reference Architecture. It is comprised of three distinct layers: i) the Unicorn Cloud IDE Plugin, ii) the Unicorn Platform and iii) the Multi-Cloud Execution Environment. 8 D1.2 Unicorn Reference Architecture The Unicorn Cloud Plugin IDE is organized into two perspectives(facets). At the Development Perspective, Application Developers, via the Annotated Source Code Editor develop secure, elastic, and privacy-aware cloud applications using the annotative Design Libraries and Product Managers define design-time, run-time and privacy policies and initiate the deployment process. At the Management Perspective, Application Administrators, using the intuitive Graphical User Interface of the plugin, can monitor and manage deployed applications. The plugin itself is built on top of the popular and open-source cloud IDE Eclipse Che [2], developed and maintained by the Eclipse Foundation community. Reasoning behind Che being Unicorn’s IDE of choice originates from Unicorn’s ICT SME/Startup survey results presented in D1.1, that have shown that Eclipse Che is currently the most popular cloud IDE among