A Classification and Comparison Framework for Cloud Service Brokerage Architectures

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 1 A Classiﬁcation and Comparison Framework for Cloud Service Brokerage Architectures

Frank Fowley, Claus Pahl, Pooyan Jamshidi, Daren Fang, Xiaodong Liu

Abstract—Cloud service brokerage and related management and marketplace concepts have been identified as key concerns for future cloud technology development and research. Cloud service management is an important building block of cloud architectures that can be extended to act as a broker service layer between consumers and providers, and even to form marketplace services. We present a 3-pronged classification and comparison framework for broker platforms and applications. A range of specific broker development concerns like architecture, programming and quality are investigated. Based on this framework, selected management, brokerage and marketplace solutions will be compared, not only to demonstrate the utility of the framework, but also to identify challenges and wider research objectives based on an identification of cloud broker architecture concerns and technical requirements for service brokerage solutions. Cloud architecture concerns such as commoditisation and federation of integrated, vertical cloud stacks emerge.

Index Terms—Cloud Broker; Service Brokerage; Architecture Patterns; Cloud Broker Classiﬁcation; Service Management. !

1 INTRODUCTION perspective. Marketplaces, like app stores, are broker Several organisations active in the cloud technology applications that assemble and provide end-user ser- area, such as Gartner, Forrester and NIST [18], [15], vices. These broker applications often run on cloud [28], have identified cloud service brokerage as an platforms with specific broker-oriented capabilities. important business model, but also as an architectural Such a dedicated framework does not exist for challenge that needs to explore how to best construct cloud brokers and goes beyond existing service tax- broker applications on top of suitable platforms. A onomies such as [20]. Grozev and Buyya [19] go cloud service broker manages the use, performance beyond this and introduce a taxonomy and compare and delivery of cloud services and negotiates rela- solutions. However, their effort is focussed on a tax- tionships between cloud providers and cloud con- onomy for inter-cloud architectures. Their comparison sumers [15]. Cloud service management, an important scheme uses 5 basic aspects (type/organisation, archi- building block of cloud architectures, can be extended tecture, brokering approach, application type, aware- to act as a brokerage layer between consumers and ness). The definitions provided by Gartner, Forrester providers, and to even form marketplaces. Archi- and NIST [18], [15], [28] are also high-level and not tecture, development and quality concerns are key suitable for a detailed classification and comparison. enablers of any service brokerage solution that inter- In [16], the need for a multi-dimensional classification mediates between different providers by integrating, is recognised and a basic multi-facetted framework aggregating and customising their individual services. is proposed. However, it does not clearly distinguish Our main contribution is a classification framework between application and platform dimensions, lacks for cloud service management, brokerage and market- a detailed vocabulary at concept instance level, nor place solutions that use some form of intermediation does it provide a systematic identification, extraction, mechanism that goes beyond Gartner, Forrester and modelling and evaluation of the framework. NIST and aims to resolve different existing interpre- This classification framework is based on a broader tations. In addition to providing a comprehensive classification in terms of capability and feature cat- definition, our framework allows us to classify and egories, architectural patterns and a more refined, compare broker solutions. We look at architectural descriptive scheme specifically looking at architec- concerns, i.e., software-based intermediation solutions ture, language and quality as technical aspects. The that support brokerage as a business model. We emphasis here is on a systematic determination of distinguish between an application and a platform classification categories based on different cloud and software bodies of knowledge. We compare selected • F. Fowley is with the Irish Centre for Cloud Computing and Commerce cloud service management and brokerage solutions IC4, Dublin City University, Ireland. to illustrate and evaluate the framework, and also to • C. Pahl is with the Free University of Bozen-Bolzano, Italy. E-mail: [email protected] (contact author) derive trends and challenges from this comparison. • P. Jamshidi is with Imperial College London, UK. The selection of subjects is again systematic, aiming • D. Fang and X. Liu are with Edinburgh Napier University, UK. to identify technically advanced solutions that cover the cloud service broker space comprehensively to 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 2

ensure that the adequacy and the completeness of the typical broker role (agent), the application types the framework is properly evaluated. Based on an (application) and the typical functionality of brokered identification of cloud broker architecture patterns for (or mediated) services (functions). service brokerage solutions, we discuss challenges. • Aggregation is about delivering two or more Service description mechanisms discussed can serve services to possibly many consumers, not neces- to abstract, manipulate and compose cloud service sarily providing new functionality, integration or offerings in an effort to commoditise the cloud. These customisation, but typically offering centralised description mechanisms serve two purposes: Firstly, management of SLAs and security. to abstractly capture, present and manipulate cloud – Agent: distributor resources. Secondly, to serve as a starting point to – Application: marketplace, cloud provisioning link to configuration and other deployment concerns – Functions: discovery, billing, marketplaces in federated clouds. Thus, commoditisation and fed- • Customisation is about altering or adding capa- eration emerge as challenges from our discussion. bilities in order to improve the service function- The paper is organised as follows. Cloud service ality and provide enhanced service analytics. management and brokerage is introduced and anal- – Agent: independent software vendor (ISV) ysed in Section 2. Section 3 defines the comparison – Application: analytics, monitoring, interface framework. In Section 4, we apply the comparison – Functions: wrapper, adaptivity framework to selected solutions and evaluate its fit- ness for purpose. This investigation leads to a broader • Integration involves making independent ser- research challenges discussion, before ending with vices work together as a combined offering. This conclusions in Section 5. could be the interworking between layers of the vertical cloud stack, or could involve the data/process integration within a single layer. 2 CLOUD SERVICE BROKERAGE -LITERA- Techniques such as transformation, mediation TURE REVIEWAND ANALYSIS and orchestration are the classic solutions. We start with a review of existing cloud service – Agent: systems integrator (SI) brokerage definitions to obtain a consolidated list – Application: integrated PaaS of broker types (Section 2.1) and a list of their – Functions: orchestration, mashup, mediation typical capabilities (Section 2.2). We revisit the bro- NIST uses aggregation, arbitrage and intermediation ker types to define the scope of emerging archi- as the three core broker types [28]. NIST and Gartner tectural patterns for cloud service brokers (Section agree on the importance of aggregation. Some further 2.3). Then, we investigate the cloud stack deploy- commonalities exist. NIST intermediation and Gartner ment models – namely, infrastructure (IaaS) and plat- customisation focus on enhancing existing services. form/applications (PaaS/SaaS) - to determine layer- NIST arbitration and Gartner integration both con- specific concerns (Section 2.4). Finally, we discuss the sider the flexible mediation and integration of differ- perspectives of different user types (Section 2.5). We ent systems. Otherwise, the views diverge. NIST in- adopt the NIST definitions [28] refering to capabilities cludes arbitrage (which supports dynamic pricing in a provided covering processing, storage, networking cloud service marketplace), whereas Gartner does not. resources (IaaS), deployment of applications created NIST intermediation differs from Gartner integration. using programming languages, libraries and tools Intermediation between consumers and providers, in (PaaS) and using a providers applications (SaaS). the NIST understanding, includes a range of capabilities (SLA management, invoicing/billing consolidation) as opposed to a merely technical integration. 2.1 Brokerage - Definitions and Analysis • Gartner’s classification is building on traditional We begin by focusing on a brokerage-as-a-service IT roles, which can be seen as a limitation. The delivery model based on intermediation as a tech- role of the aggregation broker matches the tradi- nical principle. Advanced multi-cloud management tional distributor role, the integration broker role platforms, as well as brokers as marketplaces, can be aligns with a systems integrator and, finally, the considered as specific models in the wider broker- customization broker corresponds to an indepen- age space. Forrester, Gartner and NIST define Cloud dent software vendor role – note that we have Service Brokerage (CSB) in different ways [15], [18], singled out the agent in the Gartner summary. [28]. Gartner and NIST both follow a three-pronged • NIST’s terminology is not undisputed either. A classification. They define a cloud broker as an entity service intermediary should be a party with no that manages the use, performance and delivery of commercial aims, i.e., no consumer cloud policies cloud services and negotiates between providers and should be affected by the commercial bias of an consumers, i.e., intermediates between both [15]. intermediary. An intermediary can play the role In this initial overview of key concepts, we start of a broker (receiving a commission is included in with Gartner [18]. For each broker type, we name the activity), but might choose not to. Note that 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 3

NIST defines five actors (including brokers) in its From the discussion in Section 2.1 based on the Cloud Computing Reference Architecture, which different definitions from NIST and Gartner and from we will simplify later (see Section 2.4). an analysis of features that commercial broker plat- Forrester starts with the assumption that the cloud forms provide, we extract five rather than the usual broker model offers IT and telecom service providers three core broker capabilities following [12], see Table 1. and other vendors the opportunity to overcome the We started with capabilities and features from Section rapid commoditisation of their existing services busi- 2.1 that we then validated using the descriptions of ness and build a sustainable service delivery model. A the commercial tools. These reflect how the different simple broker model compares similar cloud provider mediated services are constructed, which is part of options and dynamically provisions selected services the platform perspective. The capabilities referred to based on the actual spot prices of these resources. by Forrester (such as hosting or management) can A full broker goes beyond this to include cloud be mapped to integration and intermediation. We bursting, i.e., the dynamic relocation of workloads defined these capabilities and associated capability from private environments to cloud providers and types. Composition as a type combines application vice versa. Forrester’s model is based on three core services or application and platform services. Manage- cloud models [40]: tool vendor (software focus), in- ment refers to platform-provided management capa- frastructure provider (infrastructure focus) and cloud bilities. Adaptation refers to a specialisation of a service builder (consultancy focus). A cloud broker is defined to meet a specific user profile. Distribution is involved as the combination of three capabilities arising from if services originate from different cloud providers. three core models: To further describe the features of a broker for a multi-cloud or a federated cloud setting beyond the • SaaS Provider - combines software and infrastruc- capabilities definition, we need to provide a more ture focus with hosting and management. exhaustive list of lower-level features that are used to • Value-Added Reseller - combines software and create SLA and service management, self-service user consultancy focus with management and also access and authorisation functions. The following are customisation and integration capabilities. extracted from the list given in [40], supplemented by • Integrator - combines infrastructure and consul- other sources such as the features offered by commer- tancy focus with integration and hosting. cial broker platforms. They are associated with a set We use these definitions to extract five core broker of feature categories – which are the relevant broker types and a vocabulary of capabilities and features. capabilities they apply to, as well as the management view from the capability types, see Table 2. 2.2 Broker Mediation Construction - Capabilities and Features 2.3 Management, Broker Platform and Market- The broker architecture needs to be divided into two place - Scope and Patterns layers – broker platform and broker application: Cloud brokerage applications are built up on exist- • The platform is the implementation platform on ing virtualisation techniques, cloud platforms, and which a broker application is implemented. This IaaS/PaaS/SaaS offerings. Emerging from the earlier platform can be provided ’as-a-service’. The plat- discussion, we can single out three architecture pat- form provides a range of services to construct the terns that cover the wider brokerage platform and application through intermediation techniques. application space, and that frame the broker capabil- • The application provides a concrete broker – ities. These can help to put broker applications into possibly targeting a specific vertical sector or a context in terms of their key application direction. specific service type (e.g., for a cloud delivery Here, the description of the architecture patterns does model). The broker application is constructed us- not describe some concrete structural or behavioural ing the platform services, providing features such architecture, but rather the features and objectives as SLA management, a service catalogue, ser- of each cloud service architecture type, see Table 3. vice provisioning, including self-service access, as These patterns are characterised by technical features. well as user authentication and authorisation. However, they should not be considered as broker Several tools specifically target this architecture. We layers. For example, the Cloud Management pattern will review open-source platforms later. The com- may include a billing integration feature, whereas the mercial space also provides advanced solutions that Broker Platform pattern may not. validate the relevance of this architectural setting1. Management brokers could also be called internal brokers as their main purpose is often managing 1. Examples, in no particular order, are Jam- an internal service catalogue. On the other hand, cracker (www.jamcracker.com/solutions), Vordel classical brokers typically mediate between customers (www.axway.com/vordel-products), Gravitant (www.gravitant.com), AppDirect (www.appdirect.com) or and externally provided services. This perspective is ComputeNext (www.computenext.com). complementary to the broker platform capabilities, 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 4

TABLE 1 Broker Capabilities.

Broker Capability Capability Type Deﬁnition Aggregration Composition & delivering combined and centrally managed services to many consumers, not (internal) Management necessarily providing new functionality, integration or customisation. Customisation Adaptation altering or adding capabilities to change or improve, enhance and analyse the service function. Intermediation Composition & intermediation between cloud consumers and providers including advanced (external) Management capabilities (SLA management, invoicing/billing consolidation). Integration Distribution & building independent services and data into a combined offering – often as an Composition integration of a vertical cloud stack or data/process integration within a layer through transformation, mediation and orchestration. Arbitrage Distribution ﬂexible composition of service chosen possibly from multiple providers selected statically or dynamically based on technical as well as cost aspects.

TABLE 2 Broker Features.

Feature Category Features1 Aggregation service orchestration, service catalogue management, provider contracting, resource selection, identity Integration management, user authentication and authorisation, security management, interoperability Customisation Intermediation SLA compliance, self-service enablement, metering, dashboards, monitoring, helpdesk support, business Management transaction monitoring, billing/invocing consolidation, dynamic pricing Arbitrage Dynamic service operations, dynamic routing, capacity management, dynamic orchestration, performance Integration management, life migration, cloud bursting, replication Workload workload classiﬁcation, capacity planning, price prediction, metering, billing and chargeback, perfor- Management mance management, conﬁguration management, Note 1: Extracted from the categorised feature list, Fig. 4, p.9 presented in [40], augmented and validated by feature descriptions used in the selected commercial products listed in Footnote 1.

TABLE 3 Broker Scope and Patterns.

Pattern Scope Definition Architecture Cloud Broker Supports the broker activity types discussed earlier – Broker Platform: The origin of this is the Platform such as aggregation, customisation, integration etc. – common intermediary/broker pattern from which needs a specific language to describe services in a software design patterns, applied to a cloud uniform way and to define the integration mechanism. setting. Cloud Service Supports the design, deployment, provisioning and Broker Application: A management layer Management monitoring of cloud resources, e.g. through management is often identified in cloud architecture that portals. This is an extension of the core lifecycle facilitates efficient and scalable provisioning management (LCM), adding monitoring features or in a number of the platforms reviewed below. graphical forms of interaction. Rudimentary features for Required capabilities include intermediation and the integration of compatible services can be provided. aggregation. Cloud Builds up on intermediary/broker platform to provide a Broker Application: Marketplaces for apps Marketplace marketplace to bring providers and customers together. are omnipresent and this marketplace pattern Again, service description for core and integrated is a reflection of upcoming cloud-specific services plays a role for functionality and technical marketplaces, which will be discussed in more quality aspects. Consolidated pricing and invoicing detail in Section 5. Required capabilities include should here be included. customisation, arbitrage and often aggregation.

which focuses on the broker construction only, but agnostic concerns arising from the definitions above. not the objective embedded in the architecture. The Brokers often target service at a specific layer and, discussion below will show that a fine-grained charac- consequently, have to deal with various cloud layer- terisation of the types of cloud brokerage applications, specific concerns [5]. We identify requirements and even beyond these three, is necessary to identify refer to the earlier broker features such as replication, and distinguish specific challenges. We look at open- migration or orchestration here. source solutions (or solutions provided by publicly Specific concerns for the IaaS layer can be described: funded projects) as these are well-documented. • A key requirement for the IaaS model is elasticity management – see workload management and 2.4 Layer-specific Requirements Analysis dynamic integration capabilities. With techniques We now investigate the possible impact of the dif- such as replication, provisioned services can be ferent cloud layers, IaaS and PaaS/SaaS, on cloud scaled. Images can be replicated and dynamically service broker requirements, based on generic, layer- migrated to other, interoperable offerings and 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 5

platforms to create a virtual layered environment. vices domain in the context of Web Services (W3C, • Problems arise due to platform engines being OASIS etc), cloud services are not necessarily WS- proprietary or not replicating fully, unless stan- compliant. Some solutions exist. IaaS standards, such dards like OVF for VMs are used. Moreover, as OCCI and CIMI, cover service lifecycle manage- replicating an image with data needs bandwidth, ment; TOSCA addresses portability and CDMI data which requires optimised solutions. management. Open-source IaaS supporting these in- • Image and data management aims to 1) minimise clude Openstack, which is a lifecycle management replication and manage deletion, 2) use segmen- product in line with the CIMI and OCCI aims, and the tation for services, 3) differentiate between user- mOSAIC API that supports composition and mashups data and images/services to optimise resources at an infrastructure level [39]. Compliant PaaS sys- and 4) include intelligent data management such tems include Cloudify, a management tool for vertical as map-reduce techniques. Horizontal scaling of- cloud stack integration, and Compatible One, a broker ten requires the full dataset to be replicated. Ver- for horizontal integration [10], [12]. tical scaling can be based on data segmentation and distribution. This indicates that automation is of critical importance to IaaS, as the cloud elasticity need is the driver 2.5 Brokerage - User Perspectives for these techniques. Elasticity requires an automated management of integration tasks, specifically dynamic Brokerage is a mediation process between different integration and workload management, cf. Table 2. parties. It is important to consider different user perspectives, i.e., the consumer and service provider as Specific concerns for the PaaS layer (SaaS is sub- well as the broker itself. Both consumers, providers sumed here as we are considering PaaS-provisioned and also prosumers (that consume and provide at the application services) apart from elasticity are: same time) have their own lifecycles. They differ in • Platforms need to support composition, orches- terms of their responsibility for different features and tration and service mashups to aggregate and also the relevance of these in the context of their activ- customise service offerings [14], [4], [7]. ities. Gartner, for instance, has organised its brokerage • For most applications, base image duplication model along different provider and prosumer types. suffices. However,with many users per application, full replication with customer-specific data • App Developers and Suppliers: The app devel- and code is generally required. Where base im- oper’s work should be supported by abstract- ages (e.g., for .NET) are available, we only need to ing the lower-level work through architecture replicate service instances, and not a full image. and programming features, only exposing the • Further problems arise for composition as QoS is business-rule coding interface. This is seen by the generally not compositional, e.g., the security of progression from API libraries and frameworks, a composition is determined by its weakest link. to DevOps and now PaaS platforms for multi- language, multi-framework development and de- In contrast to IaaS, automated management of aggrega- ployment, to collaborative app composition, and tion, customisation and integration is more of a concern to app lifecycle management [6]. The developer for PaaS. Note that the financial aspects for arbitrage does not need to know about the elasticity of the apply to IaaS, PaaS and SaaS equally. Intermediation IaaS, for example. These will become prosumers and arbitrage refer to activities relevant for all layers. that construct the broker application on top of a A common concern for all layers is standardisation. broker platform. Standardisation in terms of OVF as an image format, or • End Users: Consumers of the app/cloud service OCCI as an interface for infrastructure-level resource just need the interface and use the broker ap- management, are solutions. Interoperability, which plication service. They are arguably the greatest is reflected in the first feature category in Table 2, beneficiaries of cloud brokerage. can be achieved through standardisation based on • Platform Providers: The other main player influ- open and published standards, or on de-facto stan- enced by these technologies is the new player, the dards from widely used open-source or proprietary Broker Platform Provider. Their primary task is systems. However, standards often do not succeed. the platform development addressing both end- Some proposals in the Web services stack (WS-*) are user and developer needs. Although, it is often examples. Problems encountered are 1) diversion of the case that the providers not only develop, but specifications, 2) the slow process of standardisation also host and provision the broker platform. and 3) competing standardisation bodies – the latter is an obvious problem in the cloud domain, where The different user perspectives need to be considered organisations from different areas of IT and com- when weighing and interpreting comparison results. puting are active (SNIA, DMTF, OMG, W3C, OGF Developer features, for instance, are less relevant for etc). While some mature standards exist for the ser- end users, but critical for application developers. 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 6

Generalisation Uses Architecture Architecture Core and Components Capabilities Interoperability Clouds Supported Language and Interoperability Core Programming Service Language Features Target Quality Programming Service Model Broker Application Service Layer Scope Service Layer Broker Engineering Capabilities Elasticity / Broker Platform Cloud Services Scalability Type Broker Platform Broker Generic Dimensions Broker Dimensions QoS / SLA Broker Features Monitoring

Fig. 1. Cloud Broker Architecture – Basic Ontology.

3 A CLASSIFICATION AND COMPARISON Broker Service Target Application FRAMEWORKFOR CLOUD SERVICE BRO- Scope Service Layer KERAGE ARCHITECTURES Broker Capabilities Broker Platform In this section, we introduce a dedicated 3-pronged Broker Dimensions Broker Features framework for broker classification and comparison, Architecture Language and which we will introduce first (see Fig. 1). and Quality Interoperability Programming • Broker application dimensions: categorisation in terms of cloud delivery model, broker construc- Core tion, broker scope Broker Platform Capabilities Generic Dimensions • Generic platform dimensions: a categorisation Core Features schema for a basic cloud platform classification • Broker platform dimensions: broker-specific cate- Type Layer gories plus a detailed, descriptive classification This forms a basic formal ontology, i.e., a taxon- Fig. 2. Cloud Broker Architecture – Structure. omy with defined concepts and instances, that allows the description of broker applications and platforms in terms of three dimensions, each defined through Core artefacts to provide overall conceptual and archi- concepts and either predefined instances or textual tectural views are an ontology capturing key concepts descriptions. The ontology use a mix of categorical (Fig. 1) and a reference broker architecture (Fig. 2). and descriptive elements. The former also combines single and multiple-valued categories. Fig. 2 describes 3.1 Broker Application Dimension - Service Layer the structure of a description from platform to appli- and Scope cation using a hierarchy of generic and broker-specific A cloud service broker application is a cloud-based capabilities and features. The classification is a soft- software system that builds on top of a broker plat- ware architecture specification, involving architecture form (often cloud-based) and provides an application definition and quality assurance concerns – using the service to providers and end-users as consumers. features and capabilities of the platform. Based on our requirements analysis from Section 2, Section 3 is the core contribution section with the two application dimensions can be identified: classification framework. Section 4 describes its ap- • service scope defined through patterns that iden- plication both to illustrate its utilisation and to val- tify the extent of support: Management (oper- idate the quality of the framework. Section 2 plays ations), Broker (mediation), Marketplace (open an equally important role. It analyses the literature collaboration and competition) (Section 2.3). on brokerage and extracts key concerns (through Ta- • target application services: referring to the cloud bles 1, 2 and 3). This is then converted into a 3- delivery models IaaS, PaaS and SaaS (Section 2.4). pronged framework covering application and plat- The platform dimensions will be addressed now in form concerns as identified: broker application di- Sections 3.2 and 3.3. mensions, generic platform dimensions and broker platform dimensions. Specifically the platform dimension requires more description mechanisms, covered 3.2 Generic Platform Dimension - Categorisation- in Tables 4 and 5 on generic and broker-specific based Taxonomy platform concerns. The categories emerge from the We categorise brokerage solutions by the deployment analysis. The origin of the concepts/vocabularies are model, but also specific features or functions that documents as notes to the Tables (e.g., 2, 4 and 5). each of them provides. We have defined a comparison 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 7

TABLE 4 Categorisation Taxonomy – Generic and Broker Platform Properties.

Category Concepts Source Platform - Generic: System Type Multi Cloud API Library, IaaS Fabric Controller, Open PaaS Solution, Open typical cloud product PaaS Provider description categories1 Cloud Stack Layer IaaS, PaaS, SaaS widely accepted cloud delivery models Distribution Open Source (for all solutions considered) typical software product Model2 description categories Core Capabilities Multi-IaaS Support, Multi Language / Multi Framework Support, Multi from categorisation of generic Stack Support cloud terms1 Core Features / Service Description Language, Native Data Store, Native Message Queue, from categorisation of generic Components2 Programming Model, Elasticity & Scalability, QoS/SLA Monitoring cloud terms1 Platform - Broker-speciﬁc: Broker Capabilities Aggregation, Customisation, Intermediation, Integration, Arbitrage – broad from capability discussion in support for broker and marketplace applications Section 2.2 (Table 1) Broker Features Service Management, Discovery/Composition, SLA and User Management, from feature discussion in Monitoring, Metering, etc. – direct support for broker application features Section 2.2 (Table 2) Note 1: categorisation terms (instances) extracted from these sources: [11], [13], [21], [28], [44]. Note 2: Other general cloud platform properties could have been considered. These include auxiliary information for the distribution model, e.g., license model, or properties that describe interoperability aspects, e.g., cloud tools or programming languages supported. We have left out features that are not strictly oriented towards broker construction or that would only add auxiliary descriptive information for core categories (such as languages support if support exists).

framework to categorise cloud solutions along these • categorisation-oriented: addressing the main ob- concerns in Table 4. We chose these concerns to, jective of the service broker construction (service • firstly, broadly categorise the system in terms of prosumption - how services/functions are com- its main function (the system type that indicates bined and passed on): Integration, Aggregation, its target layer and central function in that layer) Customization, Arbitration, Intermediation. This and whether it is proprietary or open-source, has been described in Section 2.2. • secondly, a range of standard properties and in- • descriptive: addressing platform-based broker dividual components are singled out. Properties construction as a software systems development chosen here (Core Capabilities) refer to the neces- based on three facets of software engineering: sary capabilities for brokers to integrate offerings. architecture & interoperability, languages & pro- The two features categories organise a number of gramming, and quality. system components into common and more advanced The second format allows us to drill down and com- ones. For the brokerage types introduced earlier, we pare the support of broker platforms using a more distinguish support dimensions for service descrip- descriptive format. The aim is to judge the support tions and the associated manipulation functions: ver- given by the platform to facilitate broker capabilities tical support for the brokerage types customisation such as aggregation, intermediation or arbitration. and aggregation through e.g., discovery and selection This second, deeper and more descriptive classifica- features for services; horizontal support for the bro- tion schema is based on three facets derived from kerage types integration and aggregation through e.g., generic software engineering concerns – see Table 5. queuing, storage or elasticity management features (in 4 APPLICATION AND EVALUATION federated environments) as applications. There are multiple dimensions for categorizing bro- We applied the categorisation and comparison frame- ker systems. An ontological system allows a product work to a number of cloud brokerage solutions. The to be tagged with multiple attributes, values, and primary aim of this application was to evaluate the relationships. The taxonomy is a simple ontology. The framework regarding two concerns: aim was to create the right structure of categories • Adequacy (fit for purpose): can it identify and attributes for the systems, assigning multiple strengths/limitations (compared to self-procla- categories, attributes, and attribute values. mation) by detecting the presence or absence of In Figs. 6, 7 and 8, we will categorise a number of common features using a unified terminology? solutions [17], [9], [10], [12], [23], [39], [30], [31], [32]. • Completeness: are major features of tools not covered by the framework? Compliance with specific definitions such as NIST 3.3 Broker Platform Dimension - Categorisation is not an objective. Neither is the comparison itself and Descriptive Facet-based Description meant to be comprehensive in terms of the solutions The broker-specific platform characteristics shall be covered, although we apply a rigorous approach to described in two ways: the selection of the solutions. 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 8

TABLE 5 Descriptive Comparison – Broker Platform Properties.

Facet1 Sub-Facets Deﬁnition Source2 Architecture - Architecture The solution architecture is a key element in the deﬁnition of a broker. Of Software and Components practical relevance are the existing, typically lower-layer solutions that the Design Interoperability - Clouds Supported system supports. This is an interoperability concern. / Interoperability Languages - Service Service description and implementation plays a key role for interoperability Software and Language [34]. For selected solutions, we look at the following three aspects: Con- Programming - Programming • service language – the core notation, including the coverage of concerns struction Model vertically (PaaS/IaaS integration) and horizontally (full lifecycle manage- - Service ment) and how this is manipulated (format and API). Engineering • programming model – using the language to program brokerage solutions, linking to SOA principles and other development paradigms. • service engineering – covering wider design and architecture concerns, including monitoring and mashups.

Service - Elasticity / A wider range of non-functional software qualities (e.g., based on ISO9126) Software and Scalability would need to be considered here. These quality concerns also need to be Quality Architecture - QoS / SLA addressed by the service description notation. Scalability and elasticity are Quality Monitoring two specific cloud quality concerns generally used to differentiate capabilities of the cloud in comparison to on-premise architectures. Therefore, we mainly focus on these. Load balancers are typically used to control elasticity based on monitored key performance indicators (KPIs). Multi-tenancy, if available, can alleviate elasticity problems. Wider quality/KPI concerns are equally important. Based on specifications, these are looked after by configuration management tools to set up probes and monitoring tools to collect and analyse data. Note 1: The descriptive elements and their sub-facets should be based on the Vocabulary introduced in the Generic Capabilities and Generic Features in Tables 1 and 2. Note 2: Based on the IEEE Software Engineering Body of Knowledge SWEBOK [43] and the Joint ACM/IEEE Curriculum Guidelines for Graduate Degree Programs in Software Engineering [1].

4.1 Selection of Cloud Solutions ated review protocols to extract, analyse and docu- ment results. We adopted a three-step review process We selected only open-source solutions (to determine that includes planning (based on Section 3), con- the state-of-the-art) and research-oriented projects (to ducting (Section 4.1) and documenting (Section 4.2, detect maturing trends). Commercial solutions have 4.3). We use an explicit characterization framework been considered in the analysis to determine the of the reviewed items. This is the foundation for a framework in Section 2.2, but have been excluded here comparative analysis based on our analysis dimen- to reduce threats arising from more biased and in- sions. Cloud and open-source were search terms complete available information. Sample open-source included, but IaaS and PaaS were distinguished: solutions (OSS) can thus be categorised: • Terms "cloud paas open-source" resulted in • IaaS: OpenStack, for instance, is a basic IaaS cloud OpenShift, CloudFoundry, Cloudify, manager that transforms data-centres to become Stratos, Pivotal, Tsuru, Paasmaker, IaaS clouds [32]. CloudBees, Cocaine • PaaS: OpenShift and CloudFoundry are open PaaS platforms assisting the cloud app developer – from which we selected the first three. by commoditising the software stack [9], [31]. • Terms "cloud iaas open-source" resulted in The Open IaaS/PaaS solutions can be differentiated Openstack, OpenNebula, jcloud, mOSAIC from respective IaaS/PaaS brokers. In the following, – from which we selected the first three again. we will try to point out the salient differences between Searching for "cloud paas open-source service some cloud brokers that go beyond IaaS/PaaS man- broker management" aimed at covering dedicated agement solutions. For instance, Optimis and Com- brokerage solutions, but resulted only in patibleOne are IaaS-oriented, and only 4CaaSt targets CompatibleOne as a new, not yet covered solution. PaaS and to some extent also the SaaS domain. There To select relevant research projects, we matched is, however, SaaS broker activity in the commercial "cloud project service broker space – as reviewed in Section 2.2. management" Solutions were selected based on key-word searches using the Google search engine following standard against project summary information provided by the selection mechanisms used in systematic literature EU (full documentation available), which resulted in reviews (SLRs) [25], [22], applied to cloud solutions. A 4CaaSt∗, Broker@Cloud, Cloud4SOA, SLR reduces bias and follows a sequence of method- Modaclouds, mOSAIC∗, PaaSage, Ocean, ological steps. SLRs rely on well-defined and evalu- Optimis∗, Reservoir∗, 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 9

from which we selected those that were finished at VM management, i.e., it provides dynamic architec- the time of this research (September 2014) – marked ture management and interoperability. For instance, with an asterisk ∗. We focussed on the EU context Mosaic assumes a Linux OS, which runs Mosaic App as the reporting and documentation mechanism is Components (called CloudLets). A number of com- consistent for the selected projects. In [19], a more mon commercial cloud solutions are supported by global review of research projects is provided – which Mosaic, including Amazon and Rackspace products. has overlapping EU projects and similar results. Languages and Programming. Solutions are com- An observation is that the broker pattern receives pared in Table 10, e.g., Cloudify uses application attention and that reusable solutions are in develop- recipes and resource node templates (Groovy scripts) ment, starting with the IaaS layer, but including PaaS as the programming model. A service recipe contains and SaaS over time. The existence of marketplaces, LCM scripts, monitoring probes and IaaS resourcesre- which are interesting for the diverse SaaS space, indi- quirements. Mosaic uses an ontology as the notation cates the existence of broker solutions. A wide range and a component-based application programming of commoditised broker platforms can be expected in model for portability of apps across compliant clouds. the future to service the different broker types defined, Patterns emerge as solutions to compose, connect and but also provide a fuller range of features. manage clouds in distributed contexts. Quality. Table 11 covers quality. Common components such as monitoring, load balancers and scalabil- 4.2 Categorisation - Platform Dimension ity engines are the key capabilities required here. In Tables 6 to 8, we categorise the solutions selected in Section 4.1 [9], [10], [23], [30], [39], [31], [32], [17], [12] 4.4 Evaluation of the Framework according to the defined classification taxonomy. As we review broker platforms, only the two platform di- We have categorised broker platforms based on the mensions are covered – although it needs to be noted platform taxonomy aspects. Specifically, looking at the that some like CompatibleOne or 4CaaSt have ap- perspective of the user types is now valuable. For plication features as well. Compatible One provides, instance, developers are supported in three categories: for instance, platform features like SLA management a) API library: jcloud, b) Devops: Cloudify and c) Full (SLAM) and intermediation and integration features PaaS: CloudFoundry, OpenShift. A trend goes from (PROCCI), and also application functions for service provider-oriented solutions to developer-oriented so- brokering (BROKER). Note that in Table 8, instead of a lutions to end user-oriented cloud management [3] detailed feature list, we only summarise this as ’Y’ if a – 4Caast being an example of the latter. A deeper sufficient number of features is covered per category2. discussion of trends and challenges emerging from this discussion shall follow in the next section. We now turn to the evaluation of the framework itself in 4.3 Descriptive Facet-based Comparison terms of adequacy and completeness. In the following, we review solutions with respect Adequacy. The framework is adequate by provid- to three facets defined in Table 5: architecture & ing a sufficiently complete framework. interoperability, languages & programming, and qual- • Broker Definition. This was achieved by using ity. We do not consider all selected solutions in the the three most cited brokerage definitions as the comparison, but only select the most advanced ones starting point. We define five types that subsume for each aspect as the objective here is framework each of the three literature sources. evaluation only. For this, a smaller number of feature- • Capabilities and Features. We identified fine- rich, advanced cases are sufficient. A comprehensive granular capabilities and features from the liter- state-of-the-art review is not aimed at here. A detailed ature. Additionally, we used common categorisa- presentation of the comparison results is presented in tions from the cloud computing community (e.g., Tables 9 to 11. Here, we single out key observations. delivery models) and software engineering (spec- Architecture and Interoperability. In Table 9, a ification and design) to add further dimensions. number of PaaS-level solutions are summarised in While only 10 solutions have been used here for terms of these two aspects. Common are the utili- validation, initially 13 open-source and 6 commercial sation of configuration management solutions, such products were considered and documented in [16]. as Chef or Git. The deployment is managed through Completeness. The completeness has been empir- consoles or APIs, mapping PaaS-level requests down ically validated through an application for a state- to IaaS operations. As many IaaS solutions are of- of-the-art comparison. For this, the cases have been ten supported, interoperability is a critical concern. systematically selected to reflect a broad range of CompatibleOne is OCCI-compatible in its support for advanced systems. An investigation of the concrete descriptions has not resulted in any relevant concepts 2. A figure of 35% of features covered was considered as sufficient. The figure was derived from the percentage of core features lacking in the framework. Some table parts (Tables 7 that are on average covered by the selected tools. and 8) are sparsely populated, but still several tools 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 10

TABLE 6 Broker Platform - Generic Category and Type.

LAYER and MAIN TYPES Name Type Cloud Layer Multi Cloud IaaS Fabric Open PaaS Open PaaS API Library Controller Solution Provider OpenNebula Cloud Fabric Controller IaaS Y OpenStack Cloud Fabric Controller IaaS Y jclouds API Library PaaS Y Cloudify Cloud Devops & LCM PaaS Y Y CloudFoundry PaaS PaaS Y Y OpenShift PaaS PaaS Y CompatibleOne IaaS Broker PaaS Y Mosaic PaaS PaaS Y Y 4Caast Service Broker PaaS Optimis IaaS Broker PaaS Y

TABLE 7 Broker Platform - Generic Capabilities and Features/Components.

CORE CAPABILITES CORE FEATURES Name Multi IaaS Multi Multi Service Native Native Program- Elasticity QoS / Support Language Stack De- Data Store Message ming Scalability SLA Mon- / Multi scription Queue Model itoring Frame- Language work OpenNebula OpenStack Y Y Y Y jclouds Y Cloudify Y Y Y Y Y Y Y CloudFoundry Y Y Y Y Y Y OpenShift Y Y Y Y Y CompatibleOne Y Y Y Y Y Mosaic Y Y Y Y Y Y 4Caast Y Y Y Y Optimis Y Y Y Y Y

TABLE 8 Broker Platform - Broker-speciﬁc Capabilities and Features.

BROKER CAPABILITES BROKER FEATURES Name Aggrega- Intermedi- Integra- Arbitrage Customi- Aggreg/ Intermed/ Dynamic Workload tion ation tion sation Integr/ Mgnt/ Integra- Manage- Custom Arbitrage tion ment OpenNebula OpenStack jclouds Cloudify Y Y Y CloudFoundry Y Y Y OpenShift CompatibleOne Y Y Y Y Y Y Y Mosaic Y Y Y 4Caast Y Y Y Y Y Y Optimis Y Y Y Y

per feature can be identified, which means that the brokerage, where independent actors in the ecosystem respective feature is present in state-of-the-art systems integrate, aggregate/compose and customise/adapt and, thus, the respective feature category is valid. existing services [18], [28]. End-to-end personalisation As a result, the solution is adequately fit for pur- becomes achievable. Prosumers may create mashups pose. It allows a neutral classification along dimen- from existing services. sions originating from different fields. It is therefore From the above comparison between various cloud consistent with a common understanding of key con- solutions, we can note a difference between the needs cepts. It also serves as an analysis tool. From the of cloud brokerage and cloud marketplaces. We did results in Tables 9 to 11, e.g., the ongoing development already introduce them as different patterns above. of scripting languages emerges as the focus in the pro- Service management is already a common solution, gramming context, or work on scalability engines as often provided as a domain-independent platform. a quality concern, emerges as indicators of activities. • A brokerage solution needs to automate the process of matching service requirements with re- 4.5 Discussion and Challenges source capacity and capabilities [41]. The ideal The need for management support and interoperabil- would be a total commoditisation of IaaS so that ity becomes apparent in the context of cloud service any compute resource could be plugged into a 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 11

user’s compute capacity. Therefore, interoperabil- requirements, for marketplaces in particular, are: ity is of importance. There are areas of com- • data integration and security enforcement as non- patibility that should be considered in match- functional requirements, ing that are not handled by brokers currently. • social network functions allowing service ratings For example, none of the solutions that were by the communities, assessed considered data integrity as a matching • SLA management to be integrated. criterion; however, they all included performance Commoditisation can be facilitated by an operational in their criteria. Security is another aspect that development and deployment model to act as an has a technical nature, but is also abstract insofar enabler. This ties in with another observation. A as it can be implemented by a cloud provider. proliferation of cloud capacity clearing-houses, which Data integrity and security policy enforcement, operate similar to a spot market to allow clouds to if considered as criteria when evaluating cloud buy and sell spare cloud capacity on a very short-term interoperability, may need to be formalised us- basis, has only started. It is less clear what is needed ing a standardised language to describe common to facilitate this from a construction perspective. aspects, similar to the languages that have been Federation is the second trend for brokerage solu- created to model other cloud entities. tions [8], i.e., to work across independently managed • A marketplace needs to additionally focus on the and provided cloud solutions of an often hetero- possibly distributed architecture of the applica- geneous nature. Some challenges and requirements tions as well as the cloud. The appstore model arising from the architecture and interoperability dis- appears to be the de-facto model of choice for the cussion can be identified: marketplace, but this seems more an admission of • Reference architectures – cf. NIST cloud broker- the success of the Apple initiative rather than re- age reference architecture [28]. search. There is a potential to explore other forms • Scope of control – the management of configura- of the online marketplace suitable to cloud apps tion and deployment based on integrated and/or and their composition [27], [14]. This can also standards-based techniques [26]. be extended to an even more commodity-based • Federation and syndication – as two forms of scenario where all services could be registered on distributed cloud architectures [38]. a wide-area multi-marketplace scale. Table 7 shows that broker-specific core and advanced features are not comprehensively supported in avail- 5 CONCLUSIONS able tools. 4Caast and Optimis cover these, but are as We have introduced the main concepts of service research projects not available as products. A wider brokerage for clouds, using some concrete systems discussion, arising from the concrete comparisons and platforms to identify current trends and chal- above, shall now follow in terms of commoditisation lenges and compare current, primarily open-source The commoditisation of cloud services is an emerging solutions. Brokerage relies on interoperability, quality- need arising from the discussion – specifically from of-service and other architectural principles. Brokers the language and programming facet. A trend is to and marketplaces can play a central role for new move from the lower IaaS layer to PaaS and onwards adopters migrating to the cloud or between providers to encompass SaaS, aiming to integrate lower layers [22], [35]. Brokers will act as first points of call. – 4CaaSt is an example. To make this work, services Our classification and comparison framework identi- at all layers need to be available for a uniform way fies three dimensions – two platform and one ap- of processing in terms of selection, adaptation, inte- plication oriented. Taxonomy-based categorisation helps gration and aggregation. Commoditisation is the con- to characterise solutions in terms of type, common cept to capture this need. Some concrete observations components and features. The second mechanism is related to the reviewed open-source solutions are: a more descriptive, layered taxonomy starting with (i) fully functional image and vertical stack building architecture and interoperability, languages and pro- capabilities (CompatibleOne leadership), (ii) program- gramming, and quality as facets. Our objectives have ming and operational support of service composition been to clarify the conceptualisation of cloud service (4CaaSt leadership) and (iii) graphical manipulation brokerage by taking more architectural concerns into of service abstractions (Optimis leadership). Com- account, but also to provide a framework for industry moditisation can be supported through a uniform and academia to classify and compare solutions. representation based on description templates. The An observation of our comparison based on the programming model with its supporting program- framework is the emergence of cloud broker solutions ming/scripting languages is crucial here. These need on top of cloud management. A further separation of to cover the architecture stack and meet language and marketplaces, often in the form of appstores, is nec- quality concerns discussed in Section 4. essary. A number of activities work in this direction. Commoditisation is an enabler of functions on top Compatible One is an example showing how OCCI is of a broker platform. Thus, additional challenges and used as an infrastructure foundation and built upon 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 12

to provider PaaS-level brokerage. 4CaaSt in a similar [8] R. Buyya, R. Ranjan, R.N. Calheiros. Intercloud: Utility- vein aims to integrate the layers and move toward Oriented Federation of Cloud Computing Environments For Scaling of Application Services. Intl Conf on Algorithms and a marketplace solution. Commercial broker applica- Architectures for Parallel Processing , LNCS 6081. 2010. tions, that have already supported the framework [9] Cloud Foundry. Open Source PaaS Cloud Provider Interface. construction here in Section 2.2, show already exist- http://www.cloudfoundry.org/. 2015. [10] Cloudify. Cloudify Open PaaS Stack. ingbrokerage and marketplace solutions ranging from http://www.cloudifysource.org/. 2015. images to software services, essentially commoditis- [11] Cloud Standards. http://cloud-standards.org/. 2015. ing the respective cloud resources. [12] CompatibleOne. Open Source Cloud Broker. Service description mechanisms discussed in [29], http://www.compatibleone.org/. 2015. [13] ETSI Cloud Standards. http://www.etsi.org/news- [37], [33] (as manifests, recipes and blueprints), but events/news/734-2013-12-press-release-report-on-cloud- also in standards and languages like TOSCA and computing-standards. 2015. CloudML, can serve to abstract, manipulate and com- [14] C. Fehling, R. Mietzner. Composite as a Service: Cloud Appli- cation Structures, Provisioning, and Management. Information pose cloud service offerings in an effort to commodi- Technology 53:4, pp. 188-194. 2011. tise the cloud. These description mechanisms, based [15] Forrester Research. Cloud Bro- on an abstract model serve two purposes: Firstly, kers Will Reshape The Cloud. 2012. http://www.cordys.com/ufc/file2/cordyscms sites/download/ to abstractly capture, present and manipulate cloud 09b57cd3eb6474f1fda 1cfd62ddf094d/pu/ resources. Secondly, to serve as a starting point to link [16] F. Fowley, C. Pahl, L. Zhang. A Comparison Framework and to configuration and other deployment concerns in Review of Service Brokerage Solutions for Cloud Architectures. 1st International Workshop on Cloud Service Brokerage. 2013 federated clouds. Thus, commoditisation and federation [17] S. Garcia-Gomez et al. Challenges for the comprehensive emerge as challenges from our discussion. management of Cloud Services in a PaaS framework. Scalable Future work includes trust as an equally important Computing: Practice and Experience 13(3). 2012. [18] Gartner - Cloud Services Brokerage. Gartner Research, concern that is more difficult to facilitate technically 2013. http://www.gartner.com/it-glossary/cloud-services- than commoditisation. A mechanism is needed for not brokerage-csb only vetting individual providers, but also to allow [19] N. Grozev, R. Buyya. InterCloud architectures and application brokering: taxonomy and survey. Software: Practice and Expe- this to happen in layered, federated and brokered rience. 2012. cloud solutions. Furthermore, end-users, intermedi- [20] C.N. Hofer,¨ G. Karagiannis. Cloud computing services: taxon- aries such as brokers and providers have different life- omy and comparison. Journal of Internet Services and Appli- cations, 2(2), 81-94. 2011. cycles that need to be integrated. Forrester Research [21] IEEE Cloud Standards. http://cloudcomputing.ieee.org/standards. [15] has already provided basic ideas, but a deeper 2015. investigation into lifecycle and workflow integration [22] P. Jamshidi, A. Ahmad, C. Pahl. Cloud Migration Research: A is necessary. And finally, cloud data brokers emerge Systematic Review. IEEE Transactions Cloud Computing. 2013. [23] Jclouds. jclouds Java and Clojure Cloud API. as a different type of brokerage. We aim to look at an http://www.jclouds.org/. 2015. adaptation of our framework. [24] A. Juan Ferrer et al. OPTIMIS: A holistic approach to cloud service provisioning. Future Generation Computer Systems, 28(1):66-77. 2012. ACKNOWLEDGMENTS [25] B. Kitchenham and S. Charters. Guideline for Performing Systematic Literature Reviews in Software engineering. Keele This research has been supported by the Irish Centre for University and University of Durham, 2007. Cloud Computing and Commerce and the Royal Irish [26] A.V. Konstantinou, T. Eilam, M. Kalantar, A.A. Totok, W. Academy/Royal Society Intl Cost Share Grant IE131105. Arnold, E. Sniblel. An Architecture for Virtual Solution Compo- sition and Deployment in Infrastructure Clouds. Intl Workshop on Virtualization Technologies in Distr Computing. 2009. [27] R. Mietzner, F. Leymann, M. Papazoglou. Defining Composite REFERENCES Configurable SaaS Application Packages Using SCA, Variability [1] ACM/IEEE. Area Definitions – Joint ACM/IEEE Curriculum Descriptors and Multi-tenancy Patterns. Intl Conf on Internet Guidelines for Graduate Degree Programs in Software Engi- and Web Applications and Services. 2008. neering GSwE2009. http://www.gswe2009.org. 2009. [28] NIST. Cloud Computing Reference Architecture. [2] R. Barrett, L. M. Patcas, C. Pahl, and J. Murphy. Model Driven http://dl.acm.org/citation.cfm?id=2385915. 2012. Distribution Pattern Design for Dynamic Web Service Compo- [29] D.K. Nguyen, F. Lelli, Y. Taher, M. Parkin, M.P. Papazoglou, sitions. International Conference on Web Engineering ICWE’06. W.-J. van den Heuvel. Blueprint Template Support for Cloud- Pages 129-136. Palo Alto, US. ACM Press. 2006. Based Service Engineering. Proceedings ServiceWave11, Poz- [3] T. Benson, A. Akella, S. Sahu, A. Shaikh. Peeking into the nan, Poland, October 2011. Cloud: Toward User-Driven Cloud Management. CloudS 2010 [30] OpenNebula. OpenNebula - Open Source Data Center Virtu- Conference, Sydney, Australia. 2010. alization. http://opennebula.org/. 2015. [4] D. Benslimane, S. Dustdar, A. Sheth. Services Mashups: The [31] OpenShift. Cloud computing platform. New Generation of Web Applications. Internet Computing, https://openshift.redhat.com/. 2015. vol.12, no.5, pp.13-15, 2008. [32] OpenStack. OpenStack Open Source Cloud Computing Soft- [5] D. Bernstein, E. Ludvigson, K.Sankar, S. Diamond, M. Morrow ware. http://www.openstack.org/. 2015. Blueprint for the Inter-cloud: Protocols and Formats for Cloud [33] C. Pahl. Layered Ontological Modelling for Web Service- Computing Interoperability. Intl Conf Internet and Web Appl oriented Model-Driven Architecture. European Conf on Model- and Services. 2009. Driven Architecture ECMDA2005. 2005. [6] Brunnert et al., Performance-oriented DevOps: A Research [34] C. Pahl, S. Giesecke and W. Hasselbring. Ontology-based Agenda. arXiv preprint arXiv:1508.04752, 2015. Modelling of Architectural Styles. Information and Software [7] R. Buyya. Compatibility-Aware Cloud Service Composition Technology (IST). 51(12): 1739-1749. 2009. under Fuzzy Preferences of Users. IEEE Transactions on Cloud [35] C. Pahl, H. Xiong. Migration to PaaS Clouds - Migration Computing, 2(1):1-13. 2014 Process and Architectural Concerns. IEEE 7th International 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 13

TABLE 9 Architecture and Interoperability.

Cloudify CloudFoundry OpenShift CompatibleOne 4Caast - Console for plat- - Console pushes - Divided into - ACCORDS - Exec Container Architecture / form commands. app to cloud; control plane exposes features REC runs Components Web management deployment (Broker) and through REST API. instances. console for management msg / hosting - Parser validates - Deployment monitoring. / configuration infrastructure Manifest against Manager maps - Service Manager through console. (nodes). CORDS schema deployment model uses scripting - Controller - Controller is and maps (service template, (recipe) to cater for runs as a VM command CLI elements to valid QoS constraints) middleware stack on the target shell, used to OCCI categories to OVF. Service - Cloud Controller IaaS; controls all create apps. which are then Manager deploys is REST endpoint Cloudfoundry GIT for app instantiated. images using to manage app (CF) spawned management / - Publisher Claudia. deployment & cloud VMs. Does deployment. provides which - REC includes an control; injects not manage - Gear is appli- endpoint serves agent (application agent on VM to any IaaS layer cation container which categories. LCM, control) install & orchestrate functions. and a virtual Parser runs and and a server app deploy / - IaaS provider server/node produces a plan of (storage, config monitor/ scale must support accessed via ssh. OCCI instances for data). Deployment - Cloud Driver: CF. Apps created Cartridge service resolution (instance Server (Chef) talks VM templates using CF are runs on a Gear. can receive/send to Service & REC for different deployed to CF App LCM scripts data). Manager. OVF IaaS clouds in VMs controlled by allow for post- - Broker processes Manager creates configuration. a Cloud Controller deployment action plan and invokes extended OVFs Triggers host on CF-compliant hooks to run on instances. from resolved provisioning IaaS clouds. VMs. BluePrints. Supports Azure, Supports AWS, Uses DeltaCloud; OCCI provider FlexiScale driver Clouds OpenStack, Openstack, app runs on interfaces (PROC- provided. Supported CloudStack, EC2, Rackspace, RedHat certified CIs) for OpenStack, OpenNebula / Interoper- Rackspace, Terra- vCloud, vSphere. public cloud OpenNebula and supported. ability mark (buildable for Hosted as public (needs deltacloud Azure (also SlapOS Generic IaaS any of the jclouds PaaS. support). and SlapGrid). Cloud API above) through Tcloud.

Symposium on the Maintenance and Evolution of Service- Claus Pahl. is an Associate Professor at the Oriented and Cloud-Based Systems MESOCA 2013. 2013. Free University of Bozen-Bolzano, Italy. Prior, [36] C. Pahl, H. Xiong, R. Walshe. A Comparison of On-premise to he was a principal investigator of the Irish Cloud Migration Approaches. Europ Conf on Service-Oriented Centre for Cloud Computing and Commerce and Cloud Computing ESOCC. 2013. IC4. His research interests include software [37] M.P. Papazoglou, W.J. van den Heuvel. Blueprinting the engineering in service and cloud computing. Cloud. IEEE Internet Computing, November 2011. He holds a Ph.D. in computing from the Uni- [38] A. Paya, D. Marinescu. Clustering algorithms for scale-free versity of Dortmund and an M.Sc from the networks and applications to cloud resource management. 2013. University of Technology in Braunschweig. [39] D. Petcu et al. Portable cloud applications - from theory to practice. Fut. Gen. Computer Systems 29(6):1417-1430. 2013. [40] S. Ried. Cloud Broker A New Business Model Paradigm. Forrester. 2011. [41] L. Rodero-Merino, L.M. Vaquero, V. Gil, F. Galan, J. Fontan, R.S. Montero, I.M. Llorente. From Infrastructure Delivery to Service Management in Clouds. Future Generation Computer Systems, vol. 26, pp. 226-240. 2010. [42] L. Sun, H. Dong, and J. Ashraf. Survey of Service Description Languages and Their Issues in Cloud Computing . Eighth International Conference on Semantics, Knowledge and Grids (SKG) 2012. pp. 128-135. IEEE. 2012. [43] IEEE. Software Engineering Body of Knowledge SWEBOK. Pooyan Jamshidi i is a research associate http://www.computer.org/portal/web/swebok. 2004. at Imperial College London. He holds a [44] Wikipedia - Cloud API,Wikipedia. 2015. PhD from Dublin City University. He received the BS and MS degrees in computing from Amirkabir University of Technology. His gen- Frank Fowley is a Senior Research Engi- eral research interests are in software engi- neer at the Irish Centre for Cloud Computing neering and his focus lies predominantly in and Commerce IC4. His research interests the areas of self-adaptive software, machine include cloud computing and security. Frank learning and big data. holds an M.Sc. from Dublin City University in Forensics and Secure Computing.

2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2016.2537333, IEEE Transactions on Cloud Computing

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. X, NO. X, JANUARY 2016 14

TABLE 10 Service Language, Programming Model and Service Engineering.

Reservoir CompatibleOne 4Caast Optimis Service Definition Units of Service Resources and Services Service Manifest Service Manifest for metadata; Manifest: Image & are described in a includes sections Language software stack (OS, Infrastructure. Image: Blueprint BP, which is per component per middleware, app, System (base OS) & an abstract description VM. Service Register config, data) in Package (stack config); of what needs to has sections for SP virtual image; service Infrastructure: Storage, be resolved into requirements and descriptions for Compute & Network. infrastructure entities. IP capabilities, VM contracts between Image is description BPs are stored and abstract description, service provider SP of manual app build. managed in a BP TREC (trust, risk, and infrastructure Image has agent that repository via a REST eco-efficiency, cost), provider IP. is embedded in VM API. A BP is resolved elasticity, data Manifests (OVF) relate & runs on startup. when all requirements protection. Has abstract entities and Agent is script to run are fulfilled by another provider description LCM of services. required configuration, BP, via the Resolution schema for a SP to Feedback between set up monitoring Engine (is service provide its capabilities SP and IP allows IP to probes, or download orchestration feature). in an XML Optimis- scale and monitor. components. compliant format. Elasticity is defined PaaS4Dev: Java EE Uses Active MQ, Java schemas, jaxb, Program- using ECA rules to services (EE5/6 web postgresql, jonas, xmlbeans, REST, ming Model scale infrastructure profile) & Enterprise ow2orchestra, apache monitor; also jax-ws, dynamically based OSGi services (http, serv bus. Ontology- cxf, javagat. IDE is on application KPI jndi, transaction) for based BP schema using Eclipse with plugin for metrics. Rules in OCL. development Jena, SPARQL. Optimis core classes. Service provisioning - Nested manifests - Request Language Toolkit provides Service described in De- support service BRL & request patterns image mgmt, context Engineering ployment Descriptor. composition. create Blueprint BP manager injects Service configuration - COSACS module service specification - context information automation based on embeds in VM image mapped to operations to VMs and Elasticity Xen configuration. mechanisms to manage and mgmt API Engine to add/remove Service Elasticity is lifecycle, e.g., post- calls. Mashup for resources. Service achieved through creation monitoring composition. Deployment Optimiser mapping Manifest setup and appliance - BP consists of BP optimises placement of KPIs with run-time config, in conjunction images, contains services. Configuration metrics gathered by with image production functional, KPI & using the Toolkit IDE. app monitoring agents. module. policy parameters.

TABLE 11 Quality: Scalability/Elasticity and SLAs.

CloudFoundry OpenShift CompatibleOne 4Caast Optimis Can add/remove Automatic gears Elasticity is Not in current The toolkit Elasticity / instances for add/remove as provided by release. includes an Scalability scalability and load changes. the load balancer Elasticity Engine change CPU & Multi-tenancy module for the to add / remove memory limits on using multi-gears IaaS resources. resources. VMs on VMs There is only a The application via COMONS Monitoring based Framework uses QoS / SLA basic logging scaling, when Monitoring on probe injection REST to get Monitoring facility with automatic, module. on PICs via CPU/disk usage Cloud foundry is based on REST. Modified from monitoring but there are concurrent ap- JASMINe provides which resides on many third-party plication request dynamic probe nodes and run Cloud Foundry thresholds. deployment & as scripts to feed monitoring The resources config. Chef recipe data to monitor plug-ins can be consumed by an configs VM probes store. SLA used to provide application can for REC manager. Manager built application be monitored and Monitoring based using WSAG4J monitoring, such viewed from the on collectd stats based on OGF as Hyperic. Console. for forecasting. WS-Agreement.

Daren Fang is a Research Associate at Ed- Xiaodong Liu is a Reader and the Director inburgh Napier University, UK. His research of Centre for Information & Software Sys- interests include cloud service modelling, tems at Edinburgh Napier University. His re- green service optimization, service adapta- search interests include context-aware adap- tion, and service evolution. tive services, cloud service evolution, perva- sive computing. He holds a PhD in software engineering from De Montfort University, a MSc and BEng from Renmin University and Xi’an Jiaotong University.