DOCSLIB.ORG

Reusable Components, Techniques and Standards for City Platform As a Service (Cpaas)

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Reusable Components, Techniques and Standards for City Platform As a Service (Cpaas) FP7-ICT-2013-EU-Japan ClouT: Cloud of Things for empowering the citizen clout in smart cities FP7 contract number: 608641 NICT management number: 167 ア Project deliverable D3.1 - Reusable components, techniques and standards for City Platform as a Service (CPaaS) ABSTRACT The objective of this document is to describe the reusable components in the City Platform-as-a- Service (CPaaS) layer. The services, and the API’s exposed by CPaaS infrastructure, depend on the requirements and on the architecture defined in WP1. WP3 also describes the non-functional requirements, such as the security requirements. This WP is focused on city information infrastructure, cloud storage components, with the dependents interfaces, security layer and all the infrastructure necessary to access and manage data. WP3 is focused also on the access control and fault-tolerance methods for access and manage data. D3.1 - Reusable components and techniques for CPaaS Disclaimer This document has been produced in the context of the ClouT Project which is jointly funded by the European Commission (grant agreement n° 608641) and NICT from Japan (management number 167ア). All information provided in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. This document contains material, which is the copyright of certain ClouT partners, and may not be reproduced or copied without permission. All ClouT consortium partners have agreed to the full publication of this document. The commercial use of any information contained in this document may require a license from the owner of that information. For the avoidance of all doubts, the European Commission and NICT have no liability in respect of this document, which is merely representing the view of the project consortium. This document is subject to change without notice. The ClouT consortium is composed of the following institutions No. Participant organisation name Short name Country Commissariat à l’énergie atomique et aux énergies CEA 1 France alternatives (coordinator) 2 Engineering Ingegneria Informatica SpA ENG Italy 3 University of Cantabria UC Spain 4 STMicroelectronics S.r.l. ST Italy 5 Santander City Municipality SAN Spain 6 Genova Municipality GEN Italy 7 Nippon telegraph and telephone East Corporation NTTE Japan (NTT East) (coordinator) 8 Nippon Telegraph and telephone corporation (NTT NTTRD Japan R&D) 9 Keio University KEIO Japan 10 Panasonic System Solution PANA Japan 11 National Institute of Informatics NII Japan ClouT – 31.07.2013 Page 2 D3.1 - Reusable components and techniques for CPaaS EU Editor Cosimo Greco, ENG JP Editor Kenji Tei, NII Authors [Cosimo Greco, ENG], [Levent Gurgen, CEA], [Yazid Benazzouz, CEA], [Stefania Manca, GEN], [Takuro Yonezawa, Keio], [Kenji Tei, NII], [Fuyuki Ishikawa, NII], [Jose Antonio Galache, UC], [Fernando Mons Nunez, SAN] Internal reviewer Marco GRELLA, ST Deliverable type R Dissemination level PU (Confidentiality) Contractual Delivery Date 31/07/2013 Actual Delivery Date 31/07/2013 Keywords ClouT, Cloud Computing, IoT, Smart Cities, CPaaS, Reusable Components Revision history Revision Date Description Author (Organisation) V0.1 10.06.2013 Table of Contents ENG created V0.2 08.07.2013 First draft with ENG contributions from all partners V0.3 12.07.2013 Second draft with ENG contributions from all partners V0.4 19.07.2013 Added Santander ENG contributions. V0.5 19.07.2013 First consolidated ENG version sent for internal review V0.6 26.07.2013 Reviewed version ST V1.0 31.07.2013 Final version ENG ClouT – 31.07.2013 Page 3 D3.1 - Reusable components and techniques for CPaaS TABLE OF CONTENTS TABLE OF CONTENTS ........................................................................................................................................ 4 LIST OF FIGURES ............................................................................................................................................... 5 LIST OF TABLES ................................................................................................................................................. 7 LIST OF ABBREVIATIONS AND DEFINITIONS ..................................................................................................... 8 EXECUTIVE SUMMARY ..................................................................................................................................... 9 1. INTRODUCTION .................................................................................................................................... 10 1.1. SCOPE OF THE DOCUMENT ....................................................................................................................... 10 1.2. TARGET AUDIENCE .................................................................................................................................. 10 1.3. STRUCTURE OF THE DOCUMENT ................................................................................................................ 10 2. SERVICE COMPOSITION PLATFORMS FOR CITIZEN’S APPLICATIONS ................................................. 12 2.1. INTRODUCTION ....................................................................................................................................... 12 2.2. SERVICE COMPOSITION AND MASH-UP TOOLS REUSABLE COMPONENTS .......................................................... 13 WIRECLOUD ................................................................................................................................................ 13 Mycocktail .................................................................................................................................................... 14 Enterprise Mashup Markup Language ....................................................................................................... 15 OpenSocial ................................................................................................................................................... 16 Yahoo! Pipes ................................................................................................................................................. 17 Apache Shindig ............................................................................................................................................ 18 iPojo ............................................................................................................................................................. 20 BPMN ........................................................................................................................................................... 22 BPEL ............................................................................................................................................................. 24 2.3. DEPENDABLE SERVICE COMPOSITIONS REUSABLE COMPONENTS ................................................................... 25 Robust Service Composition METHOD ........................................................................................................ 25 QoS-based Service/CLOUD Selection Method ............................................................................................. 26 Metadata-based Behavior Insertion FRAMEWORK ................................................................................... 27 Verification Framework of Time and Resource Constraints on Business Process ..................................... 28 Verification Framework of ECA Specification on Physical Interactions .................................................... 29 3. BIG DATA PROCESSING ........................................................................................................................ 30 3.1. INTRODUCTION ....................................................................................................................................... 30 3.2. DATA/EVENT PROCESSING AND DECISION MAKING REUSABLE COMPONENTS ................................................... 31 Esper ............................................................................................................................................................ 31 FI-WARE Gateway Data handling ............................................................................................................... 33 Jboss Drools Expert ...................................................................................................................................... 35 MongoDB ..................................................................................................................................................... 36 3.3. SELF-HEALING FOR DATA/EVENT STREAMING REUSABLE COMPONENTS .......................................................... 37 Self-healing Framework for Sensory Data .................................................................................................. 37 Fault Classification Model ........................................................................................................................... 39 4. SECURE AND DEPENDABLE ACCESS TO CITY DATA ............................................................................. 40 4.1. INTRODUCTION ....................................................................................................................................... 40 4.2. OPEN CITY DATA HOSTING AND ACCESS REUSABLE COMPONENTS .................................................................
Load more

Recommended publications
  • The Google File System (GFS)
    The Google File System (GFS), as described by Ghemwat, Gobioff, and Leung in 2003, provided the architecture for scalable, fault-tolerant data management within the context of Google. These architectural choices, however, resulted in sub-optimal performance in regard to data trustworthiness (security) and simplicity. Additionally, the application- specific nature of GFS limits the general scalability of the system outside of the specific design considerations within the context of Google. SYSTEM DESIGN: The authors enumerate their specific considerations as: (1) commodity components with high expectation of failure, (2) a system optimized to handle relatively large files, particularly multi-GB files, (3) most writes to the system are concurrent append operations, rather than internally modifying the extant files, and (4) a high rate of sustained bandwidth (Section 1, 2.1). Utilizing these considerations, this paper analyzes the success of GFS’s design in achieving a fault-tolerant, scalable system while also considering the faults of the system with regards to data-trustworthiness and simplicity. Data-trustworthiness: In designing the GFS system, the authors made the conscious decision not prioritize data trustworthiness by allowing applications to access ‘stale’, or not up-to-date, data. In particular, although the system does inform applications of the chunk-server version number, the designers of GFS encouraged user applications to cache chunkserver information, allowing stale data accesses (Section 2.7.2, 4.5). Although possibly troubling
    [Show full text]
  • A Review on GOOGLE File System
    International Journal of Computer Science Trends and Technology (IJCST) – Volume 4 Issue 4, Jul - Aug 2016 RESEARCH ARTICLE OPEN ACCESS A Review on Google File System Richa Pandey [1], S.P Sah [2] Department of Computer Science Graphic Era Hill University Uttarakhand – India ABSTRACT Google is an American multinational technology company specializing in Internet-related services and products. A Google file system help in managing the large amount of data which is spread in various databases. A good Google file system is that which have the capability to handle the fault, the replication of data, make the data efficient, memory storage. The large data which is big data must be in the form that it can be managed easily. Many applications like Gmail, Facebook etc. have the file systems which organize the data in relevant format. In conclusion the paper introduces new approaches in distributed file system like spreading file’s data across storage, single master and appends writes etc. Keywords:- GFS, NFS, AFS, HDFS I. INTRODUCTION III. KEY IDEAS The Google file system is designed in such a way that 3.1. Design and Architecture: GFS cluster consist the data which is spread in database must be saved in of single master and multiple chunk servers used by a arrange manner. multiple clients. Since files to be stored in GFS are The arrangement of data is in the way that it can large, processing and transferring such huge files can reduce the overhead load on the server, the consume a lot of bandwidth. To efficiently utilize availability must be increased, throughput should be bandwidth files are divided into large 64 MB size highly aggregated and much more services to make chunks which are identified by unique 64-bit chunk the available data more accurate and reliable.
    [Show full text]
  • F1 Query: Declarative Querying at Scale
    F1 Query: Declarative Querying at Scale Bart Samwel John Cieslewicz Ben Handy Jason Govig Petros Venetis Chanjun Yang Keith Peters Jeff Shute Daniel Tenedorio Himani Apte Felix Weigel David Wilhite Jiacheng Yang Jun Xu Jiexing Li Zhan Yuan Craig Chasseur Qiang Zeng Ian Rae Anurag Biyani Andrew Harn Yang Xia Andrey Gubichev Amr El-Helw Orri Erling Zhepeng Yan Mohan Yang Yiqun Wei Thanh Do Colin Zheng Goetz Graefe Somayeh Sardashti Ahmed M. Aly Divy Agrawal Ashish Gupta Shiv Venkataraman Google LLC [email protected] ABSTRACT 1. INTRODUCTION F1 Query is a stand-alone, federated query processing platform The data processing and analysis use cases in large organiza- that executes SQL queries against data stored in different file- tions like Google exhibit diverse requirements in data sizes, la- based formats as well as different storage systems at Google (e.g., tency, data sources and sinks, freshness, and the need for custom Bigtable, Spanner, Google Spreadsheets, etc.). F1 Query elimi- business logic. As a result, many data processing systems focus on nates the need to maintain the traditional distinction between dif- a particular slice of this requirements space, for instance on either ferent types of data processing workloads by simultaneously sup- transactional-style queries, medium-sized OLAP queries, or huge porting: (i) OLTP-style point queries that affect only a few records; Extract-Transform-Load (ETL) pipelines. Some systems are highly (ii) low-latency OLAP querying of large amounts of data; and (iii) extensible, while others are not. Some systems function mostly as a large ETL pipelines. F1 Query has also significantly reduced the closed silo, while others can easily pull in data from other sources.
    [Show full text]
  • A Study of Cryptographic File Systems in Userspace
    Turkish Journal of Computer and Mathematics Education Vol.12 No.10 (2021), 4507-4513 Research Article A study of cryptographic file systems in userspace a b c d e f Sahil Naphade , Ajinkya Kulkarni Yash Kulkarni , Yash Patil , Kaushik Lathiya , Sachin Pande a Department of Information Technology PICT, Pune, India [email protected] b Department of Information Technology PICT, Pune, India [email protected] c Department of Information Technology PICT, Pune, India [email protected] d Department of Information Technology PICT, Pune, India [email protected] e Veritas Technologies Pune, India, [email protected] f Department of Information Technology PICT, Pune, India [email protected] Article History: Received: 10 January 2021; Revised: 12 February 2021; Accepted: 27 March 2021; Published online: 28 April 2021 Abstract: With the advancements in technology and digitization, the data storage needs are expanding; along with the data breaches which can expose sensitive data to the world. Thus, the security of the stored data is extremely important. Conventionally, there are two methods of storage of the data, the first being hiding the data and the second being encryption of the data. However, finding out hidden data is simple, and thus, is very unreliable. The second method, which is encryption, allows for accessing the data by only the person who encrypted the data using his passkey, thus allowing for higher security. Typically, a file system is implemented in the kernel of the operating systems. However, with an increase in the complexity of the traditional file systems like ext3 and ext4, the ones that are based in the userspace of the OS are now allowing for additional features on top of them, such as encryption-decryption and compression.
    [Show full text]
  • Apachecon US 2008 with Apache Shindig
    ApacheCon US 2008 Empowering the social web with Apache Shindig Henning Schmiedehausen Sr. Software Engineer – Ning, Inc. November 3 - 7 • New Orleans Leading the Wave of Open Source The Official User Conference of The Apache Software Foundation Freitag, 7. November 2008 1 • How the web became social • Get out of the Silo – Google Gadgets • OpenSocial – A social API • Apache Shindig • Customizing Shindig • Summary November 3 - 7 • New Orleans ApacheCon US 2008 Leading the Wave of Open Source The Official User Conference of The Apache Software Foundation Freitag, 7. November 2008 2 ApacheCon US 2008 In the beginning... Freitag, 7. November 2008 3 ApacheCon US 2008 ...let there be web 2.0 Freitag, 7. November 2008 4 • Web x.0 is about participation • Users have personalized logins Relations between users are graphs • "small world phenomenon", "six degrees of separation", Erdös number, Bacon number November 3 - 7 • New Orleans ApacheCon US 2008 Leading the Wave of Open Source The Official User Conference of The Apache Software Foundation Freitag, 7. November 2008 5 ApacheCon US 2008 The Silo problem Freitag, 7. November 2008 6 • How the web became social • Get out of the Silo – Google Gadgets • OpenSocial – A social API • Apache Shindig • Customizing Shindig • Summary November 3 - 7 • New Orleans ApacheCon US 2008 Leading the Wave of Open Source The Official User Conference of The Apache Software Foundation Freitag, 7. November 2008 7 ApacheCon US 2008 iGoogle Freitag, 7. November 2008 8 • Users adds Gadgets to their homepages Gadgets share screen space • Google experiments with Canvas view Javascript, HTML, CSS • A gadget runs on the Browser! Predefined Gadgets API • Core APIs for IO, JSON, Prefs; optional APIs (e.g.
    [Show full text]
  • Tracking Known Security Vulnerabilities in Third-Party Components
    Tracking known security vulnerabilities in third-party components Master’s Thesis Mircea Cadariu Tracking known security vulnerabilities in third-party components THESIS submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE in COMPUTER SCIENCE by Mircea Cadariu born in Brasov, Romania Software Engineering Research Group Software Improvement Group Department of Software Technology Rembrandt Tower, 15th floor Faculty EEMCS, Delft University of Technology Amstelplein 1 - 1096HA Delft, the Netherlands Amsterdam, the Netherlands www.ewi.tudelft.nl www.sig.eu c 2014 Mircea Cadariu. All rights reserved. Tracking known security vulnerabilities in third-party components Author: Mircea Cadariu Student id: 4252373 Email: [email protected] Abstract Known security vulnerabilities are introduced in software systems as a result of de- pending on third-party components. These documented software weaknesses are hiding in plain sight and represent the lowest hanging fruit for attackers. Despite the risk they introduce for software systems, it has been shown that developers consistently download vulnerable components from public repositories. We show that these downloads indeed find their way in many industrial and open-source software systems. In order to improve the status quo, we introduce the Vulnerability Alert Service, a tool-based process to track known vulnerabilities in software projects throughout the development process. Its usefulness has been empirically validated in the context of the external software product quality monitoring service offered by the Software Improvement Group, a software consultancy company based in Amsterdam, the Netherlands. Thesis Committee: Chair: Prof. Dr. A. van Deursen, Faculty EEMCS, TU Delft University supervisor: Prof. Dr. A.
    [Show full text]
  • Return of Organization Exempt from Income
    OMB No. 1545-0047 Return of Organization Exempt From Income Tax Form 990 Under section 501(c), 527, or 4947(a)(1) of the Internal Revenue Code (except black lung benefit trust or private foundation) Open to Public Department of the Treasury Internal Revenue Service The organization may have to use a copy of this return to satisfy state reporting requirements. Inspection A For the 2011 calendar year, or tax year beginning 5/1/2011 , and ending 4/30/2012 B Check if applicable: C Name of organization The Apache Software Foundation D Employer identification number Address change Doing Business As 47-0825376 Name change Number and street (or P.O. box if mail is not delivered to street address) Room/suite E Telephone number Initial return 1901 Munsey Drive (909) 374-9776 Terminated City or town, state or country, and ZIP + 4 Amended return Forest Hill MD 21050-2747 G Gross receipts $ 554,439 Application pending F Name and address of principal officer: H(a) Is this a group return for affiliates? Yes X No Jim Jagielski 1901 Munsey Drive, Forest Hill, MD 21050-2747 H(b) Are all affiliates included? Yes No I Tax-exempt status: X 501(c)(3) 501(c) ( ) (insert no.) 4947(a)(1) or 527 If "No," attach a list. (see instructions) J Website: http://www.apache.org/ H(c) Group exemption number K Form of organization: X Corporation Trust Association Other L Year of formation: 1999 M State of legal domicile: MD Part I Summary 1 Briefly describe the organization's mission or most significant activities: to provide open source software to the public that we sponsor free of charge 2 Check this box if the organization discontinued its operations or disposed of more than 25% of its net assets.
    [Show full text]
  • Cloud Computing Bible Is a Wide-Ranging and Complete Reference
    A thorough, down-to-earth look Barrie Sosinsky Cloud Computing Barrie Sosinsky is a veteran computer book writer at cloud computing specializing in network systems, databases, design, development, The chance to lower IT costs makes cloud computing a and testing. Among his 35 technical books have been Wiley’s Networking hot topic, and it’s getting hotter all the time. If you want Bible and many others on operating a terra firma take on everything you should know about systems, Web topics, storage, and the cloud, this book is it. Starting with a clear definition of application software. He has written nearly 500 articles for computer what cloud computing is, why it is, and its pros and cons, magazines and Web sites. Cloud Cloud Computing Bible is a wide-ranging and complete reference. You’ll get thoroughly up to speed on cloud platforms, infrastructure, services and applications, security, and much more. Computing • Learn what cloud computing is and what it is not • Assess the value of cloud computing, including licensing models, ROI, and more • Understand abstraction, partitioning, virtualization, capacity planning, and various programming solutions • See how to use Google®, Amazon®, and Microsoft® Web services effectively ® ™ • Explore cloud communication methods — IM, Twitter , Google Buzz , Explore the cloud with Facebook®, and others • Discover how cloud services are changing mobile phones — and vice versa this complete guide Understand all platforms and technologies www.wiley.com/compbooks Shelving Category: Use Google, Amazon, or
    [Show full text]
  • Apache Shindig V
    ...................................................................................................................................... Apache Shindig v. 1.0 User Guide ...................................................................................................................................... The Apache Software Foundation 2012-03-11 T a b l e o f C o n t e n t s i Table of Contents ....................................................................................................................................... 1. Table of Contents . i 2. Introduction . 1 3. Download . 3 4. Overview . 6 5. Getting Started . 16 6. Documentation Centre . 22 7. Java . 23 8. Building Java . 24 9. Samples . 28 10. PHP . 29 11. Building PHP . 30 12. Features . 32 13. Community Overview . 35 14. Getting Help . 37 15. Code Conventions . 38 16. Jira Conventions . 39 17. SVN Conventions . 40 18. Shindig Release Process . 42 19. FAQ . 46 20. Powered By . 48 21. Resources . 49 © 2 0 1 2 , T h e A p a c h e S o f t w a r e F o u n d a t i o n • A L L R I G H T S R E S E R V E D . T a b l e o f C o n t e n t s ii © 2 0 1 2 , T h e A p a c h e S o f t w a r e F o u n d a t i o n • A L L R I G H T S R E S E R V E D . 1 I n t r o d u c t i o n 1 1 Introduction ....................................................................................................................................... 1.1 Welcome To Apache Shindig ! Apache Shindig is an OpenSocial container and helps you to start hosting OpenSocial apps quickly by providing the code to render gadgets, proxy requests, and handle REST and RPC requests.
    [Show full text]
  • Liferay Portlet Display
    Liferay Portal Systems Development Build dynamic, content-rich, and social systems on top of Liferay Jonas X. Yuan BIRMINGHAM - MUMBAI Liferay Portal Systems Development Copyright © 2012 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First Edition: May 2009 Second Edition: January 2012 Production Reference: 1190112 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-84951-598-6 www.packtpub.com Cover Image by Rakesh Shejwal ([email protected]) Credits Author Project Coordinator Jonas X. Yuan Joel Goveya Reviewers Proofreaders Piotr Filipowicz Lesley Harrison Christianto Sahat Kurniawan Stephen Silk Szymon V. Gołębiewski Indexer Acquisition Editor Tejal Daruwale Sarah Cullington Graphics Lead Technical Editor Manu Joseph Hyacintha D'Souza Production Coordinator Technical Editors Aparna Bhagat Ankita Shashi Manasi Poonthottam Cover Work Aparna Bhagat Sakina Kaydawala Azharuddin Sheikh Copy Editors Leonard D'Silva Brandt D'Mello About the Author Dr.
    [Show full text]
  • Mapreduce: Simplified Data Processing On
    MapReduce: Simplified Data Processing on Large Clusters Jeffrey Dean and Sanjay Ghemawat [email protected], [email protected] Google, Inc. Abstract given day, etc. Most such computations are conceptu- ally straightforward. However, the input data is usually MapReduce is a programming model and an associ- large and the computations have to be distributed across ated implementation for processing and generating large hundreds or thousands of machines in order to finish in data sets. Users specify a map function that processes a a reasonable amount of time. The issues of how to par- key/value pair to generate a set of intermediate key/value allelize the computation, distribute the data, and handle pairs, and a reduce function that merges all intermediate failures conspire to obscure the original simple compu- values associated with the same intermediate key. Many tation with large amounts of complex code to deal with real world tasks are expressible in this model, as shown these issues. in the paper. As a reaction to this complexity, we designed a new Programs written in this functional style are automati- abstraction that allows us to express the simple computa- cally parallelized and executed on a large cluster of com- tions we were trying to perform but hides the messy de- modity machines. The run-time system takes care of the tails of parallelization, fault-tolerance, data distribution details of partitioning the input data, scheduling the pro- and load balancing in a library. Our abstraction is in- gram's execution across a set of machines, handling ma- spired by the map and reduce primitives present in Lisp chine failures, and managing the required inter-machine and many other functional languages.
    [Show full text]
  • Google File System 2.0: a Modern Design and Implementation
    Google File System 2.0: A Modern Design and Implementation Babatunde Micheal Okutubo Gan Tu Xi Cheng Stanford University Stanford University Stanford University [email protected] [email protected] [email protected] Abstract build a GFS-2.0 from scratch, with modern software design principles and implementation patterns, in order to survey The Google File System (GFS) presented an elegant de- ways to not only effectively achieve various system require- sign and has shown tremendous capability to scale and to ments, but also with great support for high-concurrency tolerate fault. However, there is no concrete implementa- workloads and software extensibility. tion of GFS being open sourced, even after nearly 20 years A second motivation of this project is that GFS has a of its initial development. In this project, we attempt to build single master, which served well back in 2003 when this a GFS-2.0 in C++ from scratch1 by applying modern soft- work was published. However, as data volume, network ware design discipline. A fully functional GFS has been de- capacity as well as hardware capability all have been dras- veloped to support concurrent file creation, read and paral- tically improved from that date, our hypothesis is that a lel writes. The system is capable of surviving chunk servers multi-master version of GFS is advantageous as it provides going down during both read and write operations. Various better fault tolerance (especially from the master servers’ crucial implementation details have been discussed, and a point of view), availability and throughput. Although the micro-benchmark has been conducted and presented.
    [Show full text]