DOCSLIB.ORG

IIP-ECOSPHERE PLATFORM HANDBOOK Version 0.20

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
IIP-ECOSPHERE PLATFORM HANDBOOK Version 0.20 IIP-Ecosphere Whitepaper IIP-ECOSPHERE PLATFORM HANDBOOK Version 0.20 Holger Eichelberger, Amir Shayan Ahmadian, Andreas Dewes, Marco Ehl, Monika Staciwa, Miguel Gómez Casado White Paper IIP-2021/005 2 IIP-Ecosphere Platform Handbook Disclaimer The contents of this document has been prepared with great carefulness. Although the information has been prepared with the greatest possible care, there is no claim to factual correctness, completeness and/or timeliness of data; in particular, this publication cannot take into account the specific circumstances of individual cases. Any use is therefore the reader's own responsibility. Any liability is excluded. This document contains material that is subject to the copyright of individual or multiple IIP-Ecosphere consortium parties. All rights, including reproduction of parts, are held by the authors. This document reflects only the views of the authors at the time of publication. The Federal Ministry for Economic Affairs and Energy or the responsible project agency are not liable for the use of the information contained herein. Publication: August, 2021 on https://www.iip-ecosphere.eu/ DOI: 10.5281/zenodo.5168946 3 IIP-Ecosphere Platform Handbook Executive Summary The IIP-Ecosphere platform is a central asset developed by the IIP-Ecosphere project. The core aim of the platform is to research and demonstrate novel platform concepts for Industry 4.0, e.g., asset administration shells as interfaces for software components and resources, unified edge deployment, an AI toolkit or seamless configuration of a platform from network settings via services up to applications running on the platform. This platform handbook provides insights into the rationales, ideas and concepts that make up the design and the realization of the IIP-Ecosphere platform, ranging from an overall layered architecture over a detailed discussion of the design and realization state of each layer up to cross-cutting mechanisms such as the configuration model or the related code/artifact generation. This platform handbook addresses the technical side of the platform work in IIP-Ecosphere and builds on the intensive prior work on requirements (usage view and functional/quality requirements of the platform). This handbook shall provide means for deeper technical discussions with partners, stakeholders and interested parties, but also allow for a technical understanding to contribute to the platform, e.g., in terms of protocols, platform connectors, services or demonstration applications. Acknowledgements: We are grateful to Dr. Christian Sauer and Ahmad Alomosh from the Software Systems Engineering Group of the University of Hildesheim for cross-reading this document and providing valuable feedback and ideas for improvement. 4 IIP-Ecosphere Platform Handbook Contents 1 Introduction ................................................................................................................................6 1.1 Motivation and Goals ..........................................................................................................6 1.2 Interaction with other initiatives..........................................................................................7 1.3 Structure of the document ..................................................................................................7 2 Tooling and Basic Technical Decisions .........................................................................................9 3 Architecture .............................................................................................................................. 12 3.1 Overview ........................................................................................................................... 12 3.1.1 Relation to Reference Architectures ........................................................................... 17 3.1.2 Asset Administration Shells ........................................................................................ 18 3.1.3 Virtual Character of the Platform ............................................................................... 19 3.2 Development Streams ....................................................................................................... 19 3.3 Overall Requirements ........................................................................................................ 20 3.4 UML Profiles ...................................................................................................................... 21 3.4.1 IIP-Ecosphere Profile .................................................................................................. 21 3.4.2 UMLSec Profile .......................................................................................................... 27 3.4.3 Security and Privacy Profile ........................................................................................ 28 3.5 Support Layer .................................................................................................................... 32 3.5.1 Asset Administration Shell Abstraction ....................................................................... 32 3.5.2 Network Support ....................................................................................................... 36 3.5.3 Lifecycle Support ....................................................................................................... 36 3.6 Transport and Connection Layer ........................................................................................ 36 3.6.1 Requirements ............................................................................................................ 36 3.6.2 Transport Component ................................................................................................ 38 3.6.3 Connectors Component ............................................................................................. 48 3.6.4 Requirements Discussion ........................................................................................... 56 3.7 Services Layer .................................................................................................................... 58 3.7.1 Terminology and Background ..................................................................................... 58 3.7.2 Requirements ............................................................................................................ 59 3.7.3 Service Environments................................................................................................. 60 3.7.4 Service Control and Management .............................................................................. 65 3.7.5 Requirements Discussion ........................................................................................... 69 3.8 Resources and Monitoring Layer ........................................................................................ 70 3.8.1 ECS runtime ............................................................................................................... 70 3.8.2 Device/Resource Management .................................................................................. 77 3.8.3 Monitoring................................................................................................................. 78 3.9 Security and Data Protection Layer .................................................................................... 79 5 IIP-Ecosphere Platform Handbook 3.10 Reusable Intelligent Services Layer .................................................................................... 79 3.11 Configuration Layer ........................................................................................................... 79 3.12 Application Layer ............................................................................................................... 86 3.13 Platform Server(s).............................................................................................................. 86 4 Architectural Constraints ........................................................................................................... 89 5 Asset Administration Shells ....................................................................................................... 90 6 Platform Configuration Model ................................................................................................... 93 7 Platform Security and Data Protection ...................................................................................... 99 7.1 Internal Security and Security/Privacy Analysis .................................................................. 99 7.2 External Security .............................................................................................................. 101 8 Implementation ...................................................................................................................... 103 8.1 Implementation decisions................................................................................................ 103 8.2 Obtaining the IIP-Ecosphere platform .............................................................................. 104 8.3 Compiling the IIP-Ecosphere platform .............................................................................. 107 8.4 Installing and using the IIP-Ecosphere platform ............................................................... 109 9 How to apply, extend or contribute ........................................................................................
Load more

Recommended publications
  • Java Linksammlung
    JAVA LINKSAMMLUNG LerneProgrammieren.de - 2020 Java einfach lernen (klicke hier) JAVA LINKSAMMLUNG INHALTSVERZEICHNIS Build ........................................................................................................................................................... 4 Caching ....................................................................................................................................................... 4 CLI ............................................................................................................................................................... 4 Cluster-Verwaltung .................................................................................................................................... 5 Code-Analyse ............................................................................................................................................. 5 Code-Generators ........................................................................................................................................ 5 Compiler ..................................................................................................................................................... 6 Konfiguration ............................................................................................................................................. 6 CSV ............................................................................................................................................................. 6 Daten-Strukturen
    [Show full text]
  • Thesis Artificial Intelligence Method Call Argument Completion Using
    Method Call Argument Completion using Deep Neural Regression Terry van Walen [email protected] August 24, 2018, 40 pages Academic supervisors: dr. C.U. Grelck & dr. M.W. van Someren Host organisation: Info Support B.V., http://infosupport.com Host supervisor: W. Meints Universiteit van Amsterdam Faculteit der Natuurwetenschappen, Wiskunde en Informatica Master Software Engineering http://www.software-engineering-amsterdam.nl Abstract Code completion is extensively used in IDE's. While there has been extensive research into the field of code completion, we identify an unexplored gap. In this thesis we investigate the automatic rec- ommendation of a basic variable to an argument of a method call. We define the set of candidates to recommend as all visible type-compatible variables. To determine which candidate should be recom- mended, we first investigate how code prior to a method call argument can influence a completion. We then identify 45 code features and train a deep neural network to determine how these code features influence the candidate`s likelihood of being the correct argument. After sorting the candidates based on this likelihood value, we recommend the most likely candidate. We compare our approach to the state-of-the-art, a rule-based algorithm implemented in the Parc tool created by Asaduzzaman et al. [ARMS15]. The comparison shows that we outperform Parc, in the percentage of correct recommendations, in 88.7% of tested open source projects. On average our approach recommends 84.9% of arguments correctly while Parc recommends 81.3% correctly. i ii Contents Abstract i 1 Introduction 1 1.1 Previous work........................................
    [Show full text]
  • Code Smell Prediction Employing Machine Learning Meets Emerging Java Language Constructs"
    Appendix to the paper "Code smell prediction employing machine learning meets emerging Java language constructs" Hanna Grodzicka, Michał Kawa, Zofia Łakomiak, Arkadiusz Ziobrowski, Lech Madeyski (B) The Appendix includes two tables containing the dataset used in the paper "Code smell prediction employing machine learning meets emerging Java lan- guage constructs". The first table contains information about 792 projects selected for R package reproducer [Madeyski and Kitchenham(2019)]. Projects were the base dataset for cre- ating the dataset used in the study (Table I). The second table contains information about 281 projects filtered by Java version from build tool Maven (Table II) which were directly used in the paper. TABLE I: Base projects used to create the new dataset # Orgasation Project name GitHub link Commit hash Build tool Java version 1 adobe aem-core-wcm- www.github.com/adobe/ 1d1f1d70844c9e07cd694f028e87f85d926aba94 other or lack of unknown components aem-core-wcm-components 2 adobe S3Mock www.github.com/adobe/ 5aa299c2b6d0f0fd00f8d03fda560502270afb82 MAVEN 8 S3Mock 3 alexa alexa-skills- www.github.com/alexa/ bf1e9ccc50d1f3f8408f887f70197ee288fd4bd9 MAVEN 8 kit-sdk-for- alexa-skills-kit-sdk- java for-java 4 alibaba ARouter www.github.com/alibaba/ 93b328569bbdbf75e4aa87f0ecf48c69600591b2 GRADLE unknown ARouter 5 alibaba atlas www.github.com/alibaba/ e8c7b3f1ff14b2a1df64321c6992b796cae7d732 GRADLE unknown atlas 6 alibaba canal www.github.com/alibaba/ 08167c95c767fd3c9879584c0230820a8476a7a7 MAVEN 7 canal 7 alibaba cobar www.github.com/alibaba/
    [Show full text]
  • Apache Pulsar and Its Enterprise Use Cases
    Apache Pulsar and its enterprise use cases Yahoo Japan Corporation Nozomi Kurihara July 18th, 2018 Who am I? Nozomi Kurihara • Software engineer at Yahoo! JAPAN (April 2012 ~) • Working on internal messaging platform using Apache Pulsar • Committer of Apache Pulsar Copyright (C) 2018 Yahoo Japan Corporation. All Rights Reserved. 2 Agenda 1. What is Apache Pulsar? 2. Why is Apache Pulsar useful? 3. How does Yahoo! JAPAN uses Apache Pulsar? Copyright (C) 2018 Yahoo Japan Corporation. All Rights Reserved. 3 What is Apache Pulsar? Copyright (C) 2018 Yahoo Japan Corporation. All Rights Reserved. 4 Agenda 1. What is Apache Pulsar? › History & Users › Pub-Sub messaging › Architecture › Client libraries › Topic › Subscription › Sample codes 2. Why is Apache Pulsar useful? 3. How does Yahoo! JAPAN uses Apache Pulsar? Copyright (C) 2018 Yahoo Japan Corporation. All Rights Reserved. 5 Apache Pulsar Flexible pub-sub system backed by durable log storage ▪ History: ▪ Competitors: › 2014 Development started at Yahoo! Inc. › Apache Kafka › 2015 Available in production in Yahoo! Inc. › RabbitMQ › Sep. 2016 Open-sourced (Apache License 2.0) › Apache ActiveMQ › June 2017 Moved to Apache Incubator Project › Apache RocketMQ › June 2018 Major version update: 2.0.1 etc. ▪ Users: › Oath Inc. (Yahoo! Inc.) › Comcast › The Weather Channel › Mercado Libre › Streamlio › Yahoo! JAPAN etc. Copyright (C) 2018 Yahoo Japan Corporation. All Rights Reserved. 6 Pub-Sub messaging Message transmission from one system to another via Topic ▪ Producers publish messages to Topics ▪ Consumers receive only messages from Topics to which they subscribe ▪ Decoupled (no need to know each other) → asynchronous, scalable, resilient Subscribe Consumer 1 Publish Producer Topic Consumer 2 message (log, notification, etc.) Consumer 3 Pub-Sub system Copyright (C) 2018 Yahoo Japan Corporation.
    [Show full text]
  • Performance Comparison of Message Queue Methods
    UNLV Theses, Dissertations, Professional Papers, and Capstones August 2019 Performance Comparison of Message Queue Methods Sanika Raje Follow this and additional works at: https://digitalscholarship.unlv.edu/thesesdissertations Part of the Computer Sciences Commons Repository Citation Raje, Sanika, "Performance Comparison of Message Queue Methods" (2019). UNLV Theses, Dissertations, Professional Papers, and Capstones. 3746. http://dx.doi.org/10.34917/16076287 This Thesis is protected by copyright and/or related rights. It has been brought to you by Digital Scholarship@UNLV with permission from the rights-holder(s). You are free to use this Thesis in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s) directly, unless additional rights are indicated by a Creative Commons license in the record and/ or on the work itself. This Thesis has been accepted for inclusion in UNLV Theses, Dissertations, Professional Papers, and Capstones by an authorized administrator of Digital Scholarship@UNLV. For more information, please contact [email protected]. PERFORMANCE COMPARISON OF MESSAGE QUEUE METHODS By Sanika N. Raje Bachelor of Science - Information Technology University of Mumbai 2012 Master in Computer Applications University of Mumbai 2015 A thesis submitted in partial fulfillment of the requirements for the Master of Science in Computer Science Department of Computer Science Howard R. Hughes College of Engineering The Graduate College University of Nevada, Las Vegas August 2019 ©Sanika N Raje, 2019 All Rights Reserved Thesis Approval The Graduate College The University of Nevada, Las Vegas May 16, 2019 This thesis prepared by Sanika N.
    [Show full text]
  • ANNUAL REPORT FY2019 01 May 2018 - 30 April 2019
    ANNUAL REPORT FY2019 01 May 2018 - 30 April 2019 THE APACHE​® ​ SOFTWARE FOUNDATION (ASF) Open. Innovation. Community. The Apache Way. “The most popular Open Source software is Apache”​ —DZone Celebrating 20 years of community-led development "The Apache Way", the world's largest Open Source foundation provides $20B+ worth of software for the public good at 100% no cost. The ASF develops, incubates, and shepherds 300+ freely-available, enterprise-grade projects that serve as the backbone for some of the most visible and widely used applications. The ubiquity of Apache software is undeniable, with Apache projects managing zettabytes of data, executing teraflops of operations, and storing billions of objects in virtually every industry. Apache software is an integral part of nearly every end user computing device, from laptops to tablets to phones. Apache software is used in every Internet-connected country on the planet. Apache projects serve as the backbone for some of the world’s most visible and widely used applications in Artificial Intelligence and Deep Learning, Big Data, build management, Cloud Computing, content management, DevOps, IoT and Edge computing, mobile, servers, and Web frameworks, among many other categories. …………………………………………………………………………………………………..……………………………...…………………………… FY2019 ANNUAL REPORT 1 Examples of the breadth of applications that are "Powered by Apache" include: ➔ Adobe​: powering I/O Runtime and the core of Experience Manager; ➔ Amazon Music​: tuning recommendations for 16M+ subscribers; ➔ AOL​: ingesting
    [Show full text]
  • Video Big Data Analytics in the Cloud: a Reference Architecture, Survey, Opportunities, and Open Research Issues
    Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000. Digital Object Identifier 10.1109/ACCESS.2017.DOI Video Big Data Analytics in the Cloud: A Reference Architecture, Survey, Opportunities, and Open Research Issues AFTAB ALAM, IRFAN ULLAH, AND YOUNG-KOO LEE Department of Computer Science and Engineering, Kyung Hee University (Global Campus), Yongin 1732, South Korea Corresponding author: Young-Koo Lee (e-mail: [email protected]) This work was supported by the Institute for Information and Communications Technology Promotion Grant through the Korea Government (MSIT) under Grant R7120-17-1007 (SIAT CCTV Cloud Platform). ABSTRACT The proliferation of multimedia devices over the Internet of Things (IoT) generates an unprecedented amount of data. Consequently, the world has stepped into the era of big data. Recently, on the rise of distributed computing technologies, video big data analytics in the cloud has attracted the attention of researchers and practitioners. The current technology and market trends demand an efficient framework for video big data analytics. However, the current work is too limited to provide a complete survey of recent research work on video big data analytics in the cloud, including the management and analysis of a large amount of video data, the challenges, opportunities, and promising research directions. To serve this purpose, we present this study, which conducts a broad overview of the state-of-the-art literature on video big data analytics in the cloud. It also aims to bridge the gap among large-scale video analytics challenges, big data solutions, and cloud computing. In this study, we clarify the basic nomenclatures that govern the video analytics domain and the characteristics of video big data while establishing its relationship with cloud computing.
    [Show full text]
  • Alibaba Cloud
    AAlliibbaabbaa CClloouudd Alibaba Cloud AliAblaibbabaa C Clloouudd M Mesessasgae gQeue Queueue for fMoQrT MT QTT Comparison Between MQTT and Comparison Between MQMTTQ and MQ Document Version: 20210311 Document Version: 20210311 Alibaba Cloud Message Queue for M Comparison Bet ween MQTT and MQ QTT ·Legal disclaimer Legal disclaimer Alibaba Cloud reminds you t o carefully read and fully underst and t he t erms and condit ions of t his legal disclaimer before you read or use t his document . If you have read or used t his document , it shall be deemed as your t ot al accept ance of t his legal disclaimer. 1. You shall download and obt ain t his document from t he Alibaba Cloud websit e or ot her Alibaba Cloud- aut horized channels, and use t his document for your own legal business act ivit ies only. The cont ent of t his document is considered confident ial informat ion of Alibaba Cloud. You shall st rict ly abide by t he confident ialit y obligat ions. No part of t his document shall be disclosed or provided t o any t hird part y for use wit hout t he prior writ t en consent of Alibaba Cloud. 2. No part of t his document shall be excerpt ed, t ranslat ed, reproduced, t ransmit t ed, or disseminat ed by any organizat ion, company or individual in any form or by any means wit hout t he prior writ t en consent of Alibaba Cloud. 3. The cont ent of t his document may be changed because of product version upgrade, adjust ment , or ot her reasons.
    [Show full text]
  • Eine Analyse Von Open-Source Message-Oriented Middlewares Hinsichtlich Der Integrationsfähigkeit Von Enterprise Integration Patterns
    Institut für Architektur von Anwendungssystemen Universität Stuttgart Universitätsstraße 38 D–70569 Stuttgart Fachstudie Eine Analyse von open-source Message-oriented Middlewares hinsichtlich der Integrationsfähigkeit von Enterprise Integration Patterns Timo Gutierrez, Marcel Graf Studiengang: Softwaretechnik Prüfer/in: Prof. Dr. Dr. h. c. Frank Leymann Betreuer/in: Michael Falkenthal, M.Sc. Beginn am: 1. April 2019 Beendet am: 1. Oktober 2019 Abstract Die Integration von modernen Softwaresystem stellt auf Grund von wachsender Komplexität und großen Datenmengen eine Herausforderung dar. Um diesen Herausforderungen zu begegnen wurden von Hohpe und Woolf Enterprise Integration Patterns (EIPs) vorgeschlagen. In diesem Kontext stellt der Integrationsstil Messaging und somit Message-Oriented Middlewares (MOMs) einen wichtigen Bestandteil dar. Sie dienen dem asynchronen und zuverlässigen Austausch von Nachrichten und vereinfachen die Integration verschiedener Plattformen und Programmiersprachen. Diese Arbeit untersucht open-source MOMs hinsichtlich ihrer Integrationsfähigkeit von EIPs. Hierfür werden die EIPs zuerst in fünf Gruppen kategorisiert. So beschreibt eine Gruppe die grundlegenden EIPs, welche integraler Bestandteile einer jeden MOM sind. Weitere EIPs umfassen nicht den Scope einer MOM. Im Rahmen dieser Arbeit werden deshalb die MOMs auf Grundlage der Gruppe weiterführenden EIPs detailliert untersucht. Diese detaillierte Analyse umfasst die drei open-source MOMs Apache ActiveMQ, Apache Kafka und RabbitMQ. Apache ActiveMQ unterstützt die weiterführenden EIPs vollständig und bietet darüber hinaus auch die einfachste Form der Implementierung. RabbitMQ und Kafka unterstützen nur Teilmengen der weiterführenden EIPs. Die Untersuchung bringt hervor, dass die Integration von EIPs in MOMs ein mächtiges Werzeug, jedoch in MOMs nicht weit verbreitet ist. 3 Inhaltsverzeichnis 1 Einleitung 15 1.1 Motivation ..................................... 15 1.2 Message-oriented Middleware ..........................
    [Show full text]
  • Apache Ignite Extensions - Modularization
    Apache Ignite Extensions - Modularization Saikat Maitra Twitter @samaitra Github samaitra Motivation • To keep Apache Ignite core modules and extensions modules to have separate release lifecycles. • Few integrations which are no longer in use can be deprecated. • Help Apache Ignite community to support core and extensions separately (test, release, fix, continue development). 2 Project Structure • Ignite-core • Ignite-modules • Memory-Centric Storage, Native Persistence, RDBMS • Spark Integration Integration (CacheStore for RDBMS) • SpringData and SpringBoot • Key-Value APIs • TensorFlow Integration • SQL Cassandra Integration • Compute Grid • • Service Grid • Machine Learning APIs • Advanced queries -scan, continuous • Transactions • Data Structures and Atomics • Ignite Messaging • Core Streaming APIs such as IgniteDataStreamer • Logging • Metrics & Tracing framework • Command line tools and scripts such as Visor and control.sh • Standard (aka. thick clients) - Java, .Net, C++. • Spring Core - needed for configuration needs. 3 Apache Ignite Extensions • Flink - Ignite Flink Streamer consumes messages from an Apache Flink consumer endpoint and feeds them into an Ignite cache. • Flume - IgniteSink is a Flume sink that extracts events from an associated Flume channel and injects into an Ignite cache. • Twitter - Ignite Twitter Streamer consumes messages from a Twitter Streaming API and inserts them into an Ignite cache. • ZeroMQ - Ignite ZeroMQ Streamer consumes messages from a ZeroMQ consumer endpoint and feeds them into an Ignite cache. • RocketMQ - Ignite RocketMQ Streamer consumes messages from an Apache RocketMQ consumer endpoint and feeds them into an Ignite cache. 4 Apache Ignite Extensions • Storm - Ignite Storm Streamer consumes messages from an Apache Storm consumer endpoint and feeds them into an Ignite cache. • MQTT - Ignite MQTT Streamer consumes messages from a MQTT topic and feeds transformed key-value pairs into an Ignite cache.
    [Show full text]
  • AI and Big Data on IBM Power Systems Servers
    Front cover AI and Big Data on IBM Power Systems Servers Ivaylo B. Bozhinov Anto Ajay Raj John Rafael Freitas de Lima Ahmed (Mash) Mashhour James Van Oosten Fernando Vermelho Allison White Redbooks IBM Redbooks AI and Big Data on IBM Power Systems Servers March 2019 SG24-8435-00 Note: Before using this information and the product it supports, read the information in “Notices” on page xv. First Edition (March 2019) This edition applies to IBM Power Systems servers that are based on the POWER9 and POWER8 processor-based technology that is running the software stack that is described in “Software levels” on page 120. © Copyright International Business Machines Corporation 2019. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Figures . vii Tables . xi Examples . xiii Notices . .xv Trademarks . xvi Preface . xvii Authors. xviii Now you can become a published author, too! . xix Comments welcome. xix Stay connected to IBM Redbooks . .xx Chapter 1. Solution overview. 1 1.1 Introduction . 2 1.1.1 Types of AI . 2 1.1.2 What is a data lake . 3 1.2 Artificial intelligence solutions . 4 1.2.1 IBM PowerAI . 4 1.2.2 IBM Watson Machine Learning Accelerator . 5 1.2.3 IBM Watson Studio Local . 7 1.2.4 H2O Driverless AI . 9 1.2.5 IBM PowerAI Vision . 10 1.2.6 Key differences among the popular AI solutions. 11 1.3 Data platforms. 11 1.3.1 Apache Hadoop and Hortonworks . 11 1.3.2 IBM Spectrum Scale .
    [Show full text]
  • Apache Rocketmq 介绍
    链滴 Apache RocketMQ 介绍 作者：boccaro 原文链接：https://ld246.com/article/1588041859812 来源网站：链滴 许可协议：署名-相同方式共享 4.0 国际 (CC BY-SA 4.0) Apache RocketMQ 介绍 概要 Apache RocketMQ是一个分布式消息传递和流媒体平台，具有低延迟，高性能和可靠性，万亿级容 和灵活的可伸缩性。它的一个重要特性是支持非日志类型的可靠消息传送，非常适合运用在金融和电 商务领域。目前他是Apache社区的顶级项目，在全球有超过100家公司在其业务中使用RocketMQ 开源版本。 诞生 RocketMQ起源于阿里巴巴。阿里巴巴最初由于业务需求，需要使用消息中间件。早期使用过Notify ActiveMQ等。但在需求不断膨胀的情况下，ActiveMQ IO模块遇到了瓶颈，几经努力但改善成果不 。这时正值Kafka流行，于是引起了阿里巴巴开发团队的注意，对kafka的无限消息堆积，高效持久化 度等特性非常赞赏。但不幸的是，Kafka不能满足他们的要求，特别是在低延迟和高可靠性方面。在 种情况下，阿里巴巴决定发明一个新的消息传递引擎来处理更广泛的用例集，从传统的发布/订阅方 到大批量实时零损失容忍交易系统。 里程碑 2012年，阿里巴巴开始开发RocketMQ，经历了数次双11核心交易链路检验。 2016年11月11日，RocketMQ又一次在阿里巴巴全球购物节上处理了1.2万亿个并发在线消息传输， 后阿里巴巴将RocketMQ捐献给Apache Incubator。 2017年9月25日 – Apache软件基金会，连同350多个开源项目的全体志愿者、开发人员、管理人员 和孵化项目组织，宣布Apache®RocketMQ™从Apache孵化器毕业成为顶级项目，这表明该项目的 区和产品已根据ASF的精英流程和原则得到了很好的管理。 现今，Apache RocketMQ在社区各方面的努力下，茁壮发展，很多功能都得到了加强。 RocketMQ的技术概览 在我们看来，它最大的创新点在于能够通过精巧的横向、纵向扩展，不断满足与日俱增的海量消息在 吞吐、高可靠、低延迟方面的要求。 目前RocketMQ主要由NameServer、Broker、Producer以及Consumer四部分构成，如下图所示。 原文链接：Apache RocketMQ 介绍 NameServer以轻量级的方式提供服务发现和路由功能，每个NameServer存有全量的路由信息，提 对等的读写服务，支持快速扩缩容。 Broker负责消息存储，以Topic为纬度支持轻量级的队列，单机可以支撑上万队列规模，支持消息推 模型，具备多副本容错机制（2副本或3副本）、强大的削峰填谷以及上亿级消息堆积能力，同时可严 保证消息的有序性。除此之外，Broker还提供了同城异地容灾能力，丰富的Metrics统计以及告警机 。这些都是传统消息系统无法比拟的。 Producer由用户进行分布式部署，消息由Producer通过多种负载均衡模式发送到Broker集群，发送 延时，支持快速失败。 Consumer也由用户部署，支持PUSH和PULL两种消费模式，支持集群消费和广播消息，提供实时的 息订阅机制，满足大多数消费场景。 特点 ● RocketMQ支持异步实时刷盘，同步刷盘，同步复制，异步复制。具有高可靠性。不会因为操作系 的崩溃而导致数据丢失。 ● RocketMQ经过一系列的实践和优化，处理速度从最初的10,000TPS到目前已经超过50,000TPS。 纯比较TPS的话虽然还不如Kafka的百万级别，但在支持事务的消息中间件来说已经是非常优秀了。
    [Show full text]