DOCSLIB.ORG

Technical Report

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Technical Report Comparison and Performance Evaluation of Processing Platforms – Technical Report Gala Barquero Javier Troya Antonio Vallecillo July 21, 2020 Abstract This technical report serves as a complementary study to our paper entitled ‘Improving Query Performance on Dynamic Graphs’, where we propose an algorithm that reduces the input data in order to improve the memory consumption and time performance. To achieve this, it selects only the information that is relevant for a query. In this document we analyse six processing platforms and three languages commonly used to work with graphs, in order to find the most suitable technology to develop our algorithm. These technologies have been evaluated in two case studies and we have chosen the processing platform and DSL combination that better suit with our requirements. 1 Introduction Current companies and organizations increasingly demand the processing of data streams generated from different sources, such us social networks and streaming platforms. The main challenge of these applications is to provide efficient-enough responses when dealing with large amounts of real-time information. Despite this kind of information is sometimes represented as flows of single events without any relation among them, it is true that there are cases in which the information is composed of data related among them. In these cases, the sources generate data-structured information in the form of graphs, such as in social networks or geolocation systems domains. Furthermore, data of different nature that come from heterogeneous sources have to be considered. In this way, two types of information are normally taken into account in our approach: persistent and transitory. The former is permanently stored in the system (e.g. users in a social network or coordinates in a geolocation system), whereas the latter is temporarily stored and expires after some period of time (e.g. tweets or routes). This means that we need to consider heterogeneous nature and structure of the information, which is indeed a challenge. This technical report is a complementary document to our paper entitled ‘Improving Query Performance on Dynamic Graphs’. The approach of this paper is based on the fact that not all the information that is stored in the system at the current time is useful to obtain valid conclusions. For this reason, we have developed an algorithm, called Source DataSet Reduction (SDR) algorithm, to improve the performance of queries over the graphs by selecting only relevant data to be queried. Then, to achieve this, it was necessary to find a technology and a language that meet three fundamental requirements: (i) they must allow to perform queries and update the information as quickly as possible in order to provide real-time responses, (ii) they must cope with graph-structured information, and (iii) they must provide a clear syntax in order to be able to study the type of query to be run over the data. Then, in this technical report, we analyse the benefits and limitations of several processing platforms and query languages when working with high volumes of data. The platforms comprise TinkerGraph [37], Neo4j [28], CrateDB [10], Memgraph [25], JanusGraph [23] and OrientDB [8], whereas the languages comprise Gremlin [3], Cypher [27], and SQL. All these technologies are commonly used in Big Data applications [24]. In order to study their performance, we tested them in two case studies and we measured the execution time of 1 Figure 1: Twitter and Flickr joint Metamodel. the queries in two scenarios. First, queries are run over the data in order to obtain useful information without implying any side effect in the source information. Second, same queries are run but their results modify the source graph by adding, removing or updating the existing information. In addition, the complexity of the languages is measured and compared by counting the number of characters, operators and internal variables used for each query. Finally, a combination of platform and query language is chosen to implement the SDR algorithm, according to our requirements. The structure of this document is as follows. First, in Section 2 we present a running example in the domain of social networks to illustrate our proposal. Then, Section 3 exposes 6 processing platforms that we compare in our study and their main features, whereas Section 4 presents the query languages that are used for these platforms. Platforms and languages are evaluated and compared in Section 5 in terms of execution time and complexity of the syntax, by using two case studies where one platform and one language are chosen as the most suitable combination for our proposal. Finally, in Section 6 we discuss related work and Section 7 concludes the technical report. 2 A running example In order to illustrate the DSL syntax used for each Graph Processing Platform studied in this technical report, consider a system that works with information provided by Flickr1 and Twitter2, and analyses them together to identify some situations of interest [6]. The metamodel is depicted in Figure 1, where there is one common class between the two domains, namely Hashtag. Note that there is no automated manner to implement a completely reliable relation among the remaining elements (e.g. automatically relating users based on their Twitter and Flickr identifiers is impossible). Given such a metamodel, we are interested in identifying the following situations of interest: 1https://www.flickr.com/ 2https://twitter.com/ 2 Q1. HotTopic: A hashtag has been used by both Twitter and Flickr users at least 100 times in the last hour. We would like to generate a new HotTopic object that refers to this hashtag. Q2. PopularTwitterPhoto: The hashtag of a photo is mentioned in a tweet that receives more than 30 likes in the last hour. A PopularTwitterPhoto element is created. Q3. PopularFlickrPhoto: A photo is favored by more than 50 Flickr users who have more than 50 followers. A PopularFlickrPhoto element is created. Q4. NiceTwitterPhoto: A user, with an h-index 3 higher than 50, posts three tweets in a row in the last hour containing a hashtag that describes a photo. In this case, we generate a NiceTwitterPhoto object. Q5. ActiveUserTweeted: Considering the 10000 most recent tweets, search a user that has published one of these tweets, with h-index higher than 30 that follows more than 5K users. In this case, an ActiveUserTweeted object is generated pointing to the user. Note that we have included 5 extra objects (shaded in gray) in the metamodel. These objects refer to the objects created as a result of the queries. In this way, a model of this application contains both the information from the sources (Twitter and Flickr) and also the information we generate during its lifetime, as the system evolves. 3 Processing platforms As mentioned in Section 1, one of our requirements is to provide real-time responses, which requires low-latency in the processing. Thus, we have studied six powerful platforms for processing huge amounts of data. These platforms have been tested in two case studies with graph-structured information (explained in the following sections), in order to choose the technology with the lowest latency. In addition to this feature, other aspects have been taken into account when choosing the most suitable technology for our proposal, such as the DSL expressiveness or the possibility of updating the information. In this section, an overview of the six platforms is presented. These technologies include five graph databases (Neo4j, JanusGraph, OrientDB, TinkerGraph and Memgraph) and a distributed SQL database (CrateDB). Their features are summarized in Table 1. Platform Distributed In-memory Disk Updatable Query languages Neo4j No No Yes Yes Cypher JanusGraph Yes Yes Yes Yes Gremlin OrientDB Yes Yes Yes Yes Gremlin, SQL TinkerGraph No Yes Yes Yes Cypher, Gremlin Memgraph Yes Yes Yes Yes Cypher CrateDB Yes No Yes Yes SQL Table 1: Processing platforms used in the experiments of the performance study 3.1 Neo4j Neo4j [28] is an open-source graph database that combines native graph storage (since the data are stored as a graph and pointers are used to navigate and traverse its elements), ACID transaction compliance (that implies Atomicity, Consistency, Isolation and Durability) and a scalable architecture designed to 3In Twitter, a user has an h-index of h if she has at least h followers who have more than h followers each. 3 perform queries quickly. Neo4j stores the data in disk and its features makes it suitable for production scenarios. In order to provide an intuitive way to manipulate the database, Neo4j uses Cypher [27] as query language. This DSL is explained in Section 4.2 in detail. Some domains where Neo4j is commonly used are financial services, manufacturing, technology companies, etc. 3.2 JanusGraph JanusGraph [23] is an open source scalable graph database designed to be distributed in a multi-machine cluster. This feature makes it suitable to store and query large graphs with hundred of billions of elements between edges and nodes. However, it also allows single-node configuration. JanusGraph can store the data in memory or in disk by making use of a backend with Apache Cassandra, Apache HBase, Google Cloud Bigtable, Oracle BerkeleyDB or ScyllaDB. Among the most important benefits of JanusGraph we find support for global graph analytics through the Hadoop framework, concurrent transactions and operational graph processing as well as native support for Apache Tinkerpop [2] and Gremlin language [3]. This DSL is explained in Section 4.3 in detail. In this technical report, we have implemented JanusGraph with the BerkeleyDB backend (disk storage). 3.3 OrientDB OrientDB [8] is an open source multi-model and NoSQL database that is written in Java and it provides both the power of graphs and the flexibility and scalability of documents into one high-performance operational database.
Load more

Recommended publications
  • An Evaluation of the Design Space for Scalable Data Loading Into Graph Databases
    Otto-von-Guericke-Universit¨at Magdeburg Faculty of Computer Science Databases D B and Software S E Engineering Master's Thesis An Evaluation of the Design Space for Scalable Data Loading into Graph Databases Author: Jingyi Ma February 23, 2018 Advisors: M.Sc. Gabriel Campero Durand Data and Knowledge Engineering Group Prof. Dr. rer. nat. habil. Gunter Saake Data and Knowledge Engineering Group Ma, Jingyi: An Evaluation of the Design Space for Scalable Data Loading into Graph Databases Master's Thesis, Otto-von-Guericke-Universit¨at Magdeburg, 2018. Abstract In recent years, computational network science has become an active area. It offers a wealth of tools to help us gain insight into the interconnected systems around us. Graph databases are non-relational database systems which have been developed to support such network-oriented workloads. Graph databases build a data model based on graph abstractions (i.e. nodes/vertexes and edges) and can use different optimizations to speed up the basic graph processing tasks, such as traversals. In spite of such benefits, some tasks remain challenging in graph databases, such as the task of loading the complete dataset. The loading process has been considered to be a performance bottleneck, specifically a scalability bottleneck, and application developers need to conduct performance tuning to improve it. In this study, we study some optimization alternatives that developers have for load data into a graph databases. With this goal, we propose simple microbenchmarks of application-level load optimizations and evaluate these optimizations experimentally for loading real world graph datasets. We run our tests using JanusGraphLab, a JanusGraph prototype.
    [Show full text]
  • Data-Centric Graphical User Interface of the ATLAS Event Index Service
    EPJ Web of Conferences 245, 04036 (2020) https://doi.org/10.1051/epjconf/202024504036 CHEP 2019 Data-centric Graphical User Interface of the ATLAS Event Index Service Julius Hrivnᡠcˇ1,∗, Evgeny Alexandrov2, Igor Alexandrov2, Zbigniew Baranowski3, Dario Barberis4, Gancho Dimitrov3, Alvaro Fernandez Casani5, Elizabeth Gallas6, Carlos Gar- cía Montoro5, Santiago Gonzalez de la Hoz5, Andrei Kazymov2, Mikhail Mineev2, Fedor Prokoshin2, Grigori Rybkin1, Javier Sanchez5, Jose Salt5, Miguel Villaplana Perez7 1Université Paris-Saclay, CNRS/IN2P3, IJCLab, 91405 Orsay, France 2Joint Institute for Nuclear Research, 6 Joliot-Curie St., Dubna, Moscow Region, 141980, Russia 3CERN, 1211 Geneva 23, Switzerland 4Physics Department of the University of Genoa and INFN Sezione di Genova, Via Dodecaneso 33, I-16146 Genova, Italy 5Instituto de Fisica Corpuscular (IFIC), Centro Mixto Universidad de Valencia - CSIC, Valencia, Spain 6Department of Physics, Oxford University, Oxford, United Kingdom 7Department of Physics, University of Alberta, Edmonton AB, Canada Abstract. The Event Index service of the ATLAS experiment at the LHC keeps references to all real and simulated events. Hadoop Map files and HBase tables are used to store the Event Index data, a subset of data is also stored in the Oracle database. Several user interfaces are currently used to access and search the data, from a simple command line interface, through a programmable API, to sophisticated graphical web services. It provides a dynamic graph-like overview of all available data (and data collections). Data are shown together with their relations, like paternity or overlaps. Each data entity then gives users a set of actions available for the referenced data. Some actions are provided directly by the Event Index system, others are just interfaces to different ATLAS services.
    [Show full text]
  • Release Notes Date Published: 2021-03-25 Date Modified
    Cloudera Runtime 7.2.8 Release Notes Date published: 2021-03-25 Date modified: https://docs.cloudera.com/ Legal Notice © Cloudera Inc. 2021. All rights reserved. The documentation is and contains Cloudera proprietary information protected by copyright and other intellectual property rights. No license under copyright or any other intellectual property right is granted herein. Copyright information for Cloudera software may be found within the documentation accompanying each component in a particular release. Cloudera software includes software from various open source or other third party projects, and may be released under the Apache Software License 2.0 (“ASLv2”), the Affero General Public License version 3 (AGPLv3), or other license terms. Other software included may be released under the terms of alternative open source licenses. Please review the license and notice files accompanying the software for additional licensing information. Please visit the Cloudera software product page for more information on Cloudera software. For more information on Cloudera support services, please visit either the Support or Sales page. Feel free to contact us directly to discuss your specific needs. Cloudera reserves the right to change any products at any time, and without notice. Cloudera assumes no responsibility nor liability arising from the use of products, except as expressly agreed to in writing by Cloudera. Cloudera, Cloudera Altus, HUE, Impala, Cloudera Impala, and other Cloudera marks are registered or unregistered trademarks in the United States and other countries. All other trademarks are the property of their respective owners. Disclaimer: EXCEPT AS EXPRESSLY PROVIDED IN A WRITTEN AGREEMENT WITH CLOUDERA, CLOUDERA DOES NOT MAKE NOR GIVE ANY REPRESENTATION, WARRANTY, NOR COVENANT OF ANY KIND, WHETHER EXPRESS OR IMPLIED, IN CONNECTION WITH CLOUDERA TECHNOLOGY OR RELATED SUPPORT PROVIDED IN CONNECTION THEREWITH.
    [Show full text]
  • The Ubiquity of Large Graphs and Surprising Challenges of Graph Processing: Extended Survey
    The VLDB Journal https://doi.org/10.1007/s00778-019-00548-x SPECIAL ISSUE PAPER The ubiquity of large graphs and surprising challenges of graph processing: extended survey Siddhartha Sahu1 · Amine Mhedhbi1 · Semih Salihoglu1 · Jimmy Lin1 · M. Tamer Özsu1 Received: 21 January 2019 / Revised: 9 May 2019 / Accepted: 13 June 2019 © Springer-Verlag GmbH Germany, part of Springer Nature 2019 Abstract Graph processing is becoming increasingly prevalent across many application domains. In spite of this prevalence, there is little research about how graphs are actually used in practice. We performed an extensive study that consisted of an online survey of 89 users, a review of the mailing lists, source repositories, and white papers of a large suite of graph software products, and in-person interviews with 6 users and 2 developers of these products. Our online survey aimed at understanding: (i) the types of graphs users have; (ii) the graph computations users run; (iii) the types of graph software users use; and (iv) the major challenges users face when processing their graphs. We describe the participants’ responses to our questions highlighting common patterns and challenges. Based on our interviews and survey of the rest of our sources, we were able to answer some new questions that were raised by participants’ responses to our online survey and understand the specific applications that use graph data and software. Our study revealed surprising facts about graph processing in practice. In particular, real-world graphs represent a very diverse range of entities and are often very large, scalability and visualization are undeniably the most pressing challenges faced by participants, and data integration, recommendations, and fraud detection are very popular applications supported by existing graph software.
    [Show full text]
  • Towards Scalable Loading Into Graph Database Systems
    Piecing together large puzzles, efficiently: Towards scalable loading into graph database systems Work in progress paper Gabriel Campero Durand Jingy Ma Marcus Pinnecke University of Magdeburg University of Magdeburg University of Magdeburg [email protected] [email protected] [email protected] Gunter Saake University of Magdeburg [email protected] Transaction checks ABSTRACT Id assignation 5 Many applications rely on network analysis to extract busi- Physical storage ness intelligence from large datasets, requiring specialized graph tools such as processing frameworks (e.g. Apache Gi- 4 Process organization raph, Gradoop), database systems (e.g. Neo4j, JanusGraph) Batching or applications/libraries (e.g. NetworkX, nvGraph). A recent Partitioning survey reports scalability, particularly for loading, as the fo- Pre-processing 3 remost practical challenge faced by users. In this paper we consider the design space of tools for efficient and scalable 1 Parsing for different input characteristics File Interpretation 2 graph bulk loading. For this we implement a prototypical CSV, JSON, Adjacency lists, Implicit graphs... loader for a property graph DBMS, using a distributed mes- sage bus. With our implementation we evaluate the impact Figure 1: Bulk Loading into a Graph Storage and limits of basic optimizations. Our results confirm the expectation that bulk loading can be best supported as a server-side process. We also find, for our specific case, gains 1. INTRODUCTION from batching writes (up to 64x speedups in our evaluati- Network analysis is one of many methods used by scien- on), uniform behavior across partitioning strategies, and the tists to study large data collections. For this, data has to be need for careful tuning to find the optimal configuration of represented with models based on graph theory.
    [Show full text]
  • Using Graph Databases, We Are Moving Essential Structure from Code to Data, Together with a Migration from an Imperative to a Declarative Coding Semantics
    Using Graph Databases Julius Hrivnᡠcˇ1,∗ on behalf of the ATLAS Computing Group 1Université Paris-Saclay, CNRS/IN2P3, IJCLab, 91405 Orsay, France Abstract. Data in High Energy Physics (HEP) are usually stored in tuples (ta- bles), trees, nested tuples (trees of tuples) or relational (SQL-like) databases, with or without defined schema. But many of our data are graph-like and schema-less. They consist of entities with relations, some of which are known in advance, but many are created ad-hoc, later. Such structures are not well covered by relational (SQL) databases. We don’t only need the possibility of adding new data with the existing defined relations, we also need to add new relations. Graph Databases have existed for a long time, but have only matured recently thanks to Big Data and AI (adaptive NN). There are now very good implemen- tations and de-facto standards available. The difference between SQL and Graph Database is similar to the difference between Fortran and C++. On one side, a rigid system, which can be very opti- mized. On the other side, a flexible dynamical system, which allows expression of complex structures. GraphDB is a synthesis of OODB and SQLDB. They al- low expression of a web of objects without fragility of OO world. They capture only essential relations, without keeping a complete object dump. Migrating to Graph Database means moving structure from code to data, together with migration from imperative to declarative semantics (things don’t happen, but exist). The paper describes basic principles of the Graph Database together with an overview of existing standards and implementations.
    [Show full text]
  • Suitability of Graph Database Technology for the Analysis of Spatio-Temporal Data
    future internet Article Suitability of Graph Database Technology for the Analysis of Spatio-Temporal Data Sedick Baker Effendi 1,* , Brink van der Merwe 1 and Wolf-Tilo Balke 2 1 Department of Computer Science, Stellenbosch University, Stellenbosch 7600, South Africa; [email protected] 2 Institute for Information Systems, TU Braunschweig, D-38106 Braunschweig, Germany; balke@ifis.cs.tu-bs.de * Correspondence: [email protected]; Tel.: +27-834-324-666 Received: 30 March 2020; Accepted: 23 April 2020; Published: 26 April 2020 Abstract: Every day large quantities of spatio-temporal data are captured, whether by Web-based companies for social data mining or by other industries for a variety of applications ranging from disaster relief to marine data analysis. Making sense of all this data dramatically increases the need for intelligent backend systems to provide realtime query response times while scaling well (in terms of storage and performance) with increasing quantities of structured or semi-structured, multi-dimensional data. Currently, relational database solutions with spatial extensions such as PostGIS, seem to come to their limits. However, the use of graph database technology has been rising in popularity and has been found to handle graph-like spatio-temporal data much more effectively. Motivated by the need to effectively store multi-dimensional, interconnected data, this paper investigates whether or not graph database technology is better suited when compared to the extended relational approach. Three database technologies will be investigated using real world datasets namely: PostgreSQL, JanusGraph, and TigerGraph. The datasets used are the Yelp challenge dataset and an ambulance response simulation dataset, thus combining real world spatial data with realistic simulations offering more control over the dataset.
    [Show full text]
  • System G Distributed Graph Database
    System G Distributed Graph Database Gabriel Tanase Toyotaro Suzumura Jinho Lee IBM T.J. Watson Research IBM T.J. Watson Research IBM T.J. Watson Research Center Center Center 1101 Kitchawan Rd 1101 Kitchawan Rd 11501 Burnet Rd Yorktown Heights, NY 10598 Yorktown Heights, NY 10598 Austin, TX 78759 [email protected] [email protected] [email protected] Chun­Fu (Richard) Chen Jason Crawford Hiroki Kanezashi IBM T.J. Watson Research IBM T.J. Watson Research IBM T.J. Watson Research Center Center Center 1101 Kitchawan Rd 1101 Kitchawan Rd 1101 Kitchawan Rd Yorktown Heights, NY 10598 Yorktown Heights, NY 10598 Yorktown Heights, NY 10598 [email protected] [email protected] [email protected] ABSTRACT lems that need to be addressed when working with big data Motivated by the need to extract knowledge and value from is how to efficiently store and query them. In this paper we interconnected data, graph analytics on big data is a very focus on these challenges in the context of large connected active area of research in both industry and academia. To data sets that are modeled as graphs. Graphs are often used support graph analytics efficiently a large number of in mem- to represent linked concepts like communities of people and ory graph libraries, graph processing systems and graph things they like, bank accounts and transactions, or cities databases have emerged. Projects in each of these cate- and connecting roads. For all these examples vertices are gories focus on particular aspects such as static versus dy- in order of hundred of millions and edges or relations are in namic graphs, off line versus on line processing, small versus the order of tens of billions.
    [Show full text]
  • Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries
    1 Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries MACIEJ BESTA, Department of Computer Science, ETH Zurich ROBERT GERSTENBERGER, Department of Computer Science, ETH Zurich EMANUEL PETER, Department of Computer Science, ETH Zurich MARC FISCHER, PRODYNA (Schweiz) AG MICHAŁ PODSTAWSKI, Future Processing CLAUDE BARTHELS, Department of Computer Science, ETH Zurich GUSTAVO ALONSO, Department of Computer Science, ETH Zurich TORSTEN HOEFLER, Department of Computer Science, ETH Zurich Graph processing has become an important part of multiple areas of computer science, such as machine learning, computational sciences, medical applications, social network analysis, and many others. Numerous graphs such as web or social networks may contain up to trillions of edges. Often, these graphs are also dynamic (their structure changes over time) and have domain-specific rich data associated with vertices and edges. Graph database systems such as Neo4j enable storing, processing, and analyzing such large, evolving, and rich datasets. Due to the sheer size of such datasets, combined with the irregular nature of graph processing, these systems face unique design challenges. To facilitate the understanding of this emerging domain, we present the first survey and taxonomy of graph database systems. We focus on identifying and analyzing fundamental categories of these systems (e.g., triple stores, tuple stores, native graph database systems, or object-oriented systems), the associated graph models (e.g., RDF or Labeled Property Graph), data organization techniques (e.g., storing graph data in indexing structures or dividing data into records), and different aspects of data distribution and query execution (e.g., support for sharding and ACID).
    [Show full text]
  • Query Analysis on a Distributed Graph Database Student: Bc
    ASSIGNMENT OF MASTER’S THESIS Title: Query Analysis on a Distributed Graph Database Student: Bc. Lucie Svitáková Supervisor: Ing. Michal Valenta, Ph.D. Study Programme: Informatics Study Branch: Web and Software Engineering Department: Department of Software Engineering Validity: Until the end of summer semester 2019/20 Instructions 1. Research current practicies of data storage in distributed graph databases. 2. Choose an existing database system for your implementation. 3. Analyse how to extract necessary information for further query analysis and implement that extraction. 4. Create a suitable method to store the data extracted in step 3. 5. Propose a method for a more efficient redistribution of the data. References Will be provided by the supervisor. Ing. Michal Valenta, Ph.D. doc. RNDr. Ing. Marcel Jiřina, Ph.D. Head of Department Dean Prague October 5, 2018 Master’s thesis Query Analysis on a Distributed Graph Database Bc. et Bc. Lucie Svit´akov´a Department of Software Engineering Supervisor: Ing. Michal Valenta, Ph.D. January 8, 2019 Acknowledgements I would like to thank my supervisor Michal Valenta for his valuable suggestions and observations, my sister S´arkaSvit´akov´aforˇ spelling corrections, my friends Martin Horejˇsand Tom´aˇsNov´aˇcekfor their valuable comments and my family for their continual motivation and support. Declaration I hereby declare that the presented thesis is my own work and that I have cited all sources of information in accordance with the Guideline for adhering to ethical principles when elaborating an academic final thesis. I acknowledge that my thesis is subject to the rights and obligations stip- ulated by the Act No.
    [Show full text]
  • Graph Computing with Janusgraph Jason Plurad Open Source Developer & Advocate
    Graph Computing with JanusGraph Jason Plurad Open Source Developer & Advocate LDBC 11th Technical User Community (TUC) Meeting / University of Texas at Austin / June 8, 2018 / © 2018 IBM Corporation JanusGraph • Established in January 2017 • Scalable graph database distributed on multi-machine clusters with pluggable storage and indexing • Vendor-neutral, open community with open governance JanusGraph™ • Founders: Expero, Google, Grakn, Maintainer The Linux Hortonworks, IBM Foundation License Apache Releases 0.3.0 planned 2Q 2018 https://janusgraph.org LDBC 11th Technical User Community (TUC) Meeting / University of Texas at Austin / June 8, 2018 / © 2018 IBM Corporation 2 JanusGraph Community • Contributors • Member companies • 49 total • Amazon • Huawei • Committers • Linkurious • 14 initial, 6 added • Netflix • Newforma • Technical Steering Committee • Orchestral Developments • 6 initial, 2 added • Uber • Issues • In production • 287 open, 352 closed • Celum • Finc • Open source projects • G-Data • Apache Atlas • IBM Cloud • Open Network Automation Platform (ONAP) • Times Internet LDBC 11th Technical User Community (TUC) Meeting / University of Texas at Austin / June 8, 2018 / © 2018 IBM Corporation 3 Apache TinkerPop • Established in 2009 • Apache incubator in 2015 • Top-level project in 2016 • Open source, vendor-agnostic, graph computing framework • Gremlin graph traversal language Apache TinkerPop™ Maintainer The Apache Software Foundation License Apache Releases 3.3.3 May 2018 https://tinkerpop.apache.org LDBC 11th Technical User
    [Show full text]
  • Practicalgremlin.Pdf
    PRACTICAL GREMLIN An Apache TinkerPop Tutorial Kelvin R. Lawrence Version 283-preview, August 2nd 2021 Table of Contents 1. INTRODUCTION . 1 1.1. How this book came to be. 1 1.2. Providing feedback. 2 1.3. Some words of thanks . 2 1.4. What is this book about?. 2 1.5. Introducing the book sources, sample programs and data . 4 1.6. A word about TinkerPop 3.4 . 5 1.7. Introducing TinkerPop 3.5 . 6 1.8. So what is a graph database and why should I care? . 6 1.9. A word about terminology . 8 2. GETTING STARTED . 10 2.1. What is Apache TinkerPop? . 10 2.2. The Gremlin console . 11 2.2.1. Downloading, installing and launching the console . 11 2.2.2. Saving output from the console to a file . 15 2.3. Introducing TinkerGraph . 16 2.4. Introducing the air-routes graph . 18 2.4.1. Updated versions of the air-route data . 22 2.5. TinkerPop 3 migration notes . 22 2.5.1. Creating a TinkerGraph TP2 vs TP3 . 22 2.5.2. Loading a graphML file TP2 vs TP3 . 22 2.5.3. A word about the TinkerPop.sugar plugin . 23 2.6. Loading the air-routes graph using the Gremlin console . 24 2.7. Turning off some of the Gremlin console’s output. 25 2.8. A word about indexes and schemas. 25 3. WRITING GREMLIN QUERIES . 27 3.1. Introducing Gremlin . 27 3.1.1. A quick look at Gremlin and SQL . 27 3.2. Some fairly basic Gremlin queries .
    [Show full text]