Oracle Spatial and Graph Overview

Oracle Spatial and Graph Graph Overview Bill Beauregard Senior Principal Product Manager

Matt Perry

Principal Member of Technical Staff 2 2 Agenda

• Oracle’s Graph Database Strategy • Introduction to RDF Graph • Use Cases • Feature Overview & Performance • Summary

Support Graph Data Types… …On all enterprise platforms § Oracle Database § Add graph analytics to applications, tools, § Cloudera with Apache Hadoop and information technology platforms § Oracle NoSQL Database § Deliver a scalable, secure, and high performing product § Oracle Big Data Appliance § Simplify development with integrated graph § Oracle Exadata Database analysis, APIs and services Machine § Oracle Cloud

Use Case Graph Model Industry Domain

Spatial Network Data Model § Logistics § Transportation Network • Network path analysis Analysis § Utilities • Multi-model modeling § Telcoms

RDF Data Model Linked Data / § Life Sciences Semantic • Data federation § Finance Mediation • Knowledge representation § Publishing • Master Metadata Mgmt § Public Sector

Property Graph Model § National Intelligence Social Network • Graph Search & Analysis § Public Safety Analysis • Big Data analytics § Social Media search • Entity analytics § Marketing - Sentiment

• Unified content metadata Semantic for federated resources Metadata Layer • Validate semantic and structural consistency

§ Find related content & Text Mining & relations by navigating Entity Analytics connected entities § “Reason” across entities

§ Analyze social relations Social Media using curated metadata Analysis - Blogs, wikis, tweets, video - Calendars, IM, voice

Application 2 Application 3 Graph Metadata Mapping Mid-Tier Server SPARQL § Benefits Application 1 – Existing relational data stays in place Shared Ontologies and corresponding applications do not RDF Graph need to change SQL Inventory Graph Sales Graph – Use of virtual mapping eliminates synchronization issues HR Schema Inventory Schema Sales Schema – Common vocabulary helps with data integration issues HR Database Inventory Database Sales Database Database Server

Access & Presentation Layer

Index

Semantic Graph model (W3C RDF Metadata Model)

Data Servers Event Server Hadoop Appliance Content Mgmt BI Server Data Warehouse

Data Sources / Types Human Machine Social Sourced Subscription Transaction Generated Data Media Information Services Systems

9 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 9 Italian National Institute of Statistics Linked Open Data (LOD) OVERVIEW • Istat is the Italian National Institute of Statistics, producing Official Statistics serving the government, the economy and the public with population and housing data characterizing the economic, demographic, social and environmental situation

CHALLENGES / OPPORTUNITIES RESULTS: • Make Census data broadly available to non-technical users • A “5 star” W3C OpenData model Endpoint service • Enable schema flexibility and simplify data management for a 402,903 Census Sections data warehouse • Open: Linked Open Data system EU/Italian Gov Agency compliance • Complex data production workflow • Scalable: • Multiple data models: DDI, Neuchâtel, SDMX, GSIM SOLUTION • 1.5B Triples: 700M asserted, 885 inferred • Oracle Database 12c (12.1) Enterprise Edition • Ontology: 95 classes & 200 relationship types • Oracle Spatial and Graph 12c RDF Semantic Graph • Interoperable: with Italian federated Linked Open Data • Vocabularies (ontologies) to unify disparate datasets, define Endpoints data provenance, and enable machine driven discovery of • Extensible: Endpoint is a central element of Istat inferred relationships dissemination architecture • Front-end open source access points • Pioneering Linked Open Data in the European Statistical • SPARQL Web Services endpoint for advanced users & System (ESS) machine-to-machine communications • W3C Standards-based: RDF, SPARQL, OWL2 profiles • Faceted/Graph browser for non-technical users 10 • CopyrightGUI © for 2015, ad-hoc Oracle and/or dataset its affiliates. Alldownloads rights reserved. by non-technical users EU Publications Office Linked Metadata Platform for European Union

Objectives Benefits § Common metadata model supports: § Evolving data model that flexibly supports a § Search and discovery of EU Publications variety of business use cases § Multiple domains and languages § Scalability: Solution § Over billion RDF triples in Oracle Graph DB § 2.5 TB of compressed data in Oracle DB § Validate and tag EU law, tenders, and § Links to 3.9 TB (60M) files of EU pubs publicity to standardized vocabularies § Reliability and maintainability § Unified RDF graph metadata model § Oracle ASM (Automatic Storage Management) § Supports discovery of content through user’s terminology and language § Two failover systems § Provides variety of dissemination modes

Objectives “[This technology…] provides improved insight § Unified vocabulary for scientific into our business by bringing together related investigation information from diverse data sources,” § Easier, more complete investigations J. Phil Brooks Information Consultant, Eli Lilly and Company Solution § Integrate patient records, chemical structures, biological sequences & pathways, images, scientific papers… § View related data as a graph § Traverse graphs to discover relationships, search for a term, or browse ontologies

BRIEF ORGANIZATIONAL OVERVIEW NIBR is the global pharmaceutical organization for Novartis committed to discovering innovative medicines to treat diseases with high unmet medical need 6000+ scientists, physicians, business professionals worldwide

BUSINESS CHALLENGES / OPPORTUNITIES Scientific knowledge portal does not provide ability to: • Link database information on genes, proteins, metabolic pathways, compounds, ligands, etc. to original sources. • Increase productivity for accessing, sharing, searching, BUSINESS BENEFITS REALIZED BY ORACLE SOLUTION navigating, cross-linking, analyzing internal /external data • Performance: met design goals for comparable performance SOLUTION to legacy application with enhanced usability Provide a semantic integration layer on existing relational tables: • Better Analysis: enables discovery of unknown relationships based on the meaning (the semantics) of the data • Rich domain-specific terminology (biology, chemistry and medicine) containing 1.6 M terms • Flexible Data Modeling: supports discovery and allows easy changes to incorporate new kinds of data and relationships • Terminology Hub: 8 GB of referential data that cross- references between data repositories. • Manageability: RDF triple store benefits from Oracle Database functionality: backup, security, replication,….

Objectives Benefits § Social connectivity and collaboration § Unifies metadata model - forum, blog, wiki, etc. through semantic enablement § Tagging media documents, pictures, blogs, etc. § Connect knowledge silos to user-defined and/or enterprise vocabularies. Solution § Validates tag semantic/structural consistency § Persistent unified graph metadata model § Concepts tagged with unique meaning § Find related content & groups by navigating connected entities, recommendations

Objectives Benefits § Profile suspects through telephone, email § Standards-based tools: W3C RDF & SPARQL and social network communications § Semantic tagging for 600 TB / 10b triples graph § Produce “data products” for analysts § Top-secret , compartmented security for data Solution § New discovery on ~100 million triples / month § Find & label “same-as” relationships § RDF Graph modeling of the social network: people, groups and places of interest § Inferencing & graph analytics discover relationships among individuals & meaning of pseudonyms, aliases, codes, terminology

Industries

§ Life Sciences § Finance § Media § Networks & Communications § Defense & Intelligence Hutchinson § Public Sector 3G Austria Thomson Reuters

RDF GRAPH DATABASE FEATURES

Transaction Transform & Load, Query Applications & Systems Modeling Tools & Inference Analysis Tools Unstructured R2RML • RDF/OWL Data • Java, HTTP access Content Management Protégé for Oracle • JSON, XML output • SQL & SPARQL Query Jena for Oracle • Graph visualization RSS, email • OWL Inferencing (Cytoscape) Sesame for Oracle NLP Extraction • Semantic Rules • Oracle Advanced Analytics (R, Mining) Other Data (partners) • Scalability & Security Formats • Oracle Business • Semantic Indexing Intelligence (OBIEE) • Map (GIS) Data Visualization Sources Oracle Database 12c

• Native RDF graph data store RDF Semantic Graph Load / • Manages 10s of billions of triples Leverages Oracle Manageability: Storage • Relational to RDF mapping • RAC & Exadata scalability • Compression & partitioning • SPARQL-Jena/Joseki, Sesame Query • SQL/graph query, B-tree indexing • SQL*Loader direct path load • Ontology assisted SQL query • Parallel load, inference, query

• High Availability • RDFS, OWL2 RL, EL+, SKOS • User-defined rules • Triple-level label security Reasoning • Incremental, parallel reasoning • User-defined inferencing • Ladder based inference • Plug-in architecture • Choice of SPARQL, SQL, or Java • Semantic indexing framework • Native inference engine Analytics • OBIEE • Oracle Advanced Analytics • Enterprise Manager • SPARQL Property Paths

19 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. World’s Fastest Big Data Graph Benchmark 1 Trillion Triple RDF Benchmark with Oracle Spatial and Graph Oracle Database 12c can load, query and inference millions of RDF graph edges § World’s fastest data loading per second performance Millions of triples per § World’s fastest query performance second

§ Worlds fastest inference performance 1.42 1.52 2.00 1.13 § Massive scalability: 1.08 trillion edges 1.50 1.00 § Platform: Oracle Exadata X4-2 Database 0.50 Machine 0.00 § Source: w3.org/wiki/LargeTripleStores, 9/26/2014

20 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 2 0 Trillion Triple Benchmark Set-up Open cursors = 1000 Exadata X4-2 High capacity full rack Processes = 1000 ZS3-2 with 2 controllers, 8 trays of disk SGA = 132GB, PGA = 100GB 8 database nodes (total 192 cores) 32K blocksize for tablespaces 14 storage nodes (total 168 cores) A mix of DOP used: 296, 256, 192 Oracle 12.1.0.1 DB standard install of TEMP group created with 3 bigfile tablespaces Exadata * Test performed in Aug/Sept 2014 Degrees of Data set Load OWL Query Parallelism (B triples/hr) Inference (B answers/hr) (B triples/hr) 256* LUBM 605.4B / 475.6+ B / 92.5B / 22.5 hrs 4400K 115.2hrs 86hrs 30m

RDF_Value$ Table (compressed) • Stores the canonical & user value of each S, P and O • Globally unique Hash id for each data element • Reconciles non-differences like numbers 25 & 0025 • Canonical values enable more accurate queries & joins

RDF_Link$ Table • Stores the triples (quads) using the hash Ids in Value$ • Partitioned w/ local indexing to accelerate loading, querying and inferencing

Ingest / Replicate / Tune / Analyze Manage Recover Bulk load: Tune load/ query/ inference: Control query execution: • Apache Jena bulk loader • Parallelism • in database & Jena client • Oracle external tables & • Btree indexing triple/quad Create & monitor graph • SQL*Loader (Direct Path) • Typed literals indexing w/ SQL Developer: w/ PL/SQL Bulk Load API • SPARQL query hints • Semantic Network • Statistics gathering • Models, virtual models • Dynamic Sampling Replicate & recover: • Btree indexes • Data Guard: physical standby Analyze performance: • Rule bases • Data Pump: staging tables • Enterprise Manager: view • Entailments • Recovery Manager: RMAN optimizer plans, monitor • Security data labels execution / resource usage • Semantic index policies

23 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. How Does Semantics Address Finding Patterns in Data? • Has its own graph query language - W3C SPARQL • It’s a simpler way to write query patterns that need to be joined together e.g., Find pairs of siblings (same parents) SPARQL SQL SELECT ?x ?y SELECT g1.subject x, g3.subject y FROM FROM rdf_graph g1, rdf_graph g2, rdf_graph WHERE { g3, rdf_graph g4 ?x hasFather ?f . WHERE g1.predicate = ‘hasFather’ ?x hasMother ?m . è AND g2.predicate = ‘hasMother’ ?y hasFather ?f . ?y hasMother ?m . AND g3.predicate = ‘hasFather’ AND g4.predicate = ‘hasMother’ FILTER( ?x != ?y) } AND g1.subject = g2.subject AND g3.subject = g4.subject AND g1.object = g3.object AND g2.object = g4.object AND g1.subject != g3.subject

• Not at all, Oracle SQL extended to include graph queries • SEM_MATCH table function is based on SPARQL • Uses patented SQL table function rewrite • Converts graph query to SQL, the whole SQL query is optimized • Returns a whole set of results rather than one result per table function call

25 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. SEM_MATCH: Adding SPARQL to SQL Query example: “List everyone’s name and anyone else they know” (Analogous to an outer join) SELECT n1, n2 n1 n2 FROM TABLE( Alex Jerry SEM_MATCH( Alex Tom ’PREFIX foaf: SELECT ?n1 ?n2 Alice Jerry FROM Alice Tom WHERE {?p foaf:name ?n1 OPTIONAL {?p foaf:knows ?f . Alice John ?f foaf:name ?n2 } Jerry Tom } ORDER BY ?n1 ?n2’, Tom Jerry SEM_MODELS(’M1’),…)); Zack -

Standard SPARQL Endpoint HTTP Enhanced with query management control

Java Jena API Jena Adapter

SPARQL-to-SQL Translation Logic

SEM_MATCH SQL rewritable table function

• 40+ new query functions/operators: IF, COALESCE, STRBEFORE, REPLACE, ABS, • Aggregates: COUNT, SUM, MIN, MAX, AVG, GROUP_CONCAT, SAMPLE • Subqueries • Value Assignment: BIND, GROUP BY Expressions, SELECT Expressions • Negation: NOT EXISTS, MINUS

On the fly inference: Social Networking: Transitivity of rdf:subClassOf Find all of John’s friends SELECT ?c SELECT ?name WHERE { WHERE { ?x rdf:type ?sc ?x foaf:name “John” ?sc rdfs:subClassOf * ?c } ?x (foaf:knows | foaff:friendsOf)* ?f ?f foaf:name ?name }

§ Insert triples into an RDF Graph § Delete triples from an RDF Graph § Load an RDF Graph § Clear an RDF Graph § Create a new RDF Graph § Drop an RDF Graph § Copy, move or add the content of one RDF Graph to another § Perform a group of update operations as a single action

W3C Property Paths

§ Answers question, “Yes or No: does a path exist from Tim to Sam?” Ray Mary Works with Manages Works with § Extends matching of a triple pattern to Works with any length path Tim John Sam Works with § A more succinct way to write parts of Manages Works with basic graph patterns Bob Alice

Relational to RDF Modeling

§ W3C Standard Specification § Oracle Spatial and Graph 12c can represent relational schema as graph view § Integrate content from distributed sources § Federate distributed databases § Apply SPARQL queries on tables, views, SQL query results § No duplication of data and storage

Query: “Find all entries in patient diagnosis column related to ‘Upper_Extremity_Fracture’ "

Patients diagnosis table No results w/ the usual SELECT

P_ID DIAGNOSIS SELECT p_id, diagnosis 1 Hand_Fracture FROM Patients 2 Rheumatoid_Arthritis WHERE diagnosis = ’Upper_Extremity_Fracture’; 3 Finger_Fracture

National Cancer Institute (NCI) medical ontology Patients diagnosis table Upper_Extremity_Fracture P_ID DIAGNOSIS SELECT p_idrdfs:subClassOf, diagnosis 1 Hand_Fracture FROM Patients WHERE diagnosisArm_Fracture = 2 Rheumatoid_Arthritis rdfs:subClassOf “Upper_Extremity_Fracturerdfs:subClassOf ”; 3 Finger_Fracture Forearm Elbow Hand_Fracture Fracture Fracture rdfs:subClassOf Finger_Fracture

33 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. How Does Semantics Address Better SQL Results SEM_RELATED SQL operator expands the SQL WHERE clause with related terms from the Ontology

Patients diagnosis table Results: Hand_Fracture, Finger Fracture P_ID DIAGNOSIS SELECT p_id, diagnosis 1 Hand_Fracture FROM Patients 2 Rheumatoid_Arthritis WHERE SEM_RELATED ( 3 Finger_Fracture diagnosis, ’rdfs:subClassOf’, NCI medical ontology: ’:Upper_Extremity_Fracture’, Finger_Fracture è ’Medical_ontology’ = 1) Hand_Fracture è Arm_Fracture è Upper_Extremity_Fracture

• Fine grained security through integration with Oracle Label Security • Oracle Label Security - mandatory access control • Labels assigned to both users and data • Data labels determine the sensitivity of the rows or the rights a person must posses in order to read or write the data. • User labels indicate their access rights to the data records. • Model level security through GRANT/REVOKE privileges

35 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. How Does Semantics Address Discovery: Inferencing • Business Constraint: An applicant can have only 1 line of credit • Finds implicit (unstated) relationships in the data • Allows machine-driven discovery based on W3C standard

Data: User-defined rule: Bank Data: John T Smith if PersonA hasSocialSecNum SS1 hasSocialSecNum ’123’ and PersonB hasSocialSecNum SS1 Credit bureau report: JT Smith then PersonA sameAs PersonB hasSocialSecNum ’123’ Credit bureau report: JT Smith Using OWL Construct: hasHomeEquityLoan ’789’ hasSocialSecNum rdf:type owl:InverseFunctionalProperty Machine adds the inference: Therefore: John T Smith John T Smith sameAs JT Smith hasHomeEquityLoan ’789’ and isn’t qualified for another line of credit

36 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Core Inferencing Features • Forward-chaining based inference engine in the database • Native rulebases: RDFS, OWL 2 RL, OWL 2 EL, SKOS • Validation of inferred data • Proof generation • User defined inferencing - Temporal reasoning, Spatial reasoning • Ladder Based Inference - Fine grained security for inference graph • Integration with external OWL 2 reasoners (e.g., TrOWL, Pellet)

§ W3C Simple Knowledge Organization System (SKOS) – New rulebase supporting the emerging SKOS standard on RDF – Enforces integrity constraints § Dublin Core § SNOWMED § NCI (National Cancer Institute, Gene Ontology_ § FOAF (Friend of a Friend) § GeoRSS § SIOC (Semantically-Interlinked Online Communities) § GoodRelations (eCommerce Product Ontology)

• Support for Apache Jena (Adapter) provides the following features: • A set of easy-to-use and performant Java APIs to access Oracle database • SPARQL 1.1, Named graphs, web service endpoint, REST APIs, JSON output • SPARQL Protocol, Federated SPARQL, SPARQL update • Various data loading (RDF/XML, N-TRIPLES, Turtle, …) with long literal support • Oracle-specific extensions for query execution control and management • Timeout, abort, S2S, hints in SPARQL syntax, property path, result cache, mid-tier cache, user-defined functions… • Integration with open source and commercial tools and reasoners

39 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. Can I Apply Semantics to Text Data - Semantic Indexing On Graph Data On Data Graph Queries Analytical Auto maintained like a Triples table with rowid references B-tree index Subject Property Object graph CREATE INDEX ArticleIndex p:Marcus rdf:type rc::Person <…/r1> ON Newsfeed (Article) INDEXTYPE IS SemContext p:Marcus :hasName “Marcus”^^… <…/r1> PARAMETERS (‘my_policy’) p:Marcus :hasAge “38”^^xsd:.. <…/r1> LOCAL PARALLEL 4 … … … …

Newsfeed table SemContext index on Article column

Rowid docId Article Source SELECT Sem_Contains_Select(1) FROM Newsfeed 1 Indiana authorities filed felony charges and a CNN r1 court issued an arrest warrant for a financial WHERE Sem_Contains (Article, Contentmanager type: who apparently tried to fake his ‘{?x rdf:type rc:Person . • Textdeath by crashing his airplane in a Florida ?x :hasAge ?age . • Fileswamp. (path) Marcus Schrenker, 38 … FILTER(?age >= 35)}’,1)=1 • URL r2 2 Major dealers and investors … NW AND Source = ‘CNN’

Graph Visualization Semantic Modeling

Cytoscape Protégé

• Powerful BI dashboards – Visually appealing – 100% thin client • Across all styles of analysis – R-OLAP, M-OLAP, Scorecards, Reporting, Collaboration, Actions • Across all data sources – Federated data access – Share, collaborate, & publish

Problem Classification Sample Problem Anomaly Detection Given demographic data about a set of customers, identify customer purchasing behavior that is significantly different from the norm

Association Rules Find the items that tend to be purchased together and specify their relationship – market basket analysis

Clustering Segment demographic data into clusters and rank the probability that an individual will belong to a given cluster

Feature Extraction Given demographic data about a set of customers, group the attributes into general characteristics of the customers

F1 F2 F3 F4

• Open source language • Statistical computing and chart for graph data • Produces publication quality plots • Highly extensible with open source R packages

• Oracle brings enterprise-class RDF semantic graph data management • Scalable, Secure, and High Performance: load, query, inference features • Supports W3C Semantic Standards • Works with structured and unstructured data • Enterprise-class Oracle tools can now mine insight from semantic data – OBIEE – Oracle Data Mining – Oracle R Enterprise • Graph DB now available on Oracle NoSQL Database EE

Oracle RDF

RDF/Graph Product Management [email protected] [email protected]

oracle.com

48 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 49 Copyright © 2015, Oracle and/or its affiliates. All rights reserved. 50 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.