Oracle Spatial and Graph: RDF Semantic Graph Feature

1 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. "THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISION. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE."

Part 1: Overview of Graph Part 2: SPARQL and GeoSPARQL Part 3: Semantics (RDF, RDFS, OWL) Part 4: RDF View on Relational Data (RDB2RDF) Part 5: Key Features of Oracle Spatial and Graph Part 6: Performance and Scalability Part 7: Tools Demonstration Part 8: SQLbased Graph Analytics Summary

• What is a graph? – A set of vertexes and edges (and optionally attributes) – A graph is simply linked data • Why do we care? C B – Graphs are everywhere A D • Road networks, power grids, biological networks F • Social networks/Social Web (Facebook, Linkedin, Twitter, Baidu, Google+,) E • Knowledge graphs (RDF, OWL) – Graphs are intuitive and flexible • Easy to navigate, easy to form a path, natural to visualize

• There are many kinds of graphs – Simple graph, Weighted graph, Vertexlabeled graph, Edgelabeled graph, Directed graph (digraph), Undirected graph, Hypergraph, • Different application scenarios – Linknode graphs representing physical/logical networks used in transportation, utilities and telco (Oracle Spatial and Graph Network Data Model (NDM) – RDF Semantic Graphs modeling data as triples for social network, linked data and other semantic applications (Oracle Spatial and Graph) – Property Graphs allowing the association of K/V pairs (attributes) with vertexes/edges for social network analytics (investigating)

• Resource Description Framework – URIs are used to identify • Resources, entities, relationships, concepts • Creates SubjectPropertyObject “triples” • Data identification is a must for integration • URIs are globally unique • Properties of subjects are triples • RDF Graph defines semantics • Standards defined by W3C & OGC – RDF, RDFS, OWL, SKOS – SPARQL, RDFa, RDB2RDF, GeoSPARQL • Implementations – Oracle, IBM, Cray, Bigdata ® – Franz, Ontotext, Openlink, Jena, Sesame,

• A set of vertices (or nodes) – each vertex has a unique identifier (not globally unique). – each vertex has a set of in/out edges. – each vertex has a collection of keyvalue properties. • A set of edges – each edge has a unique identifier (not globally unique). – each edge has a head/tail vertex. – each edge has a label denoting type of relationship between two vertices. – each edge has a collection of keyvalue properties. • Blueprints Java APIs open source, no standards • Implementations • Neo4j, InfiniteGraph, Dex, Sail, MongoDB • A property graph can be modeled as an RDF Graph (Oracle Spatial and Graph NDM is an example of a property graph optimized for consistent/homogeneous properties for edges)

8 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. https://github.com/tinkerpop/blueprints/wiki/PropertyGraphModel Introduction to RDF Semantic Graph, a feature of Oracle Spatial and Graph

• Core Technologies • URI • Uniform resource identifier • RDF • Resource description framework • RDFS • RDF Schema • OWL • Web ontology language

10 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. http://www.w3.org/2007/03/layerCake.svg What is RDF A graph data model for web resources and their relationships http://www.foobar.com http:///locatedIn The graph can be serialized into “CA” RDF/XML, N3, NTRIPLE, http:///produce

Construction unit: Triple http://www.foobar.com/products/mp3 (or assertion, or fact) http:///customerOf <:produces> <:mp3> http://www.oracle.com Subject Predicate Object (vertex /node) (property/edge) (vertex/node) http:///uses http:///produce Quads (named graphs) add context, provenance, identification, etc. to http://www.oracle.com/products/RDF assertions <:produces> <:mp3 > <:ProductGraph>

• Instances E.g. :John, :MovieXYZ, :PurchaseOrder432 • Classes • Class represents a group/category/categorization of instances E.g. :John rdf:type :Student • Properties • Linking data together E.g. :John :brother :Mary, :John :hasAge “33”^^xsd:integer.

• Graph modeling is flexible • Adding/removing a new edge or vertex is simple • Adding an edge is like adding a new column to a table but easier to do • Standard based graph representation • RDF was defined by W3C. Allows interoperability • Computers can understand the semantics RDF graphs (triples) • Same URI means same resource http://www.oracle.com/products/RDF http://www.foobar.com :locatedIn :customerOf :produce “CA” :produce http://www.oracle.com http://www.oracle.com http://www.foobar.com/products/mp3

• Graph modeling is flexible • Adding/removing a new edge or vertex is simple • Adding an edge is like adding a new column to a table but easier to do • Standard based graph representation • RDF was defined by W3C. Allows interoperability • Computers can understand the semantics RDF graphs (triples) • Discover hidden relationships, or detect inconsistency via logical inference http://www.oracle.com/products/RDF http://www.foobar.com :locatedIn :customerOf rdf:type :produce “CA” :produce http://www.oracle.com http://www.foobar.com/products/mp3 :customerOf rdfs:range :ServiceProvider

• Graph modeling is flexible • Adding/removing a new edge or vertex is simple • Adding an edge is like adding a new column to a table but easier to do • Standard based graph representation • RDF was defined by W3C. Allows interoperability • Computers can understand the semantics RDF graphs (triples) • Discover hidden relationships, or detect inconsistency via logical inference http://www.foobar.com :hasOracleCSI “CID1234” :locatedIn :customerOf :hasOracleCSI rdf:type “CID987” ! “CA” :produce http://www.oracle.com http://www.foobar.com/products/mp3 :hasOracleCSI rdf:type owl:FunctionalProperty

• DBPedia • Wordnet • Semanitc XBRL • SIOC • Drug Bank • US Census • NCI • ACM • YAGO • SNOMED • Daily Med • Cyc/Open Cyc • FOAF • Linked CT • PubMed • Geonames • Eurostat • Freebase • CIA World Fact Book • KEGG • Gene Ontology • DBLP • Data.gov.uk • UniRef • UniProt • Music Brainz Data • Smart Link • UniParc • Semantic Tweet • Reactome • CiteSeer • CO2 Emission • Diseasome And so much more !

• Flexible graph modeling adds agility • Resource identification adds precision • Integrate full breadth of enterprise content (structured, spatial, email, documents, web services) • Reconcile differences in data semantics so that they can all “talk” and interoperate; • Resolve semantic discrepancies across databases, applications • Create consolidated “single” views across business applications • Model and implement common Business Processes

Oracle Spatial and Graph works well with these important enterprise technologies • Relational, XML, Spatial, Text, Security, Clustering, Compression, Data Guard – Oracle’s RDF/OWL support is native to the Database • Web Services, SOA, BPMN, Hadoop (Map Reduce) – Support of popular Java APIs and standard complaint Web Service endpoint • Advanced Analytics – Support integration with OBIEE, Oracle Data Mining, Oracle R Enterprise • A rich set of third party tools including – Ontology editing, knowledge management, Complete DL reasoners – Graph/network visualization – NLP, text processing –

• Unified content metadata Semantic for federated resources MetadataMetadata Layer • Validate semantic and structural consistency

Find related content & Text Mining & relations by navigating EntityEntityAnalyticsAnalytics connected entities “Reason” across entities

Analyze social relations Social Media using curated metadata AnalysisAnalysis Blogs, wikis, tweets, video Calendars, IM, voice

Industries

• Life Sciences • Finance • Media • Networks & Communications • Defense & Intelligence • Public Sector Hutchinson 3G Austria Thomson Reuters

Objectives Benefits Profile suspects through telephone, email Standardsbased tools: W3C RDF & SPARQL and social network communications Semantic tagging for 600 TB / 10b triples graph Produce “data products” for analysts Topsecret , compartmented security for data Solution New discovery on ~100 million triples / month Find & label “sameas” relationships RDF Graph modeling of the social network: people, groups and places of interest Inferencing & graph analytics discover relationships among individuals & meaning of pseudonyms, aliases, codes, terminology

Objectives Benefits Social connectivity and collaboration Unifies metadata model forum, blog, wiki, etc. through semantic enablement Tagging media documents, pictures, blogs, etc. Connect knowledge silos to userdefined and/or enterprise vocabularies. Solution Validates tag semantic/structural consistency Persistent unified graph metadata model Concepts tagged with unique meaning Find related content & groups by navigating connected entities, recommendations

Objectives “[This technology] provides improved insight Unified vocabulary for scientific into our business by bringing together related investigation information from diverse data sources,” Easier, more complete investigations J. Phil Brooks Information Consultant, Eli Lilly and Company Solution Integrate patient records, chemical structures, biological sequences & pathways, images, scientific papers View related data as a graph Traverse graphs to discover relationships, search for a term, or browse ontologies

25 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Oracle Spatial and Graph Partners: Integrated Tools and Solution Providers Ontology Engineering & Visualization Reasoners NLP Entity Extractors

Open Source Frameworks Standards

Joseki Sesame Applications & Tools SI / Consulting

SPARQL Protocol and RDF Query Language – W3C standard for querying and manipulating RDF content – Queries/updates and corresponding results are communicated via HTTP with a SPARQL endpoint – A SPARQL endpoint implements the SPARQL protocol and serves RDF data from a RDF triplestore or RDF view

Query Language Update Protocol Service Description Query Results JSON Format Query Results CSV and TSV Format Query Results XML Format Federated Query Entailment Regimes Graph Store HTTP Protocol

Query Language A comprehensive query language Update for RDF Protocol Many useful constructs: optional Service Description patterns, aggregates, subqueries, Query Results JSON Format negation, property paths, extensive Query Results CSV and TSV Format function library, etc. Query Results XML Format Federated Query Entailment Regimes Graph Store HTTP Protocol

Query Language A comprehensive language for Update manipulating RDF graphs Protocol Allows you to create, update and Service Description remove RDF graphs Query Results JSON Format Query Results CSV and TSV Format Query Results XML Format Federated Query Entailment Regimes Graph Store HTTP Protocol

Query Language Defines a protocol for sending Update queries or updates to SPARQL endpoint and returning the results Protocol via HTTP Service Description Query Results JSON Format Query Results CSV and TSV Format Query Results XML Format Federated Query Entailment Regimes Graph Store HTTP Protocol

Query Language Defines a mechanism and RDF Update vocabulary for describing the features supported by a SPARQL Protocol endpoint Service Description Query Results JSON Format Query Results CSV and TSV Format Query Results XML Format Federated Query Entailment Regimes Graph Store HTTP Protocol

Query Language Alternative formats used to Update serialize and exchange answers to SPARQL queries Protocol Service Description Query Results JSON Format Query Results CSV and TSV Format Query Results XML Format Federated Query Entailment Regimes Graph Store HTTP Protocol

Query Language SPARQL extension for executing Update queries distributed over different SPARQL endpoints Protocol Service Description Query Results JSON Format Query Results CSV and TSV Format Query Results XML Format Federated Query Entailment Regimes Graph Store HTTP Protocol

Query Language Extends SPARQL so that logically Update entailed RDF triples (hidden edges in RDF Graphs) are matched in Protocol addition to directly asserted RDF Service Description triples Query Results JSON Format Query Results CSV and TSV Format Query Results XML Format Federated Query Entailment Regimes Graph Store HTTP Protocol

Query Language Simple alternative to SPARQL 1.1 Update Update that describes HTTP operations for managing a Protocol collection of RDF graphs outside of Service Description a SPARQL 1.1 graph store Query Results JSON Format Query Results CSV and TSV Format Query Results XML Format Federated Query Entailment Regimes Graph Store HTTP Protocol

Basic unit of SPARQL queries rdf:type rdf:type govtrack:Politician?t govtrack:A000041 rdf:type govtrack:A000045 foaf:name foaf:gender

foaf:name foaf:gender govtrack:A000041?p “John Adams” “male”

foaf:name foaf:gender vcard:BDAY “male” “Samuel Adams” “John?n Adams” “male”?g vcard:BDAY “17670711”^^xsd:date vcard:BDAY “17220927”^^xsd:date “17670711”^^xsd:date?b

Result 1: {?t=govtrack:Politician, ?p=govtrack:A000041, ?n=“John Adams”, ?g=“male”, ?b=“17670711”^^xsd:date} Result 2: {?t=govtrack:Politician, ?p=govtrack:A000045, ?n=“Samuel Adams”,?g=“male”, ?b=“17220927”^^xsd:date}

?t How do we express this with SPARQL? rdf:type PREFIX rdf: PREFIX foaf: ?p PREFIX vcard: foaf:name foaf:gender SELECT ?t ?n ?b ?g

?n ?g WHERE { ?p rdf:type ?t . vcard:BDAY ?p foaf:name ?n . Basic Graph ?p vcard:BDAY ?b . ?b Pattern (BGP) ?p foaf:gender ?g }

Find all distinct family names for senators PREFIX rdf: PREFIX foaf: PREFIX vcard: PREFIX xsd: SELECT DISTINCT ?f WHERE { ?p vcard:N ?vn . ?vn vcard:Family ?f . ?p foaf:title "Sen." . }

Find all people with family name Adams born before Independence Day PREFIX rdf: PREFIX foaf: PREFIX vcard: PREFIX xsd: SELECT ?n ?b ?g WHERE { ?p vcard:N ?vn . ?vn vcard:Family ?f . ?p foaf:name ?n . ?p vcard:BDAY ?b . ?p foaf:gender ?g FILTER ( ?f = "Adams" && ?b < "1776-07-04"^^xsd:date )}

Basic: arithmetic, comparisons, booleanconnectors RDFrelated: isLiteral(), isURI(), isBlank(), datatype(), lang(), BOUND(), String Functions: SUBSTR(), STRSTARTS(), STRENDS(), REGEX(), Numerics: abs(), floor(), ceil(), Dates and Times: now(), year(), month(), day(), Miscellaneous: IN(), NOT IN(), IF(), COALESCE(), Constructors: xsd:int(), xsd:decimal(), xsd:dateTime(), plus userdefined

Find vcard given name or foaf name for all Clintons SELECT * WHERE { ?p vcard:N ?vn . ?p vcard:BDAY ?b . ?vn vcard:Family ?f . { ?p foaf:name ?n } UNION { ?vn vcard:Given ?n } FILTER ( ?f = "Clinton" ) }

Find all people with family name Smith and optionally their title and homepage SELECT ?t ?n ?b ?h WHERE { ?p vcard:N ?vn . ?vn vcard:Family ?f . ?p foaf:name ?n . ?p vcard:BDAY ?b . OPTIONAL { ?p foaf:title ?t } OPTIONAL { ?p foaf:homepage ?h } FILTER ( ?f = "Smith" )}

Find all representatives and optionally their homepage if it is not a standard www.house.gov address SELECT ?n ?b ?h WHERE { ?p foaf:name ?n . ?p vcard:BDAY ?b . ?p foaf:title "Rep." OPTIONAL { ?p foaf:homepage ?h FILTER (!STRSTARTS(STR(?h),"http://www.house.gov")) } }

Find all Kennedys that do not have a homepage SELECT ?n ?b WHERE { ?p vcard:N ?vn . ?vn vcard:Family "Kennedy" . ?p foaf:name ?n . ?p vcard:BDAY ?b . MINUS { ?p foaf:homepage ?h } }

Find all Kennedys that do not have a homepage SELECT ?n ?b WHERE { ?p vcard:N ?vn . ?vn vcard:Family "Kennedy" . ?p foaf:name ?n . ?p vcard:BDAY ?b . FILTER ( NOT EXISTS { ?p foaf:homepage ?h } ) }

Order all representatives by ascending family name and descending homepage SELECT ?gn ?fn ?h WHERE { ?p vcard:N ?v . ?v vcard:Given ?gn . ?v vcard:Family ?fn . ?p foaf:title "Rep." . ?p foaf:homepage ?h } ORDER BY ASC(?fn) DESC(STR(?h))

Get information about representatives 11 through 20 in alphabetical order SELECT ?gn ?fn ?h WHERE { ?p vcard:N ?v . ?v vcard:Given ?gn . ?v vcard:Family ?fn . ?p foaf:title "Rep." . ?p foaf:homepage ?h } ORDER BY ASC(?fn) LIMIT 10 OFFSET 10

Generate a full title “ <FI> <family name>” for each member of congress SELECT (CONCAT(?t," ",SUBSTR(?gn,1,1),". ",?fn) AS ?fullTitle) WHERE { ?p vcard:N ?v . ?v vcard:Given ?gn . ?v vcard:Family ?fn . ?p foaf:title ?t . }51 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Grouping and Aggregation Find all distinct pairs of family name and title SELECT ?fn ?t WHERE { ?p vcard:N ?v . ?v vcard:Family ?fn . ?p foaf:title ?t . } GROUP BY ?fn ?t52 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Grouping and Aggregation Find the top 10 bill sponsors SELECT ?n (COUNT(*) AS ?cnt) WHERE { ?s foaf:name ?n . ?b bill:sponsor ?s . } GROUP BY ?n ORDER BY DESC(?cnt) LIMIT 10Available Aggregates: COUNT(), SUM(), MIN(), MAX(), AVG(), GROUP_CONCAT(), SAMPLE()53 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Grouping and Aggregation Find members of congress who have sponsored very few bills SELECT ?n (COUNT(*) AS ?cnt) WHERE { ?s foaf:name ?n . ?b bill:sponsor ?s . } GROUP BY ?n HAVING (COUNT(?b) < 5) ORDER BY ?cnt ?n54 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 SubqueriesFind information about politicians who sponsored more than 150 bills SELECT DISTINCT ?n ?o ?p ?st ?cnt WHERE { ?s foaf:name ?n . ?s pol:hasRole ?r . ?r pol:party ?p . ?r pol:forOffice ?o . ?o pol:represents ?st { SELECT ?s (COUNT(?b) AS ?cnt) WHERE { ?b bill:sponsor ?s } GROUP BY ?s HAVING (COUNT(?b) > 150) } } ORDER BY DESC(?cnt) ASC(?n)55 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Value Assignment: BINDFind information about political offices of John McCain SELECT ?n ?o ?p ?st WHERE { BIND (CONCAT("John"," ","McCain") AS ?n) ?s foaf:name ?n . ?s pol:hasRole ?r . ?r pol:party ?p . ?r pol:forOffice ?o . ?o pol:represents ?st }56 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Inline Data: VALUESFind information for all Browns that are Republican and Smiths that are Democrat SELECT ?gn ?fn ?o ?p ?st WHERE { ?s vcard:N ?n . ?n vcard:Family ?fn . ?n vcard:Given ?gn . ?s pol:hasRole ?r . ?r pol:party ?p . ?r pol:forOffice ?o . ?o pol:represents ?st VALUES (?fn ?p) {("Brown" "Republican") ("Smith" "Democrat") } }57 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Federated QueryWhat information about NH senators can be found from DBPedia SELECT ?n ?p ?o WHERE { ?person foaf:name ?n . ?person pol:hasRole ?role . ?role pol:forOffice ?office . ?office pol:represents geo:nh SERVICE <http://dbpedia.org/sparql> { ?x foaf:name ?n . ?x ?p ?o } }58 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL ASK QueriesIs there a Republican Kennedy? ASK WHERE { ?person vcard:N ?n . ?n vcard:Given "Kennedy". ?person pol:hasRole ?role . ?role pol:party "Republican" . }59 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL Construct QueriesBuild a graph of names and political parties for all senators CONSTRUCT { ?person foaf:name ?n . ?person pol:memberOf ?party } WHERE { ?person foaf:name ?n . ?person pol:hasRole ?role . ?role pol:party ?party . ?person foaf:title "Sen." . }60 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL Describe QueriesDescribe all politicians with family name “Paul” DESCRIBE ?person WHERE { ?person vcard:N ?n . ?n vcard:Family "Paul" . }61 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Property Paths62 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Property Paths Enhanced path searching in SPARQL� Uses regular expression style syntax to express path patterns over RDF properties � Allows syntactic shortcuts for fixed length paths � Allows searching arbitrary length paths – Uses connectivity semantics instead of path counting semantics63 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Property Path ConstructsSyntax Form Matches iri An IRI (path of length 1) êlt Reverse path (object to subject) elt1 / elt2 Sequence path of elt1 followed by elt2 elt1 | elt2 Alternative path of elt1 or elt2 elt* Path composed of zero or more repetitions of elt elt+ Path composed of one or more repetitions of elt elt? Path composed of zero or one repetition of elt!iri or !(iri1|iri2|�|irin) A path of length 1 that is not one of irii!îri or !(îri1|îri2|�|îrin) A path of length 1 that is not one of irii as reverse paths!(iri1|�|irij|îrij+1|�|îrin) A path of length 1 that is not one of irii in the indicated direction (elt) Grouping used to control precedence iri is an IRI elt is a path element, which may itself be composed of other path constructs64 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Property PathFind each politician’s classification and the district he or she represents SELECT ?n ?dist ?sc WHERE { ?person foaf:name ?n . ?person pol:hasRole/pol:forOffice/pol:represents ?dist . ?person rdf:type ?t . ?t rdfs:subClassOf+ ?sc . } LIMIT 10065 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Property Path Limitations� For arbitrary length paths – Cannot return actual path itself – Cannot get length of found path – Difficult to place conditions on nodes along the path � Length�limited path search is hard to express for longer lengths � No shortest path function66 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL Named Graphs67 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL Named Graphs The concept of an RDF Dataset� An RDF Dataset is a collection of RDF graphs – Contains one default graph, which does not have a name – Contains zero or more named graphs, where each graph is identified by an IRI � A SPARQL query is executed against an RDF Dataset � FROM and FROM NAMED keywords are used to construct the RDF Dataset for a query � The GRAPH keyword is used to control the active graph for different parts of a query68 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Constructing the RDF DatasetContents of RDF Triplestore SPARQL query with RDF Graph Name Triples Dataset specification Default Graph �� {t1,t2,t3} SELECT * { t4, t5, t8, t9 } FROM <urn:g1> <urn:g1> {t4,t5} FROM <urn:g3> <urn:g2> {t6,t7} FROM NAMED <urn:g2> Named Graphs FROM NAMED <urn:g3> { (<urn:g2>, { t6, t7 }), <urn:g3> {t8,t9} FROM NAMED <urn:g4> (<urn:g3>, { t8, t9 }), <urn:g4> {t10,t11} WHERE { … } (<urn:g4>, { t10, t11 }) }69 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Using the GRAPH Keyword SPARQL query with RDF Dataset specification Within a GRAPH SELECT * Active Graph (BGP1) clause: FROM <urn:g1> { <urn:g1> UNION <urn:g3> } FROM <urn:g3> � BGP is executed FROM NAMED <urn:g2> Active Graph (BGP2) against each active FROM NAMED <urn:g3> { <urn:g2>, <urn:g3>, <urn:g4> } graph separately FROM NAMED <urn:g4> (e.g. BGP2 against WHERE { Active Graph (BGP3) g2, g3, g4). BGP1 {<urn:g4> } GRAPH ?g { BGP2 } � Subgraph match must occur within a GRAPH <urn:g4> { BGP3 } Active Graph (BGP4) single graph. GRAPH <urn:g1> { BGP4 } { } }70 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL Named Graph QueryFind the number of bills sponsored by each politician in the 110th and 111th congress SELECT ?n ?g (count(?b) as ?bcnt) FROM usgov:people FROM NAMED usgov:bills_110 FROM NAMED usgov:bills_111 WHERE { ?s foaf:name ?n GRAPH ?g { ?b bill:sponsor ?s } } GROUP BY ?n ?g ORDER BY ?n ?g71 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update72 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update Capabilities of SPARQL Update� Insert triples into an RDF Graph � Delete triples from an RDF Graph � Load an RDF Graph � Clear an RDF Graph � Create a new RDF Graph � Drop an RDF Graph � Copy, move or add the content of one RDF Graph to another � Perform a group of update operations as a single action73 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: INSERT DATAInsert simple triple data PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX my: <http://www.mydomain.com/> INSERT DATA { my:person1 foaf:name "John Smith" . my:person1 foaf:knows my:person1 }Insert simple named graph data PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX my: <http://www.mydomain.com/> INSERT DATA { GRAPH my:g1 { my:person1 foaf:name "John Smith" . my:person1 foaf:knows my:person1 } }74 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: DELETE DATADelete triple data PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX my: <http://www.mydomain.com/> DELETE DATA { my:person1 foaf:name "John Smith" . my:person1 foaf:knows my:person1 }Delete named graph data PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX my: <http://www.mydomain.com/> DELETE DATA { GRAPH my:g1 { my:person1 foaf:name "John Smith" . my:person1 foaf:knows my:person1 } }75 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: DELETE INSERTDelete all foaf:worksFor “Oracle USA” triples PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX my: <http://www.mydomain.com/> DELETE { ?p foaf:worksFor "Oracle USA, Inc." } WHERE { ?p foaf:worksFor "Oracle USA, Inc." }Insert foaf:worksFor “Oracle America” triples for all “Oracle USA” employees PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX my: <http://www.mydomain.com/> INSERT { ?p foaf:worksFor "Oracle America" } WHERE { ?p foaf:worksFor "Oracle USA" }76 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: DELETE INSERTReplace all foaf:worksFor “Oracle USA” triples with foaf:worksFor “Oracle America” triples PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX my: <http://www.mydomain.com/> DELETE { ?p foaf:worksFor "Oracle USA" } INSERT { ?p foaf:worksFor "Oracle America" } WHERE { ?p foaf:worksFor "Oracle USA" }77 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: LOADLoad graph1 into my:g1 PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX my: <http://www.mydomain.com/> LOAD <http://www.graphs.com/graph1> INTO GRAPH my:g1Load graph1 into default graph PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX my: <http://www.mydomain.com/> LOAD <http://www.graphs.com/graph1>78 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: CLEARClear my:g1 PREFIX my: <http://www.mydomain.com/> CLEAR GRAPH my:g1Clear all named graphs PREFIX my: <http://www.mydomain.com/> CLEAR NAMED79 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: CLEARClear default graph PREFIX my: <http://www.mydomain.com/> CLEAR DEFAULTClear all named graphs and default graph PREFIX my: <http://www.mydomain.com/> CLEAR ALL80 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: CREATECreate graph my:g1 PREFIX my: <http://www.mydomain.com/> CREATE GRAPH my:g181 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: DROPDrop my:g1 PREFIX my: <http://www.mydomain.com/> DROP GRAPH my:g1Drop all named graphs PREFIX my: <http://www.mydomain.com/> DROP NAMED82 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: DROPDrop default graph PREFIX my: <http://www.mydomain.com/> DROP DEFAULTDrop all named graphs and default graph PREFIX my: <http://www.mydomain.com/> DROP ALL83 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: COPYReplace contents of graph g2 with contents of graph g1 PREFIX my: <http://www.mydomain.com/> COPY GRAPH my:g1 TO GRAPH my:g2Replace contents of graph g2 with contents of default graph PREFIX my: <http://www.mydomain.com/> COPY DEFAULT TO GRAPH my:g284 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: MOVEReplace contents of graph g2 with contents of graph g1 and drop graph g1 PREFIX my: <http://www.mydomain.com/> MOVE GRAPH my:g1 TO GRAPH my:g2Replace contents of default graph with contents of graph g2 and drop graph g2 PREFIX my: <http://www.mydomain.com/> MOVE my:g2 TO DEFAULT85 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. SPARQL 1.1 Update: ADDAppend contents of graph g1 to graph g2 PREFIX my: <http://www.mydomain.com/> ADD GRAPH my:g1 TO GRAPH my:g2Append contents of graph g2 to default graph PREFIX my: <http://www.mydomain.com/> ADD my:g2 TO DEFAULT86 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Summary Components of SPARQL 1.1� Query Language � Update � Protocol � Service Description � Query Results JSON Format � Query Results CSV and TSV Format � Query Results XML Format � Federated Query � Entailment Regimes � Graph Store HTTP Protocol87 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Open Geospatial Consortium (OGC) GeoSPARQL Specification88 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Linked Geo Data� Many Linked Open Data (LOD) datasets have geospatial components � Barriers to integration – Vendor�specific geometry support – Different vocabularies � W3C Basic Geo, GML XMLLiteral, Vendor�specific – Different spatial reference systems � WGS84 Lat�Long, British National Grid89 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Semantic GIS� GIS applications with semantically complex thematic aspects – Logical reasoning to classify features � Land cover type, suitable farm land, etc. – Complex Geometries � Polygons and Multi�Polygons with 1000’s of points – Complex Spatial Operations Find parcels with an area of at least 3 sq. � Union, Intersection, Buffers, etc. miles that touch a local feeder road and are inside an area of suitable farm land.90 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Requirements for GeoSPARQL� Provide a common target for implementers & users – Representation and query � Work within SPARQL’s extensibility framework � Simple enough for general users – Keep the common case simple (WGS 84 point data) � Capable enough for GIS professionals – Multiple SRS’s, complex geometries, complex operators � Don’t re�invent the wheel! <a href="/tags/Simple_Features/" rel="tag">Simple Features</a> Well Known Text (WKT) GML ISO 19107 – Spatial Schema KML ISO 13249 – SQL/MM GeoJSON91 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Standardization TimelineOAB vote on Process candidate comments and Standard standard Published Form SWG update document (June 2011) (June 2012) (June 2010) (Feb. 2012)1 2 3 4 5 6 7Release 30�day public candidate TC/PC vote comment period (March 2012) standard (July 2011) (May 2011)92 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 From SPARQL to GeoSPARQL93 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SPARQL QueryRDF Data SPARQL Query :res1 rdf:type :House . SELECT ?r ?ba ?br :res1 :baths "2.5"^^xsd:decimal . WHERE { ?r rdf:type :House . :res1 :bedrooms "3"^^xsd:decimal . ?r :baths ?ba . ?r :bedrooms ?br } :res2 rdf:type :Condo . :res2 :baths "2"^^xsd:decimal . :res2 :bedrooms "2"^^xsd:decimal . Result Bindings :res3 rdf:type :House ?r | ?ba | ?br :res3 :baths "1.5"^^xsd:decimal . ======:res3 :bedrooms "3"^^xsd:decimal . :res1 | "2.5" | "3" :res3 | "1.5" | "3"94 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SPARQL QueryRDF Data SPARQL Query :res1 rdf:type :House . SELECT ?r ?ba ?br :res1 :baths "2.5"^^xsd:decimal . WHERE { ?r rdf:type :House . :res1 :bedrooms "3"^^xsd:decimal . ?r :baths ?ba . ?r :bedrooms ?br :res2 rdf:type :Condo . FILTER (?ba > 2) } :res2 :baths "2"^^xsd:decimal . :res2 :bedrooms "2"^^xsd:decimal . Result Bindings :res3 rdf:type :House ?r | ?ba | ?br :res3 :baths "1.5"^^xsd:decimal . ======:res3 :bedrooms "3"^^xsd:decimal . :res1 | "2.5" | "3"95 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Spatial SPARQL QUERY Spatial RDF Data :res1 rdf:type :House . This is what GeoSPARQL :res1 :baths "2.5"^^xsd:decimal . standardizes :res1 :bedrooms "3"^^xsd:decimal . :res1 ogc:hasGeometry :geom1 . :geom1 ogc:asWKT "POINT(-122.25 37.46)"^ôgc:wktLiteral .:res3 rdf:type :House Vocabulary & :res3 :baths "1.5"^^xsd:decimal . Datatypes :res3 :bedrooms "3"^^xsd:decimal . :res3 ogc:hasGeometry :geom3 . :geom3 ogc:asWKT "POINT(-122.24 37.47)"^ôgc:wktLiteral .Find houses GeoSPARQL Query Extension within a SELECT ?r ?ba ?br Functions search WHERE { ?r rdf:type :House . ?r :baths ?ba . ?r :bedrooms ?br . polygon ?r ogc:hasGeometry ?g . ?g ogc:asWKT ?wkt FILTER(ogcf:sfWithin(?wkt, "POLYGON(…)"^ôgc:wktLiteral)) }96 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 GeoSPARQL Vocabulary: Basic Classes and Relations ogc:SpatialObjectSame as ISO GM_Object ogc:Feature ogc:hasGeometry 0 .. * ogc:Geometry metadata 0 .. 1 ogc:dimension : xsd:int ogc:hasDefaultGeometry ogc:coordinateDimension : xsd:int Same as ISO ogc:spatialDimension : xsd:int GFI_Feature ogc:isEmpty : xsd:boolean ogc:isSimple : xsd:boolean serializations Geometry encoded ogc:asWKT : ogc:wktLiteral as a Literal ogc:asGML : ogc:gmlLiteral � 97 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Details of ogc:wktLiteralAll RDFS Literals of type ogc:wktLiteral shall consist of an optional IRI identifying the spatial reference system followed by Simple Features Well Known Text (WKT) describing a geometric value [ISO 19125�1]."<http://www.opengis.net/def/crs/OGC/1.3/CRS84> POINT(-122.4192 37.7793)"^ôgc:wktLiteral WGS84 longitude – latitude is the default CRS"POINT(-122.4192 37.7793)"^ôgc:wktLiteralEuropean Petroleum Survey Group (EPSG) maintains a set of CRS identifiers.98 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Topological Relations between ogc:SpatialObjectB A/B A B A B A ogc:sfEquals ogc:sfTouches ogc:sfOverlaps ogc:sfContains A A B A B A B B ogc:sfWithin ogc:sfDisjoint ogc:sfIntersects ogc:sfCrosses• Assumes Simple Features Relation Family • Also support Egenhofer and RCC899 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Example Data:City rdfs:subClassOf ogc:Feature . Meta Information :Park rdfs:subClassOf ogc:Feature . :exactGeometry rdfs:subPropertyOf ogc:hasGeometry .Non�spatial Properties :SanFrancisco rdf:type :City . :UnionSquarePark rdf:type :Park . :UnionSquarePark :commissioned "1847-01-01"^^xsd:date .Spatial Properties :UnionSquarePark :exactGeometry :geo1 . :geo1 ogc:asWKT "Polygon((…))"^ôgc:wktLiteral .:SanFrancisco :exactGeometry :geo2 . :geo2 ogc:asWKT "Polygon((…))"^ôgc:wktLiteral .:UnionSquarePark ogc:sfWithin :SanFrancisco .100 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 GeoSPARQL Query Functions– ogcf:distance(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral, geom1 geom2 units: xsd:anyURI): xsd:double– ogcf:buffer(geom: ogc:wktLiteral, radius: xsd:double, geom units: xsd:anyURI): ogc:wktLiteral– ogcf:convexHull(geom: ogc:wktLiteral): ogc:wktLiteral geom101 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 GeoSPARQL Query Functions– ogcf:intersection(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): ogc:wktLiteral geom2 geom1 – ogcf:union(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): ogc:wktLiteral geom2 geom1 102 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 GeoSPARQL Query Functions– ogcf:difference(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): ogc:wktLiteral geom2 geom1 – ogcf:symDifference(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): ogc:wktLiteral geom2 geom1 103 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 GeoSPARQL Query Functions– ogcf:envelope(geom: ogc:wktLiteral): ogc:wktLiteral geom– ogcf:boundary(geom1: ogc:wktLiteral): ogc:wktLiteral geom– ogcf:getSRID(geom: ogc:wktLiteral): xsd:anyURI104 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 GeoSPARQL Topological Query Functions– ogcf:sfEquals(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): xsd:boolean– ogcf:sfDisjoint(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): xsd:boolean– ogcf:sfIntersects(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): xsd:boolean– ogcf:sfTouches(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): xsd:boolean– ogcf:sfCrosses(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): xsd:boolean– ogcf:sfWithin(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): xsd:boolean– ogcf:sfContains(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): xsd:boolean– ogcf:sfOverlaps(geom1: ogc:wktLiteral, geom2: ogc:wktLiteral): xsd:booleanAssumes Simple Features Relation Family105 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Example Query Find all land parcels that are within the intersection of :City1 and :District1 PREFIX : <http://my.com/appSchema#> PREFIX ogc: <http://www.opengis.net/ont/geosparql#> PREFIX ogcf: <http://www.opengis.net/def/geosparql/functions/> PREFIX epsg: <http://www.opengis.net/def/crs/EPSG/0/>SELECT ?parcel WHERE { ?parcel rdf:type :Residential . ?parcel :exactGeometry ?pGeo . ?pGeo ogc:asWKT ?pWKT. :District1 :exactGeometry ?dGeo . ?dGeo ogc:asWKT ?dWKT. :City1 :extent ?cGeo . ?cGeo ogc:asWKT ?cWKT. FILTER(ogcf:sfWithin(?pWKT, ogcf:intersection(?dWKT,?cWKT)))}106 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Summary� GeoSPARQL Defines: – Basic vocabulary, Query functions, Entailment component � Based on existing OGC/ISO standards – WKT, GML, Simple Features, ISO 19107 � Uses SPARQL’s built�in extensibility framework � Modular specification – Allows flexibility in implementations – Easy to extend107 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Part 3: Semantics10 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8 RDF Schema (RDFS) • Core language constructs • rdfs:subClassOf:A rdfs:subClassOf :B � instance of A is also instance of B Derives implicit • rdfs:subPropertyOf (property transfer) relationships using inference :p1 rdfs:subPropertyOf :p2, :a :p1 :b � :a :p2 :b :firstAuthor rdfs:subPropertyOf :Author skos:prefLabel rdfs:subPropertyOf rdfs:label • rdfs:domain and rdfs:range (specify how a property can be used) :p1 rdfs:domain :D, :a :p1 :b � :a rdf:type :D :p2 rdfs:range :R, :a :p2 :b � :b rdf:type :R E.g. :performSurgeryOn rdfs:domain :Surgeon :performSurgeryOn rdfs:range :Patient • rdfs:label, seeAlso, isDefinedBy, � :Jack rdfs:seeAlso http://�/Jack_Blog10 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9 Web Ontology Language (OWL) • More expressive compared to RDFS • Property related constructs Derives implicit • owl:inverseOf E.g. :write owl:inverseOf :authoredBy relationships • owl:SymmetricProperty using inference :relatedTo rdf:type owl:SymmetricProperty foaf:knows is not defined as a symmetric property! • owl:TransitiveProperty :partOf rdf:type owl:TransitiveProperty. skos:broader rdf:type owl:TransitiveProperty • owl:equivalentProperty • owl:FunctionalProperty :hasBirthMother rdf:type owl:FunctionalProperty • owl:InverseFunctionalProperty foaf:mbox rdf:type owl:InverseFunctionalProperty • Instances (owl:sameAs, owl:differentFrom)11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 0 Web Ontology Language (OWL) • Class related constructs • owl:equivalentClass • owl:disjointWith :Boys owl:disjointWith :Girls • owl:complementOf :Boys owl:complementOf :Non_Boys • owl:unionOf, owl:intersectionOf, owl:oneOf • owl:Restriction is used to define a class whose members have certain restrictions w.r.t a property • owl:someValuesFrom • owl:allValuesFrom • owl:hasValue11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1 Web Ontology Language (OWL) • Class related constructs • owl:equivalentClass • owl:disjointWith :Boys owl:disjointWith :Girls owl:someValuesFrom • owl:complementOf :Boys owl:complementOf :Non_Boys • owl:unionOf, owl:intersectionOf,•:ApprovedPurchaseOrder owl:oneOf owl:equivalentClass [ a owl:Restriction ; • owl:Restriction is used to define a classowl:onProperty whose membe :approvedByrs have ; certain restrictions w.r.t a property owl:someValuesFrom :Manager ] • owl:someValuesFrom • owl:allValuesFrom :PO1 :approvedBy :managerXyz :managerXyz rdf:type :Manager • owl:hasValue � :PO1 rdf:type :ApprovedPurchaseOrder11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2 Web Ontology Language (OWL) • Class related constructs • owl:equivalentClass owl:allValuesFrom • owl:disjointWith :Boys owl:disjointWith :Girls • owl:complementOf •:Vegetarian rdfs:subClassOf [ a owl:Restriction ; :Boys owl:complementOf :Non_Boys owl:onProperty :eats ; • owl:unionOf, owl:intersectionOf,owl:allValuesFrom owl:oneOf :VegetarianFood ] • owl:Restriction is used to define a class whose members have certain restrictions w.r.t a property:Jen rdf:type :Vegetarian . • owl:someValuesFrom :Jen :eats :Marzipan . � :Marzipan rdf:type :VegetarianFood . • owl:allValuesFrom• owl:hasValue We SHOULD not use :eats rdfs:range :VegetarianFood11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3 Web Ontology Language (OWL) • Class related constructs • owl:equivalentClass • owl:disjointWith :Boys owl:disjointWith :Girls • owl:complementOf owl:hasValue :Boys owl:complementOf :Non_Boys • owl:unionOf, owl:intersectionOf, owl:oneOf•:HighPriorityItem owl:equivalentClass • owl:Restriction is used to define a class whose[ a owl:Restriction members ;have certain restrictions w.r.t a property owl:onProperty :hasPriority ; owl:hasValue :High ] • owl:someValuesFrom • owl:allValuesFrom :Item1 rdf:type :HighPriorityItem . • owl:hasValue � :Item1 :hasPriority :High11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4 Web Ontology Language (OWL) • Class related constructs • Cardinality restrictions constrain the number of distinct individuals that can associate with a class instance via a particular property • owl:minCardinality • owl:maxCardinality • owl:cardinality E.g. To express that a basketball game has at least 2 players :BasketBallGame rdfs:subClassOf [ a owl:Restriction; owl:onProperty :hasPlayer; owl:minCardinality 2 ] • Others • DatatypeProperty, AnnotationProperty, OntologyProperty,� 11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5 OWL 2• More language constructs, better expressivity • Property Chains, Keys, Punning, etc. • http://www.w3.org/TR/owl2�new�features/• OWL 2 Profiles • OWL 2 RL • OWL 2 EL • OWL 2 QL11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6 OWL 2 RL• Syntactic subset of OWL 2 – W3C standard profile – Inspired by DLP, pD* – Has more than 70 entailment rules – Reasoning/conjunctive query answering is PTIME w.r.t data/taxonomy complexity – Defines a standard set of rules for implementation• Oracle Spatial and Graph provides full support for OWL 2 RL/RDF ruleset11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7 Example OWL2 RL Entailment Rules• OWL2RL has 70+ entailment rules. – E.g. rule : T(?p, owl:propertyChainAxiom, ?x)LIST[?x, ?p1, ..., ?pn]T(?u1, ?p1, ?u2)T(?u2, ?p2, ?u3) ...T(?un, ?pn, ?un+1) . �� T(?u1, ?p, ?un+1)T(?p, rdf:type, owl:FunctionalPropertyT(?x, ?p, ?y1)T(?x, ?p, ?y2) . �� T(?y1, owl:sameAs, ?y2) • These rules have efficient implementations in Oracle Spatial and Graph11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8 OWL 2 EL• SNOMED�CT is a major application of OWL 2 EL – Suitable for applications employing ontologies that define very large numbers of classes and/or properties – One of the largest commercial biomedical ontologies• Example rule for EL+ inference ?A rdfs:subClassOf ?A1 … ?A rdfs:subClassOf ?An T(?C, owl:intersectionOf, ?x) LIST[?x, ?A1, ..., ?An] � ?A rdfs:subClassOf ?C • Oracle Spatial and Graph provides full support for OWL 2 EL11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9 Part 4: RDF View on Relational Data (RDB2RDF)12 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 0 A Look at Some System Architectures Typical Traditional Application Architecture � Mid�tier server and database Mid�Tier Server serverApplication 1 Application 2 Application 3 � Applications communicate via SQL/JDBC with RDBMS backend SQL � Multiple traditional relational schemas HR Schema Inventory Schema Sales Schema � Issues HR Database Inventory Database Sales Database – Inflexible schema – Limited semantics Database Server – Limited interoperability121 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 A Look at Some System Architectures Typical Semantic Application Architecture � Common ontologies used to integrate datasets Mid�Tier Server � Applications communicate Application 1 Application 2 Application 3 via SPARQL / HTTP with native triplestore backend SPARQL � Issues – Radical change for Shared Ontologies customer – Need to ETL to RDFHR Graph – RDF/OWL may not be Sales Graph Inventory Graph Triplestore necessary for the entire data122 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Relational Data to RDF (RDB2RDF) W3C Specification The mission of the RDB2RDF Working Group, part of the Semantic Web Activity, is to standardize languages for mapping relational data and relational database schemas into RDF and OWL*. The two languages are: � Direct Mapping – Automatically generates a mapping based on an input relational schema � R2RML – Language for expressing customized mappings* http://www.w3.org/2001/sw/rdb2rdf/ 123 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 A Look at Some System Architectures How about RDB2RDF? � Use virtual RDF data � Benefits Mid�Tier Server Application 2 Application 3 Application 1 SPARQL – Existing relational data stays in place and corresponding applications Shared Ontologies do not need to change RDB2RDF SQL – Use of virtual mapping Inventory Graph Sales Graph eliminates synchronization issues HR Schema Inventory Schema Sales Schema – Common vocabulary helps HR Database Inventory Database Sales Database with <a href="/tags/Data_integration/" rel="tag">data integration</a> issues Database Server124 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Using R2RML: Overall FlowQuery Writer SPARQL QUERYSchema: Classes Map: Classes and SPARQL to SQL and Predicates Predicates � Translator DB ObjectsR2RML R2RML Processor DocumentR2RML Map Author Source Relational Database 125 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 R2RML Basics: Mapping class subject predicates class subject predicates constraintsLogical Logical Table A Table B•TriplesMap: Row of a Logical Table (Table / View / SQL query) � triples • SubjectMap: Primary key value of a Row � subject (+ static class) • PredicateMap: Names of columns and constraints � predicates (incl. rdf:type) • ObjectMap: Values in columns or foreign keys � objects (+ dynamic class)126 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Vocabulary: R2RML Classes and Relations rr:parentTriplesMap rr:logicalTable rr:predicateObjectMap rr:subjectMap (rr:subject) rr:graphMap rr:predicateMap rr:objectMap (rr:graph) (rr:predicate) (rr:object) rr:graphMap (rr:graph) rr:joinConditionSource: (annotated with relation names) R2RML: RDB to RDF Mapping Language W3C Candidate Recommendation 23 February 2012 http://www.w3.org/TR/2012/CR�r2rml�20120223/127 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Mapping EMP and DEPT Tables to RDF x:Department class Subject http://x.com/Dept/{DNO} <http://x.com/Dept/100> rdf:type x:Department ;<../Dept/Deptno> 100 ; <../Dept/DeptName> “Sales” ; <../Dept/Location> “NYC” . Predicate�object pairs constraints pkey ref. pkey ENO ENAME EXPERTISE DNO DNO DNAME LOC 1 John DB 100 100 Sales NYC EMP DEPT128 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Mapping EMP and DEPT Tables to RDF class x:EmployeeSubject http://x.com/Emp/{ENO} <http://x.com/Emp/1> rdf:type x:Employee ;<../Emp/Empno> 1 ; <../Emp/EmpName> “John” ; <../Emp/Expertise> “DB” ; <../Emp/DeptNum> 100 ;<../Emp/Department> <http://x.com/Dept/100> Predicate�object pairs . constraints pkey ref. pkey ENO ENAME EXPERTISE DNO DNO DNAME LOC 1 John DB 100 100 Sales NYC EMP DEPT129 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Schema for Generated RDF data� Classes � Properties – x:Department – <../Dept/Deptno> – <../Dept/DeptName> – <../Dept/Location>– x:Employee – <../Emp/Empno> – <../Emp/EmpName> – <../Emp/Expertise> – <../Emp/DeptNum>– <../Emp/Department>130 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 R2RML Mapping for EMP (w/ multi�ObjectMap) Empno Ename W_phone C_phone H_phone W_addr H_addr DeptNo NUMBER Varchar NUMBER Varchar Varchar Varchar Varchar NUMBERTmap �� <#EmpTM> Smap �� [] POmap �� [] rr:logicalTable [ rr:template rr:predicate em:phone ; rr:tableName “EMP” “http://ex.org/E/{EMPNO}” ; rr:objectMap ]. rr:class ex:Employee . [ rr:column “W_PHONE” ] , [ rr:column “C_PHONE” ] POmap�� [] , [ rr:column “H_PHONE” ] . rr:predicate em:dept ; rr:objectMap [ rr:parentTriplesMap <#DeptTM> ; RefObjectMap rr:joinCondition [ rr:child “DEPTNO” ; rr:parent “DEPTNO” ]].131 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 R2RML Mapping: using “R2RML Views” <#DeptTableView> rr:sqlQuery """ SELECT DEPTNO, DNAME, LOC , (SELECT COUNT(*) FROM EMP WHERE EMP.DEPTNO=DEPT.DEPTNO) AS STAFF FROM DEPT """.<#TriplesMap1> rr:logicalTable <#DeptTableView>; rr:subjectMap [ rr:template "http://data.example.com/department/{DEPTNO}"; rr:class ex:Department; ]; rr:predicateObjectMap [ rr:predicate ex:name; rr:objectMap [ rr:column "DNAME" ]; ]; rr:predicateObjectMap [ rr:predicate ex:location; rr:objectMap [ rr:column "LOC" ]; ]; rr:predicateObjectMap [ rr:predicate ex:staff; rr:objectMap [ rr:column "STAFF" ]; ]. 132 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 R2RML Mapping: Translating type codes to IRIs <#TriplesMap1> R2RML rr:logicalTable [ rr:sqlQuery """ Mapping SELECT *, (CASE JOB WHEN 'CLERK' THEN 'general�office' WHEN 'NIGHTGUARD' THEN 'security' WHEN 'ENGINEER' THEN 'engineering' END) ROLE FROM EMP """ ]; rr:subjectMap [ rr:template "http://data.example.com/employee/{EMPNO}"; ]; rr:predicateObjectMap [ rr:predicate ex:role; rr:objectMap [ rr:template "http://data.example.com/roles/{ROLE}" ]; ]. Generated RDF triple <http://data.example.com/employee/7369> ex:role <http://data.example.com/roles/general�office> .133 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Part 5: Key Features of RDF Semantic Graph a feature of Oracle Spatial and Graph13 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4 Oracle Spatial and Graph Option RDF Semantic Graph Feature:RDF Semantic Graph Feature• Native RDF Database • Named graph support • Supports SPARQL 1.1, SPARQL/SQL, GeoSPARQL • Jena, Sesame, & Joseki Web Services • W3C standards: RDFS, OWL 2 RL, OWL 2 EL, SKOS, RDF, RDB2RDF, SPARQL 1.1, RDFa • Scales with hardware – petabytes of triples • Works with OBIEE, Oracle BPM, Advanced Analytics • Exploits: Exadata, RAC, Parallelism, Label Security13 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5 Oracle Spatial and Graph RDF Triple Store • Native RDF graph data store Load / Leverages Oracle Manageability: • Manages billions of triples Storage • RAC & Exadata scalability • Optimized storage architecture • Compression & partitioning • SPARQL-Jena/Joseki, Sesame • SQL*Loader direct path load Query • SQL/graph query, B-tree indexing • Parallel load, inference, query • Ontology assisted SQL query • High Availability • RDFS, OWL2 RL, EL, SKOS • Triple�level label security • User-defined rules Reasoning • Incremental, parallel reasoning • Ladder based inference • User-defined inferencing • Choice of SPARQL, SQL, or Java • Plug-in architecture • Native inference engine • Semantic indexing framework Analytics • Integration with • Enterprise Manager • OBIEE, Oracle R Enterprise • Oracle Data Mining 13 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6 New functions in Oracle Spatial and Graph • Open Geospatial Consortium (OGC) GeoSPARQL • Native SPARQL 1.1 query support – 40+ new query functions/operators: IF, COALESCE, STRBEFORE, REPLACE, ABS, – Aggregates: COUNT, SUM, MIN, MAX, AVG, GROUP_CONCAT, SAMPLE – Subqueries – Value Assignment: BIND, GROUP BY Expressions, SELECT Expressions – Negation: NOT EXISTS, MINUS – Improved Path Searching with Property Paths13 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7 New functions in Oracle Spatial and Graph• RDF views on relational tables (through RDB2RDF) – RDF views can be created on a set of relational tables and/or views – SPARQL queries access data from both a relational and RDF store – Support RDF view creation using • Direct Mapping: simple and straightforward to use • R2RML Mapping: customizations allowed • Inference – Native OWL 2 EL inference support – User defined inferencing – Ladder Based Inference – Performance optimization for user defined rules – Integration with TrOWL, an external OWL 2 reasoner13 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8 * http://trowl.eu/ Jena Adapter for <a href="/tags/Oracle_Database/" rel="tag">Oracle Database</a>13 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9 Jena Adapter for Oracle Database • Requires Apache Jena 2.7.2, ARQ 2.9.2, Joseki 3.4.4, Oracle Database release 11.2.0.3 or higher • SPARQL 1.1 compliance • Named Graph (quads) support: DatasetGraphOracleSem • N�QUADS, TriG data format • Updated StatusListener interface • Named graph queries through Joseki web service endpoint • SPARQL Update through Joseki web service endpoint • JSON output14 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 0 Jena Adapter for Oracle Database (2)• An efficient approach to better support OntModel APIs OracleGraphWrapperForOntModel • Support for reserved SQL (PL/SQL) keywords select ?date { :event :happenedOn ?date } • Named graph based local inference: Attachment� public void setUseLocalInference(boolean useLocalInference) � public boolean getUseLocalInference()� public void setDefGraphForLocalInference(String defaultGraphName) � public String getDefGraphForLocalInference()� public String getInferenceOption() � public void setInferenceOption(String inferenceOption)14 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1 Jena Adapter for Oracle Database (3) • Analytical functions for RDF data: SemNetworkAnalyst • Integrating Oracle Spatial and Graph network data model (NDM) with RDF Semantic Graph feature • Provides functions including shortest path, within cost, partitioning, � • Extensible architecture14 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2 Oracle Spatial and Graph Native Inference Engine14 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3 Core Inference Features in Oracle Database• Oracle provides native inference in the database for • RDFS, RDFS++ • OWLPRIME, OWL2RL, OWL2EL, SKOS • User�defined rules • Inference done using forward chaining • Triples inferred and stored ahead of query time • Removes on�the�fly reasoning and results in fast query times • Proof generation • Shows one deduction path14 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4 Native Inference Engine in Oracle: APIsTypical Usage: SEM_APIS.CREATE_ENTAILMENT( • First load RDF/OWL data • entailment_name • Call create_entailment to generate • sem_models(‘GraphTBox’, ‘GraphABox’, �), inferred graph • sem_rulebases(‘OWL2RL’), • passes, • Query both original graph and • inf_components, inferred data • Options,� Inferred graph contains only new triples! ) Saves time & resources Use “PROOF=T” to generate inference proofTypical Usage: SEM_APIS.VALIDATE_ENTAILMENT( • First load RDF/OWL data • sem_models((‘GraphTBox’, ‘GraphABox’, �), • Call create_entailment to generate • sem_rulebases(‘OWL2RL’), • Criteria, inferred graph • Max_conflicts, • Call validate_entailment to find • Options inconsistencies ) Java API: performInference, deleteInference, setInferenceOption, analyze methods in • GraphOracleSem, DatasetGraphOracleSem (Jena Adapter)14 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5 Native Inference Engine in Oracle• Leverage SQL and relational technologies (partitioning, compression) e.g. RDFS9 Rule Implemented in SQL Parallel select distinct T2.SID, Execution ID(rdf:type), T1.OID from <IVIEW> T1, <IVIEW> T2 where T1.PID=ID(rdfs:subClassOf) and T2.PID=ID(rdf:type) and T1.SID=T2.OID and not exists ( select 1 from <IVIEW> m where m.SID=T2.SID and m.PID=ID(rdf:type) and m.OID=T1.OID) � Implementing an Inference Engine for RDFS/OWL Constructs, ICDE 2008 � Optimizing Enterprise�scale OWL 2 RL Reasoning in a Relational Database System, ISWC 2010 � Advancing the Enterprise�class OWL Inference Engine in Oracle Database, ORE 2012 � Making the Most of your Triple Store: Query Answering in OWL 2 Using an RL Reasoner, WWW 201314 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6 Native Parallel Inference Engine in Oracle (3)• Transitive closure calculation – E.g. ?a owl:sameAs ?b � ?a owl:sameAs ?c ?b owl:sameAs ?c – Partition (distance d) based Iterative approach – P1 has :a => :b, :b => :c (asserted) – Join P1 with P1 to produce P2 – P2 has :a => :c (assign distance 2) – Join P1 with P2 to produce P3 – �:a d=1 d=2 1 Implementing an Inference Engine for RDFS/OWL Constructs, ICDE 2008 owl:sameAs On the Computation of the Transitive Closure of Relational Operators, VLDB 1986 :b :c d=114 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7 Native Parallel Inference Engine in Oracle (4) family:John • Incremental Maintenance when there is a :hasParent small addition Delta to a big graph G :hasUncle family:Mary ?x :hasParent ?y . ?y :hasBrother ?z �� ?x :hasUncle ?y :hasBrother family:Jack Graph G self join Graph G :relatedTo ns:Jill ns:Jedi :relatedTo How to efficiently handle a new edge :Mary :hasBrother :Tony ?14 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Oracle Confidential April 2013 8 Native Parallel Inference Engine in Oracle (4) family:John • Incremental Maintenance when there is a :hasParent small addition Delta to a big graph G :hasUncle family:Mary ?x :hasParent ?y . ?y :hasBrother ?z �� ?x :hasUncle ?y :hasBrother family:Jack Graph G self join Graph G :relatedTo ns:Jill ns:Jedi :relatedToAdding a new edge :Mary :hasBrother :Tony family:John :hasUncle :hasParent family:Tony :hasBrother :hasUncle family:Mary :hasBrother family:Jack :relatedTo ns:Jill ns:Jedi :relatedTo 14 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9 Native Parallel Inference Engine in Oracle (4) family:John • Incremental Maintenance when there is a :hasParent small addition Delta to a big graph G :hasUncle family:Mary ?x :hasParent ?y . ?y :hasBrother ?z �� ?x :hasUncle ?y :hasBrother family:Jack Graph G self join Graph G – To update the closure when Graph G becomes :relatedTo ns:Jill ns:Jedi Graph G’, we can perform 3 efficient small joins than :relatedTo 1 big join.Approach ?x :hasParent ?y ?y :hasBrother ?z Adding a new edge :Mary :hasBrother :Tony :hasUncle I: A single big Graph G’ Graph G’ family:John join :hasParent family:Tony Delta Graph G :hasBrother :hasUncle family:Mary II: 3 small joins Graph G Delta :hasBrother family:Jack Delta Delta :relatedTo � Optimizing Enterprise�scale OWL 2 RL Reasoning in a Relational Database System, ISWC 2010 ns:Jill ns:Jedi :relatedTo 15 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 0 Extending Semantics Supported by Native OWL Inference Engine • Option 1: Add user�defined rules • Oracle supports, from release 10g, user�defined rules :Antecedents Consequents ?z :parentOf ?x . � ?x :siblingOf ?y ?z :parentOf ?y . ?x owl:differentFrom ?y . • Option 2: Leverage external DL reasoners• Option 3: User�defined inferencing in Oracle Database Release 12c15 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1 Extensible Architecture for External ReasonersOracle’s External Native Inference In�Memory Jena APIs Engine for OWL DL through OWL 2 RL, EL & Reasoners Jena Adapter user�defined rules TrOWL/REL Materialized Inference15 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2 Enabling Advanced Inference Capabilities • Parallel inference option EXECUTE sem_apis.create_entailment('M_IDX',sem_models('M'), sem_rulebases('OWLPRIME'), null, null, 'DOP=x'); – Where ‘x’ is the degree of parallelism (DOP) • Incremental inference option EXECUTE sem_apis.create_entailment ('M_IDX',sem_models('M'), sem_rulebases('OWLPRIME'),null,null, 'INC=T'); • Enabling owl:sameAs option to limit duplicates EXECUTE Sem_apis.create _entailment('M_IDX',sem_models('M'), sem_rulebases('OWLPRIME'),null,null,'OPT_SAMEAS=T'); • Compact data structures EXECUTE Sem_apis.create _entailment('M_IDX',sem_models('M'), sem_rulebases(‘OWLPRIME'),null,null, 'RAW8=T'); • OWL2RL/SKOS inference EXECUTE Sem_apis.create_entailment('M_IDX',sem_models('M'), sem_rulebases(x),null,null…); • x in (‘OWL2RL’,’SKOSCORE’)15 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3 Querying RDF Semantic Graph15 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4 SPARQL Query ArchitectureStandard SPARQL Endpoint HTTP Enhanced with query management controlJena API Sesame API Java Jena Adapter Sesame AdapterSPARQL�to�SQL Core SQL SEM_MATCH Logic155 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SPARQL vs. SQL SPARQL NULL�accepting JOIN vs. SQL NULL�rejecting JOIN?p ?n ?e ?p ?e <p1> "Jon Stewart" "js@oracle.com" <p1> "js@oracle.com" <p2> "jsmith@oracle.com" <p2> "John Smith" NULL ?p = ?p AND {?p :name ?n ?e = ?e OPTIONAL {?p :email ?e} OPTIONAL {?p :mbox ?e} }SQL ?p ?n ?e <p1> "Jon Stewart" "js@oracle.com" <p2> "John Smith" NULL?p ?n ?e SPARQL <p1> "Jon Stewart" "js@oracle.com" <p2> "John Smith" "jsmith@oracle.com"156 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SPARQL vs. SQL• SPARQL weak typing vs. SQL strong typing FILTER (?a + ?b > 10) � doesn’t complain if ?a, ?b not numbers.Also, when ?a and/or ?b are not numbers, FILTER(?a + ?b > 10) and FILTER(!(?a + ?b > 10)) are both FALSEFILTER(?a != ?b), etc. are complicated• No Boolean type in SQL• SPARQL UNION SPARQL UNION is actually SQL UNION ALL SPARQL graph patterns DO NOT have to be UNION compatible157 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SEM_MATCH: SPARQL in SQL158 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SEM_MATCH: Adding SPARQL to SQL� Extends SQL with SPARQL constructs – Graph Patterns, OPTIONAL, UNION – Dataset Constructs – FILTER – including SPARQL built�ins – Solution Modifiers � Benefits: – Integrates graph data with existing enterprise data – JOINs with other object�relational data – Allows SQL constructs/functions – DDL Statements: create tables/views159 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SEM_MATCH: Adding SPARQL to SQLSELECT n1, n2 FROM TABLE( SEM_MATCH( 'PREFIX foaf: <http://...> SELECT ?n1 ?n2 FROM <http://g1> WHERE {?p foaf:name ?n1 OPTIONAL {?p foaf:knows ?f . ?f foaf:name ?n2 } FILTER (REGEX(?n1, "Â")) } ORDER BY ?n1 ?n2', SEM_MODELS('M1'),…));160 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SEM_MATCH: Adding SPARQL to SQLSELECT n1, n2 SQL Table Function FROM TABLE( Sn1EM_MATCH( n2 'PREFIX foaf: <http://...> AlexSELECT ?n1 ?n2 Jerry FROM <http://g1> AlexWHERE {?p foaf:name ?n1Tom Alice OPTIONAL {?p foaf:knowsBill ?f . ?f foaf:name ?n2 } Alice FILTER (REGEX(?n1,Jill "Â")) } ORDER BY ?n1 ?n2', SEM_MODELS('M1'),…));Alice John161 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SEM_MATCH: Adding SPARQL to SQLSELECT n1, n2 Rewritable SQL Table Function FROM (TABLE( SELECT v1.value AS n1, v2.value AS n2 SFROMEM_MATCH(VALUES v1, VALUES v2 'PREFIXT RIPLESfoaf: <http://...>t1, TRIPLES t2, … WHERESELECTt1.obj_id ?n1 ?n2 = v1.value_id FROMAND <http://g1>t1.pred_id = 1234 WHEREAND … {?p foaf:name ?n1 ) OPTIONAL {?p foaf:knows ?f . Get ?f1 unified foaf:name SQL ?n2 query } FILTER� Query(REGEX(?n1, optimizer "Â"))sees 1 query } ORDER BY ?n1 �?n2', Get all the performance of Oracle SQL Engine SEM_MODELS('M1'),…));� optimizer, compression, indexes, parallelism, etc.162 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 SEM_MATCH Table Function Arguments'SELECT ?a SEM_MATCH( WHERE { ?a foaf:name ?b }' query, Basic unit of Container(s) for access control models, asserted quads Entailed rulebases, + triples options Built�in (e.g. OWL2RL) and user�defined ); rulebases'ALLOW_DUP=T STRICT_TERM_COMP=F'163 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 GovTrack RDF DataRDF/OWL data about activities of US Congress • Political Party Membership • Voting Records GOV_TBOX • Bill Sponsorship • Committee Membership GOV_PEOPLE• Offices and Terms GOV_BILLS_110GOV_BILLS_111 Asserted data only GOV_ASSERT_VM (4.3M triples) OWL2RL GOV_VOTES_110Asserted + Inferred GOV_ALL_VM (4.7M triples) GOV_VOTES_111 INFERENCEGOV_DISTRICTS (US Census)Virtual Models Rulebases GOV_TRACK_OWL Semantic Models Entailments GovTrack in Oracle164 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: SPARQL SELECT ModifiersFind all distinct family names for senators SELECT f$rdfterm FROM TABLE(SEM_MATCH( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX vcard: <http://www.w3.org/2001/vcard-rdf/3.0#> SELECT DISTINCT ?f WHERE { ?p vcard:N ?vn . ?vn vcard:Family ?f . ?p foaf:title "Sen." . }' ,sem_models('gov_all_vm'), null, null, null ,null, ' ALLOW_DUP=T PLUS_RDFT=T '));165 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: SPARQL FILTERFind information about Adams that were born before the American Revolution select n$rdfterm, b$rdfterm, g$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX vcard: <http://www.w3.org/2001/vcard-rdf/3.0#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT ?n ?b ?g WHERE { ?p vcard:N ?vn . ?vn vcard:Family ?f . ?p foaf:name ?n . ?p vcard:BDAY ?b . ?p foaf:gender ?g FILTER ( ?f = "Adams" && xsd:dateTime(?b) < xsd:dateTime("1776-07-04") )}' ,sem_models('gov_all_vm'), null, null ,null, null,' ALLOW_DUP=T PLUS_RDFT=T '));166 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: SPARQL MINUSFind information about Kennedys that don’t have a homepage select n$rdfterm, b$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX vcard: <http://www.w3.org/2001/vcard-rdf/3.0#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT ?n ?b WHERE { ?p vcard:N ?vn . ?vn vcard:Family "Kennedy" . ?p foaf:name ?n . ?p vcard:BDAY ?b . MINUS { ?p foaf:homepage ?h } }' ,sem_models('gov_all_vm'), null, null ,null, null,' ALLOW_DUP=T PLUS_RDFT=T ‘));167 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: AggregationWho sponsored the most bills? select n$rdfterm, cnt$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX vcard: <http://www.w3.org/2001/vcard-rdf/3.0#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> PREFIX bill: <http://www.rdfabout.com/rdf/schema/usbill/> SELECT ?n (COUNT(*) AS ?cnt) WHERE { ?s foaf:name ?n . ?b bill:sponsor ?s . } GROUP BY ?n ORDER BY DESC(?cnt) LIMIT 10' ,sem_models('gov_all_vm'), null, null ,null, null,' ALLOW_DUP=T PLUS_RDFT=T '));168 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: Named Graph QueryFind number of bills sponsored in 110th and 111th congress select n$rdfterm, g$rdfterm, bcnt$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX bill: <http://www.rdfabout.com/rdf/schema/usbill/> SELECT ?n ?g (count(?b) as ?bcnt) FROM usgov:people FROM NAMED usgov:bills_110 FROM NAMED usgov:bills_111 WHERE { ?s foaf:name ?n GRAPH ?g { ?b bill:sponsor ?s } } GROUP BY ?n ?g ORDER BY ?n ?g' ,sem_models('gov_all_vm'), null, null ,null, null,' ALLOW_DUP=T PLUS_RDFT=T '));169 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: Inference GovTrack Bill Types170 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: EntailmentFind bills sponsored by Barack Obama and their types select title$rdfterm, dt$rdfterm, btype$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX bill: <http://www.rdfabout.com/rdf/schema/usbill/> PREFIX dc: <http://purl.org/dc/elements/1.1/> SELECT ?title ?dt ?btype WHERE { ?s foaf:name "Barack Obama" . ?b bill:sponsor ?s . ?b dc:title ?title . ?b rdf:type ?btype . ?b bill:introduced ?dt FILTER(xsd:dateTime("2007-03-28") <= xsd:dateTime(?dt) && xsd:dateTime(?dt) < xsd:dateTime("2007-04-01") ) }' ,sem_models('gov_all_vm'), null, null, null ,null, ' ALLOW_DUP=T PLUS_RDFT=T’));171 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: Entailment with Property PathFind bills sponsored by Barack Obama and their types select title$rdfterm, dt$rdfterm, btype$rdfterm, sc$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX bill: <http://www.rdfabout.com/rdf/schema/usbill/> PREFIX dc: <http://purl.org/dc/elements/1.1/> SELECT ?title ?dt ?btype ?sc WHERE { ?s foaf:name "Barack Obama" . ?b bill:sponsor ?s . ?b dc:title ?title . ?b rdf:type ?btype . ?btype rdfs:subClassOf+ ?sc . ?b bill:introduced ?dt FILTER(xsd:dateTime("2007-03-28") <= xsd:dateTime(?dt) && xsd:dateTime(?dt) < xsd:dateTime("2007-04-01") ) }' ,sem_models('gov_assert_vm'), null, null, null ,null, ' ALLOW_DUP=T PLUS_RDFT=T’));172 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Full Text Indexing with Oracle Text� Filters graph patterns based on text search string � Indexes all RDF Terms – URIs, Literals, Language Tags, etc. � Provide SPARQL extension function – orardf:textContains(?var, "Oracle text search string") – Search String � Group Operators: AND, OR, NOT, NEAR, … � Term Operators: stem($), soundex(!), wildcard(%)SQL> exec sem_apis.add_datatype_index( 'http://xmlns.oracle.com/rdf/text');173 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: Full Text SearchFind information about bills related to children and taxes select n$rdfterm, title$rdfterm, dt$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX bill: <http://www.rdfabout.com/rdf/schema/usbill/> PREFIX dc: <http://purl.org/dc/elements/1.1/> SELECT ?n ?title ?dt WHERE { ?b bill:sponsor ?s . ?s foaf:name ?n . ?b dc:title ?title . ?b bill:introduced ?dt FILTER (orardf:textContains(?title, "$children AND $taxes"))}' ,sem_models('gov_all_vm'), null, null, null ,null, ' ALLOW_DUP=T PLUS_RDFT=T ' ));174 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 GeoSPARQL with Oracle Spatial and Graph� Support geometries encoded as ogc:wktLiterals:semTech2011 ogc:asWKT "POINT(-122.4192 37.7793)"^ôgc:wktLiteral .� Provide a library of spatial functionsSELECT ?s WHERE { ?s ogc:asWKT ?geom FILTER(ogc:distance(?geom, "POINT(-122.4192 37.7793)"^ôgc:WKTLiteral, uom:KM) <= 10)175 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 OGC wktLiteral Datatype� Optional leading Spatial Reference System URI followed by OGC WKT geometry string. <http://xmlns.oracle.com/rdf/geo/srid/{srid}> � WGS 84 Longitude, Latitude is the default SRS (assumed if SRS URI is absent)SRS: WGS84 Longitude, Latitude "POINT(-122.4192 37.7793)"^ôgc:wktLiteralSRS: NAD27 Longitude, Latitude "<http://xmlns.oracle.com/rdf/geo/srid/8260> POINT(-122.4181 37.7793)"^ôgc:wktLiteral176 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 OGC wktLiteral Datatype� Prepare for spatial querying by creating a spatial index for the ogc:wktLiteral datatypeSQL> exec sem_apis.add_datatype_index( 'http://xmlns.oracle.com/rdf/geo/WKTLiteral', options=>'TOLERANCE=1.0 SRID=8307 DIMENSIONS=((LONGITUDE,-180,180)(LATITUDE,-90,90))');177 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 What Types of Spatial Data are Supported?� Spatial Reference Systems – Built�in support for 1000’s of SRS – Plus you can define your own – Coordinate system transformations applied transparently during indexing and query � Geometry Types – Support OGC Simple Features geometry types � Point, Line, Polygon � Multi�Point, Multi�Line, Multi�Polyon � Geometry Collection – Up to 500,000 vertices per Geometry178 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Spatial Function Library Standard OGC functions � Topological Relations – ogcf:relate, ogcf:sfContains, ogcf:sfCrosses, ogcf:sfDisjoint, ogcf:sfEquals, ogcf:sfIntersects, ogcf:sfOverlaps, ogcf:sfTouches, ogcf:sfWithin � Distance�based Operations – ogcf:distance, ogcf:buffer � Geometry Operations – ogcf:boundary, ogcf:convexHull, ogcf:envelope, ogcf:getSRID, � Geometry�Geometry Operations – ogcf:difference, ogcf:intersection, ogcf:symDifference, ogcf:union179 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Spatial Function Library Oracle Extensions � Topological Relations – orageo:relate � Distance�based Operations – orageo:distance, orageo:withinDistance, orageo:buffer, orageo:nearestNeighbor � Geometry Operations – orageo:area, orageo:length – orageo:centroid, orageo:mbr, orageo:convexHull � Geometry�Geometry Operations – orageo:intersection, orageo:union, orageo:difference, orageo:xor180 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 GovTrack Spatial Demo� Congressional District Polygons (435) – Complex Geometries – Average over 1000 vertices per geometryLoad .shp file from US Census Load into Oracle into Oracle Spatial semantic model Generate triples using sdo_util.toWKTGeometry()181 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: Spatial QueryFind the representative and district for Nashua, NH select name$rdfterm, cdist$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX pol: <http://www.rdfabout.com/rdf/schema/politico/> PREFIX ogc: <http://www.opengis.net/ont/geosparql#> PREFIX ogcf: <http://www.opengis.net/def/function/geosparql/> SELECT ?name ?cdist WHERE { ?person foaf:name ?name . ?person pol:hasRole/pol:forOffice/pol:represents ?cdist . ?cdist ogc:asWKT ?cgeom FILTER (ogcf:sfContains(?cgeom, "POINT(-71.46444 42.7575)"^ôgc:wktLiteral)) } ' ,sem_models('gov_all_vm'), null, null, null ,null, ' ALLOW_DUP=T PLUS_RDFT=T ' ));182 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: Spatial Query Find the 10 nearest congressional districts to Nashua, NH ordered by distance select name$rdfterm, cdist$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX pol: <http://www.rdfabout.com/rdf/schema/politico/> PREFIX ogc: <http://www.opengis.net/ont/geosparql#> PREFIX ogcf: <http://www.opengis.net/def/function/geosparql/> PREFIX uom: <http://xmlns.oracle.com/rdf/geo/uom/> SELECT ?name ?cdist WHERE { ?person foaf:name ?name . ?person pol:hasRole/pol:forOffice/pol:represents ?cdist . ?cdist ogc:asWKT ?cgeom FILTER (orageo:nearestNeighbor(?cgeom, "POINT(-71.46444 42.7575)"^ôgc:wktLiteral, "sdo_num_res=10")) } ORDER BY ASC(ogcf:distance(?cgeom, "POINT(-71.46444 42.7575)"^ôgc:wktLiteral, uom:KM))' ,sem_models('gov_all_vm'), null, null, null, null, ' ALLOW_DUP=T PLUS_RDFT=T ' )) order by sem$rownum;183 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: Spatial Query Find the 10 nearest congressional districts ordered by distance to center point select name$rdfterm, cdist$rdfterm from table(sem_match( 'PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX pol: <http://www.rdfabout.com/rdf/schema/politico/> PREFIX ogc: <http://www.opengis.net/ont/geosparql#> PREFIX ogcf: <http://www.opengis.net/def/function/geosparql/> PREFIX uom: <http://xmlns.oracle.com/rdf/geo/uom/> SELECT ?name ?cdist WHERE { ?person foaf:name ?name . ?person pol:hasRole/pol:forOffice/pol:represents ?cdist . ?cdist ogc:asWKT ?cgeom FILTER (orageo:nearestNeighbor(?cgeom, "POINT(-71.46444 42.7575)"^ôgc:wktLiteral, "sdo_num_res=10")) } ORDER BY ASC(ogcf:distance(orageo:centroid(?cgeom), "POINT(-71.46444 42.7575)"^ôgc:wktLiteral, uom:KM))' ,sem_models('gov_all_vm'), null, null, null, null, ' ALLOW_DUP=T PLUS_RDFT=T ' )) order by sem$rownum;184 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 RDB2RDF: Supporting RDF Views of Relational Data with Oracle Spatial and Graph185 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 RDB2RDF Architecture � Use virtual RDF data Mid�Tier Server � Benefits Application 2 Application 3 Application 1 – Existing relational data SPARQL/SQL stays in place and corresponding applications Shared Ontologies SQL do not need to change RDB2RDF – Use of virtual mapping Inventory Graph Sales Graph eliminates synchronization issues HR Schema Inventory Schema Sales Schema – Common vocabulary helps HR Database Inventory Database Sales Database with data integration issues Database Server186 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 RDB2RDF Support in Oracle Spatial and Graph� Support both Direct Mapping and R2RML: RDB to RDF Mapping Language � API for creating, dropping and exporting (materializing) RDF views – sem_apis.create_rdf_view_model – sem_apis.drop_rdf_view_model – sem_apis.export_rdf_view_model � Query like native RDF data187 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Relational Data: HR SchemaEMPLOYEES ( JOBS (EMPLOYEE_ID NOT NULL NUMBER(6) JOB_ID NOT NULL VARCHAR2(10) FIRST_NAME VARCHAR2(20) JOB_TITLE NOT NULL VARCHAR2(35) LAST_NAME NOT NULL VARCHAR2(25) MIN_SALARY NUMBER(6) EMAIL NOT NULL VARCHAR2(20) MAX_SALARY NUMBER(6) PHONE_NUMBER VARCHAR2(20) ) HIRE_DATE NOT NULL DATE JOB_ID NOT NULL VARCHAR2(10) SALARY NUMBER(8,2) JOB_HISTORY ( COMMISSION_PCT NUMBER(2,2) MANAGER_ID NUMBER(6) EMPLOYEE_ID NOT NULL NUMBER(6) DEPARTMENT_ID NUMBER(4) START_DATE NOT NULL DATE ) END_DATE NOT NULL DATE JOB_ID NOT NULL VARCHAR2(10) DEPARTMENT_ID NUMBER(4) )DEPARTMENTS ( LOCATIONS ( DEPARTMENT_ID NOT NULL NUMBER(4) DEPARTMENT_NAME NOT NULL VARCHAR2(30) LOCATION_ID NOT NULL NUMBER(4) MANAGER_ID NUMBER(6) STREET_ADDRESS VARCHAR2(40) LOCATION_ID NUMBER(4) POSTAL_CODE VARCHAR2(12) ) CITY NOT NULL VARCHAR2(30) STATE_PROVINCE VARCHAR2(25) COUNTRY VARCHAR2(30) GEO_LOCATION SDO_GEOMETRY )188 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Relational Data: E�CommercePRODUCT_INFORMATION ( CUSTOMERS (PRODUCT_ID NOT NULL NUMBER(6) CUSTOMER_ID NOT NULL NUMBER(6) PRODUCT_NAME VARCHAR2(50) CUST_FIRST_NAME NOT NULL VARCHAR2(20) PRODUCT_DESC VARCHAR2(2000) CUST_LAST_NAME NOT NULL VARCHAR2(20) CATEGORY_ID NUMBER(2) CUST_LOCATION_ID NUMBER(4) WEIGHT_CLASS NUMBER(1) PHONE_NUMBERS PHONE_LIST_TYP WARRANTY_PERIOD INTERVAL YEAR(2) TO MONTH CUST_EMAIL VARCHAR2(30) SUPPLIER_ID NUMBER(6)) ) PRODUCT_STATUS VARCHAR2(20) LIST_PRICE NUMBER(8,2) MIN_PRICE NUMBER(8,2) REVIEWS ( CATALOG_URL VARCHAR2(50) ) REVIEW_ID NOT NULL NUMBER(6) CUSTOMER_ID NUMBER(6) PRODUCT_ID NUMBER(6) WAREHOUSES ( RATING NUMBER(1) REVIEW_TEXT VARCHAR2(4000) WAREHOUSE_ID NOT NULL NUMBER(3) ) WAREHOUSE_SPEC SYS.XMLTYPE WAREHOUSE_NAME VARCHAR2(35) LOCATION_ID NUMBER(4) PRODUCT_CATEGORIES ( ) CATEGORY_ID NOT NULL NUMBER(6) CATEGORY_NAME VARCHAR2(50) INVENTORIES ( CATEGORY_DESCRIPTION VARCHAR2(2000) ) PRODUCT_ID NOT NULL NUMBER(6) WAREHOUSE_ID NOT NULL NUMBER(3) QUANTITY_ON_HAND NOT NULL NUMBER(8) )189 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Creating the RDF ViewBEGIN sem_apis.create_rdfview_model( 'r2r_model_full', SYS.ODCIVarchar2List ( 'DEPARTMENTS', 'EMPLOYEES', 'JOBS', 'JOB_HISTORY', 'LOCATIONS', 'PRODUCT_INFORMATION', 'WAREHOUSES', 'INVENTORIES', 'CUSTOMERS', 'REVIEWS', 'PRODUCT_CATEGORIES' ) , 'http://mydb/', options => ' SCALAR_COLUMNS_ONLY=T'); END; /190 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: RDB2RDFWhat classes do we have? select o$rdfterm from table(sem_match( 'SELECT DISTINCT ?o WHERE { ?s rdf:type ?o }' ,sem_models('r2r_model_full'),null,null,null ,null,' PLUS_RDFT=T '));191 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: RDB2RDFWhat properties do we have? select p$rdfterm from table(sem_match( 'SELECT DISTINCT ?p WHERE { ?s ?p ?o }' ,sem_models('r2r_model_full'),null,null,null ,null,' PLUS_RDFT=T '));192 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: RDB2RDFDescribe Employee 1 select p$rdfterm, o$rdfterm from table(sem_match( 'SELECT ?p ?o WHERE { <http://mydb/EMPLOYEES/EMPLOYEE_ID=1> ?p ?o }' ,sem_models('r2r_model_full'),null,null,null ,null,' PLUS_RDFT=T '));193 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: RDB2RDFFind all employees and their job title select fname$rdfterm, lname$rdfterm, t$rdfterm from table(sem_match( 'SELECT ?fname ?lname ?t WHERE { ?e <http://mydb/EMPLOYEES#FIRST_NAME> ?fname . ?e <http://mydb/EMPLOYEES#LAST_NAME> ?lname . ?e <http://mydb/EMPLOYEES#ref-JOB_ID> ?j . ?j <http://mydb/JOBS#JOB_TITLE> ?t }' ,sem_models('r2r_model_full'),null,null,null ,null,' PLUS_RDFT=T '));194 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Semantic Indexing: Extracting Semantic Data from Unstructured Text with Oracle Spatial and Graph195 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Semantic Indexing� Overview Analytical Queries On Graph Data Auto maintained like a Triples table with rowidreferences B�tree index Subject Property Object graph CREATE INDEX ArticleIndex p:Marcus rdf:type rc::Person <�/r1> ON Newsfeed (Article) INDEXTYPE IS SemContext p:Marcus :hasName “Marcus”^^� <�/r1> PARAMETERS (‘my_policy’) p:Marcus :hasAge “38”^^xsd:.. <�/r1> LOCAL PARALLEL 4 � � � �Newsfeed table SemContext index on Article columnRowid docId Article Source SELECT Sem_Contains_Select(1) 1 Indiana authorities filed felony charges and a CNN FROM Newsfeed r1 court issued an arrest warrant for a financial WHERE Sem_Contains (Article, Contentmanager type: who apparently tried to fake his ‘{?x rdf:type rc:Person . •Textdeath by crashing his airplane in a Florida ?x :hasAge ?age . swamp. Marcus Schrenker, 38 � •File (path) FILTER(?age >= 35)}’,1)=1 •URL r2 2 Major dealers and investors � NW AND Source = ‘CNN’196 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Semantic Indexing � Key Components� Extensible Information Extractor – Programmable API to plug�in 3rd party extractors into the database. � SemContext Indextype – A custom indexing scheme that interacts with the extractor to manage the metadata extracted from the documents efficiently and facilitates semantic search via SQL queries. � SEM_CONTAINS Operator – To identify documents of interest based on their extracted metadata, using standard SQL queries. � SEM_CONTAINS_SELECT Ancillary Operator – To return additional information (SPARQL Query Results XML) about the documents identified using SEM_CONTAINS operator.197 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Semantic Indexing � Key Concepts� Policy – Base Policy: <policy_name, extractor_type> – Dependent Policy: <policy_name, base_policy_name, ontology> � Association between indexes and policies – Multiple policies may be associated with an index – Metadata extracted from each base policy is stored separately � Sem_Contains invocation is restricted to one policy – Policy to be used can be specified by user � Inference – Document�centric – Corpus�centric198 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Product Descr.: “Harry Potter” series book 2� [from amazon.com] The Dursleys were so mean that hideous that summer that all Harry Potter wanted was to get back to the Hogwarts School for Witchcraft and Wizardry. But just as he's packing his bags, Harry receives a warning from a strange, impish creature named Dobby who says that if Harry Potter returns to Hogwarts, disaster will strike.And strike it does. For in Harry's second year at Hogwarts, fresh torments and horrors arise, including an outrageously stuck�up new professor, Gilderoy Lockheart, a spirit named Moaning Myrtle who haunts the girls' bathroom, and the unwanted attentions of Ron Weasley's younger sister, Ginny.But each of these seem minor annoyances when the real trouble begins, and someone��or something��starts turning Hogwarts students to stone. Could it be Draco Malfoy, a more poisonous rival than ever? Could it possibly be Hagrid, whose mysterious past is finally told? Or could it be the one everyone at Hogwarts most suspects...Harry Potter himself?199 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 News report: “Solving a Riddle of Primes”� [from nytimes.com: Solving a Riddle of Primes � Published: May 20, 2013] Three and five are prime numbers — that is, they are divisible only by 1 and by themselves. So are 5 and 7. And 11 and 13. And for each of these pairs of prime numbers, the difference is 2. � The proof has been elusive. But last month, a paper � arrived “out of the blue” at the journal Annals of Mathematics, said Peter Sarnak, a professor of mathematics at Princeton University and the Institute for Advanced Study and a former editor at the journal, which plans to publish it. The paper, by Yitang Zhang of the University of New Hampshire, � does show an infinite number of prime pairs whose separation is less than a finite upper limit — 70 million, for now. (Dr. Zhang used 70 million in his proof — basically an arbitrary large number where his equations work.) � Dr. Zhang’s proof takes advantage of a 2005 paper by Daniel Goldston of San Jose State University, Janos Pintz of the Alfred Renyi Institute of Mathematics in Budapest and Cem Yildirim of Bogazici University in Istanbul, which had shown there would always be pairs of primes closer than the average distance between two primes. � Dr. Zhang also used techniques developed in the 1980s by Henryk Iwaniec of Rutgers, Enrico Bombieri of the Institute for Advanced Study and John B. Friedlander of the University of Toronto, adding his own ingenuity to tie everything together in a way others had been unable to. �200 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Live Demo: Semantic IndexingFind information about professionals and their position name if they are a professor select docid, sem_contains_select(1) sparqlrslt from doctable where sem_contains (doc, 'SELECT ?nm ?orgnm ?posnm WHERE { ?s rdf:type ctype:PersonCareer . ?s c:careertype "professional" . ?s c:person ?o . ?o c:name ?nm . ?s c:organization ?org . ?org c:name ?orgnm . OPTIONAL{?s c:position ?pos . ?pos c:name ?posnm . FILTER (regex(?posnm,"^professor"))} }' , sem_aliases(sem_alias('ctype','http://s.opencalais.com/1/type/em/r/') ,sem_alias('c','http://s.opencalais.com/1/pred/')), 1) = 1 order by docid;201 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Part 6: Performance and Scalability20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2 Graphs Are Big and Are Getting Bigger • Social Scale* – 1 Billion vertices, 100 billion edges • Web Scale* • 50 billion vertices, 1 trillion edges • Brain Scale* • 100 billion vertices, 100 trillion edges20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. * An NSA Big Graph Experiment 3 http://www.pdl.cmu.edu/SDI/2013/slides/big_graph_nsa_rd_2013_56002v1.pdf A Big RDF Graph Linking Many Data Sources20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4 Graphs at ScaleProperties of Graph Problems1. Data Driven 2. Unstructured 3. Poor Locality 4. High IO to Compute ratio20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5 Graphs at ScaleProperties of Graph Problems1. Data Driven A. Dictated by Node and Link structure B. Structure not known a priori 2. Unstructured 3. Poor Locality 4. High IO to Compute ratio20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6 Graphs at ScaleProperties of Graph Problems1. Data Driven 2. Unstructured A. Irregular Structure of Graphs B. Difficult to parallelize C. Difficult to partition 3. Poor Locality 4. High IO to Compute ratio20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7 Graphs at ScaleProperties of Graph Problems1. Data Driven 2. Unstructured 3. Poor Locality A. Represent relationships between entities B. Esp. for graphs derived from analysis 4. High IO to Compute ratio20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8 Parallel Architectures and Computing Models1. Distributed Memory Architectures 2. Partitioned Global Address Space 3. Shared memory 4. Massively Multi�threaded20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9 Parallel Graph Queries and Hardware1. Task Granularity – where to introduce ||’ism? A. IO parallelism – DBWR, partitioning, || hintsB. CPU parallelism – multi�threading, single thread multi core 2. Memory Contention A. Shared memory in a Global Address Space B. Not an issue in Oracle 3. Load Balancing 1. How to load balance multi threaded systems 2. Less Pronounced on Shared Memory 4. Shared Graphs A. Focus on throughput B. IO ||’ism21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 0 Setup for Performance • Use a balanced hardware system for databases and mid�tier servers – A single, huge physical disk for everything is not recommended. • Multiple hard disks tied together through ASM is a good practice – A virtual machine for multiple databases and applications is not recommended – Make sure throughput of hardware components matches up Component Hardware spec Sustained throughput CPU core � 100 � 200 MB/s 1/2 Gbit HBA 1/2 Gbit/s 100/200 MB/s 16 port switch 8 * 2 Gbit/s 1,200 MB/s Fiber channel 2 Gbit/s 200 MB/s Disk controller 2 Gbit/s 200 MB/s GigE NIC (interconnect) 2 Gbit/s 80 MB/s* Disk (spindle) 30 � 50 MB/s MEM 2k�7k MB/s21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1 Or��1. Plan on Going to Disk 2. Running Out of Memory 3. Being Smart About It 4. Have IB already in place 5. Locate data close to compute nodes 6. Realizing It is Part of a Larger EcoSystem21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2 Oracle Engineered SystemsPurpose Built General PurposeExalogic Exadata SPARC SuperCluster21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. © 2011 Oracle Corporation – Proprietary and Confidential 3 Oracle Exadata Database Machine• Designed to be the best platform for Oracle Database • Integrates and optimizes servers, storage, network – Using high�volume servers in a scale�out architecture – Flash�optimized • Special software brings database intelligence to storage, flash, and networking – To deliver extreme performance and data compression • Deployments are fast, reliable, and supportable • Optimal for all database workloads – OLTP, Data Warehousing, and Database Clouds21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4 Exadata Storage Software Unique Features• Exadata Smart Scans • Exadata Smart Flash Cache – 10X or greater reduction in data sent – Breaks random I/O bottleneck by to database servers increasing IOPs by up to 20X – Doubles user data scan bandwidths • Exadata Storage Indexes – Eliminate unnecessary I/Os • I/O Resource Manager (IORM) – Enables storage grid by prioritizing • Hybrid Columnar Compression I/Os to ensure predictable – Efficient compression increases performance effective storage capacity and increases user data scan bandwidths • Quality of Service (QoS) by a factor of up to 10X – Actively meet and maintain SLAs – Very efficient on triple, quad – Memory Guard to protect existing current transactions from memory� based failures21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5 Components � Exadata, Exalogic, and SuperCluster Exadata X3-8 Exalogic X3-2 SuperCluster Compute Grid - Compute Servers 2 Sun Server X2-8 30 Sun Server X3-2 4 SPARC T4-4 - Compute Cores 160 Xeon 480 Xeon 128 Sparc T4 - Compute Server Memory 4 TB 7.5 TB 4 TB - Operating Systems Linux Linux / Solaris Solaris - Oracle Virtual Machine Partitioning - Yes Yes - Exalogic Elastic Cloud Software - Yes Yes Storage Grid - ZFS 7320 NAS Appliance - Yes (60TB) Yes (60TB) - Exadata Storage Servers 14 (up to 504TB) - 6 (up to 216TB) Networking - Ethernet 1 GbE / 10GbE 10GbE 10GbE - InfiniBand Fabric Yes Yes Yes- Fibre Channel SAN Connectivity - - Yes (optional)21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. © 2011 Oracle Corporation – Proprietary and Confidential 6 Configuration for Performance21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7 Configure OS and Network• Network configuration is important to data integration performance – Network MTU (TCP, Infiniband) – net core rmem_max, wmem_max • Linux OS Kernel parameters – shmmax, – shmall, – aio�max�nr, – sem, �21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8 Configure Database • Database parameters – SGA, PGA, filesystemio_options, – db_cache_size, auto dop, � • Calibrate I/O performance ‒ DBMS_RESOURCE_MANAGER.CALIBRATE_IO • Gather statistics • Run a typical workload on a typical data set ‒ Check AWR report to see top waits ‒ Check SQL Monitor report to find bottlenecks in SQL executions21 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9 Configure Mid�Tier Server • Understand bottleneck ‒ Use tools, jstack/top for example, to identify top threads • Set a proper JVM heap size ‒ Pay close attention to GC activities and memory related settings • Try –XX:+UseParallelGC, �XX:+UseConcMarkSweepGC, :NewRatio, :SurvivorRatio, etc. • For Java clients using JDBC (through Jena Adapter) ‒ Network MTU, Oracle SQL*Net parameters including SDU, TDU, SEND_BUF_SIZE, RECV_BUF_SIZE, ‒ Linux Kernel parameters: net.core.rmem_max, wmem_max, net.ipv4.tcp_rmem, tcp_wmem, �22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 0 Performance and Scalability• Scales to 100s of billions of triples (petabytes) and more � Scales linearly with Oracle database and hardware � No limitations as with other in�memory approaches• Fast loading of triples – Incremental and bulk loading• Parallelism is exploited – Load, Query, Inference• Fast access by persisting asserted and inferred triples22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1 Oracle Spatial and Graph - LUBM 200K on 3-Node RAC Sun Server X2-4 Load, Inference and Query Performance • The LUBM 200K Graph has 48+ Billion triples (edges) – Original graph has 26.6 Billion unique triples (quads) – Inference produced another 21.4 Billion triples • Data Loading Performance – Triples Loaded and Indexed Per Second (TLIPS): 273K • Inference Performance – Triples Inferred and Indexed Per Second (TIIPS): 327K • SPARQL Query Performance Setup:– Query Results Per Second (QRPS): 459K Hardware: Sun Server X2-4, 3-node RAC - Each node configured with 1TB RAM, 4 CPU 2.4GHz 10-Core Intel E7-4870) - Storage: Dual Node 7420, both heads configured as: Sun ZFS Storage 7420 4 CPU 2.00GHz 8-Core (Intel E7-4820) 256G Memory 4x SSD SATA2 512G (READZ) 2x SATA 500G 10K. Four disk trays with 20 x 900GB disks @10Krpm, 4x SSD 73GB (WRITEZ) Software: Oracle Database 11.2.0.3.0, SGA_TARGET=750G and PGA_AGGREGATE_TARGET=200G Note: Only one node in this RAC was used for performance test. Test performed in April 2013.22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2 Oracle Spatial and Graph - LUBM 200K on 3-Node RAC Sun Server X2-4 Load Performance Degrees of Data Set Quads Loaded Time Parallelism LUBM200K Load into Staging Table: 27.4 billion Quads (with duplicates) 2 hrs 6 min. DOP = 66 Load into the RDF graph: 26.6 billion Quads (unique quads) 22 hrs 23 min. DOP = 80•Data loading included de�duplication and building of two indexes on the quads. A significant portion (11 hrs 18 minutes) of the total load time was spent in building the two indexes. •Loading from the 198 compressed N�Quad formatted files was done by defining an External Table (with gunzip preprocessor) on those files and then using sem_apis.LOAD_INTO_STAGING_TABLE •Load flags => parse mbv_method=shadow parallel=80 parallel_create_index DEL_BATCH_DUPS=USE_INSERT Setup: Hardware: Sun Server X2-4, 3-node RAC - Each node configured with 1TB RAM, 4 CPU 2.4GHz 10-Core Intel E7-4870) - Storage: Dual Node 7420, both heads configured as: Sun ZFS Storage 7420 4 CPU 2.00GHz 8-Core (Intel E7-4820) 256G Memory 4x SSD SATA2 512G (READZ) 2x SATA 500G 10K. Four disk trays with 20 x 900GB disks @10Krpm, 4x SSD 73GB (WRITEZ) Software: Oracle Database 11.2.0.3.0, SGA_TARGET=750G and PGA_AGGREGATE_TARGET=200G Note: Only one node in this RAC was used for performance test. Test performed in April 2013.22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3 Oracle Spatial and Graph - LUBM 200K on 3-Node RAC Sun Server X2-4 Inference Performance Data Set (# quads) Quads Inferred Time Degrees of Parallelism LUBM 200K (27.4B) 21.4 billion 17 hrs 56 min. DOP = 80Inference included building 2 indexes on the inferred triples that took a little over 5 hrs.Inference Semantics: OWLPrime + the following components: INTERSECT, INTERSECTSCOH, SVFH, THINGH, THINGSAM, UNION Inference Options: RAW8=T, Dynamic Sampling level 1Setup: Hardware: Sun Server X2-4, 3-node RAC - Each node configured with 1TB RAM, 4 CPU 2.4GHz 10-Core Intel E7-4870) - Storage: Dual Node 7420, both heads configured as: Sun ZFS Storage 7420 4 CPU 2.00GHz 8-Core (Intel E7-4820) 256G Memory 4x SSD SATA2 512G (READZ) 2x SATA 500G 10K. Four disk trays with 20 x 900GB disks @10Krpm, 4x SSD 73GB (WRITEZ) Software: Oracle Database 11.2.0.3.0, SGA_TARGET=850G and PGA_AGGREGATE_TARGET=150G Note: Only one node in this RAC was used for performance test. Test performed in April 2013.22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4 Oracle Spatial and Graph - LUBM 200K on 3-Node RAC Sun Server X2-4 Query Performance Ontology LUBM 200K – 48B quads LUBM Benchmark Queries 27.4 billion asserted quads 26.6 billion inferred quads Query Q1 Q2 Q3 Q4 Q5 Q6 Q7# answers 4 494.5M 6 34 719 2.067B 67 OWLPrime Time (sec) 0.01 1160 0.01 609.22 0.04 1105.07 712.48 & new Query Q8 Q9 Q10 Q11 Q12 Q13 Q14 inference # answers 7790 4 224 15 components 53.86M 926088 1.568B Time (sec) 1228.95 3139.28 0.01 0.01 1.2 208.88 946.01DOP = 40, Dynamic sampling level = 6. 4.18 Billion answers generated in 2.53 hrs on a single node.Setup: Hardware: Sun Server X2-4, 3-node RAC - Each node configured with 1TB RAM, 4 CPU 2.4GHz 10-Core Intel E7-4870) - Storage: Dual Node 7420, both heads configured as: Sun ZFS Storage 7420 4 CPU 2.00GHz 8-Core (Intel E7-4820) 256G Memory 4x SSD SATA2 512G (READZ) 2x SATA 500G 10K. Four disk trays with 20 x 900GB disks @10Krpm, 4x SSD 73GB (WRITEZ) Software: Oracle Database 11.2.0.3.0, SGA_TARGET=850G and PGA_AGGREGATE_TARGET=150G Note: Only one node in this RAC was used for performance test. Test performed in April 2013.22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5 Parallel Execution Performance on M8000• LUBM 25K local inference on Sun M8000 • 6.1B+ quads (3.4B asserted, 2.7B inferred) 400 345 350300 Oracle’s Parallel Execution 250 is completely transparent! 214 • Cross CPUs/Cores on a single node 200 180 • Cross multiple nodes in a cluster150 150 145Time (in minutes) minutes) (in(in TimeTime 121 100500 0 20 40 60 80 100 120 140 Parallel Degree (DOP)22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6 Inference Performance on Exadata V2Data Set (# triples) Triples Inferred Time Degrees of Parallelism LUBM 100K (13B) 5B 1h, 58’ 1 DOP = 32 LUBM 25K (3.3B) 2.7B 4h, 7’ 2 DOP = 32 LUBM 8K (1.1B) 869M 46’ 2 DOP = 641 Preliminary result: 1 round of OWLPrime (OWL Horst semantics)2 Inference: OWLPrime + components: INTERSECT,INTERSECTSCOH,SVFH,THINGH,THINGSAM,UNIONSetup:Hardware: Full Rack Sun Oracle Database Machine X2�2 (8 nodes, 72GB RAM per node), and Exadata Storage ServerStorage required: LUBM8K: 330GB or LUBM25K 1TB + 110GB temp table spaceSoftware: Oracle Database 11.2.0.1.0 + Patch 9819833: SEMANTIC TECHNOLOGIES 11G R2 FIX BUNDLE 2 Each node: SGA_TARGET=32G and PGA_AGGREGATE_TARGET=31G22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7 Query Performance on Exadata V2 Auto DOP used. 465,849,803 answers generated for LUBM 25K in 274.2 sec.Ontology LUBM 25K LUBM Benchmark Queries 3.3 billion triples & 2.7 billion inferred Query Q1 Q2 Q3 Q4 Q5 Q6 Q7# answers 4 2528 6 34 719 260M 67Complete? YYYYYYYTime 0.01 20.65 0.01 0.01 0.02 23.07 4.99 OWLPrime (sec) & new inference Query Q8 Q9 Q10 Q11 Q12 Q13 Q14 components# answers 7790 6.8M 4 224 15 0.11M 197MComplete? Y Y Y Y Y Y YTime 0.48 203.06 0.01 0.02 0.02 2.40 19.45 (sec)22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8 Part 7: Demonstration Managing and Mining RDF Graph Data with Oracle Spatial and Graph22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9 Tools: Ontology Editing and Engineering http://protege.stanford.edu/ Protégé 4.1 Integration with Protégé • Open source ontology editor • Java APIs (Jena Adapter) allows an easy integration of Protégé with Oracle Spatial and Graph23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 0 Tools: Ontology Editing and EngineeringIntegration with TopQuadrant Tools• Commercial tool • Java APIs (Jena Adapter) allows an easy integration of TopQuadrant TopBraid Suite with Oracle Spatial and Graph23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1 Tools: Navigation and Visualization of Graphs http://www.cytoscape.org/ Integration with Cytoscape• Open source visualizer • Visualizes RDF and OWL stored in Oracle Database • Enables Fish Eye views by building summary�detail graphs • Performant with large RDF data sets23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2 Tools: Navigation and Visualization of GraphsTom Sawyer Perspectives Integration with Visualization• Commercial tool • Standards�based SPARQL web service endpoint allows an easy integration with Tom Sawyer’s Perspective23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3 Tools: Reporting RDF in Business IntelligenceIntegration with Oracle Business Intelligence EE • Standards based SPARQL web service endpoint and SPARQL Gateway feed XML (transformed from SPARQL XML response) to BI23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4 Tools: Statistical Graph Analytics Oracle R Enterprise feature of Oracle Advanced AnalyticsIntegration with Oracle R Enterprise• Open source language • Statistical computing and chart for graph data • Produces publication quality plots • Highly extensible with open source R packages23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5 Tools: Discovery & Predictive Analysis Oracle Data Mining feature of Oracle Advanced AnalyticsProblem Classification Sample Problem Anomaly Detection Given demographic data about a set of customers, identify customer purchasing behavior that is significantly different from the normAssociation Rules Find the items that tend to be purchased together and specify their relationship – market basket analysisClustering Segment demographic data into clusters and rank the probability that an individual will belong to a given clusterFeature Extraction Given demographic data about a set of customers, group the attributes into general characteristics of the customersF1 F2 F3 F423 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6 Using Oracle Data Mining with RDF Graph Data • Make the semantic data available to a data mining tool in an appropriate format – Turn a semantic data store into yet another data source for DM tool create view N_COUNTRY_BD_RATE as select name , to_number(brate) as brate , to_number(drate) as drate , to_number(popu) as population , to_number(mig) as net_migration_rate , to_number(imr) as infant_motal_rate , to_number(leab) as life_expectancy from table(sem_match('{ SQL ?subject <http://www.cia.gov/cia/publications/factbook#Birth_rate> ?brate . ?subject <http://www.cia.gov/cia/publications/factbook#Death_rate> ?drate . ?subject <http://www.cia.gov/cia/publications/factbook#Name> ?name . ?subject <http://www.cia.gov/cia/publications/factbook#Population> ?popu . SPARQL ?subject <http://www.cia.gov/cia/publications/factbook#Net_migration_rate> ?mig . ?subject <http://www.cia.gov/cia/publications/factbook#Infant_mortality_rate> ?imr . ?subject <http://www.cia.gov/cia/publications/factbook#Life_expectancy_at_birth> ?leab . }‘ … ))23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7 Using Oracle Data Mining with RDF Graph Data • Tie it all together – Turn a semantic data store into yet another data source to DM – Follow the conventional DM process: • Data preparation, build/evaluate model, deployment • This is one example of what you may get:23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8 Using Oracle Data Mining with RDF Graph Data • Tie it all together – Turn a semantic data store into yet another data source to DM – Follow the conventional DM process: • Data preparation, build/evaluate model, deployment • Some Mining results can be saved back as RDF into Oracle database Anomaly Detection output in SQLConvert into RDF :AbnormalCase1 :hasSubject :Dominica . :AbnormalCase1 :probability “0.54”23 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9 Oracle Enterprise Manager Understand exactly what is the going on • Configuration• Storage• Security• Performance – Real time monitor – CPU – Memory – I/O – Sessions – Activity – Workload – � • � 24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 0 Oracle Enterprise Manager Understand exactly what is the going on • Configuration• Storage• Security• Performance – Real time monitor – CPU – Memory – I/O – Sessions – Activity – Workload – � • � 24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1 Part 8: SQL�Based Graph Analytics24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2 Why Use SQL for Graph Analytics?• A standard language that runs every platform • Modern RDBMS provides many cool features – Cost based optimizer – Parallel execution – Partition pruning – Hierarchical queries, Match_Recognize – Smart scans, Table Compression, Index Compression, IMDB – Temporal query – Transparent query rewrite (OLS/VPD) – No need to worry about memory management – No need to worry about workload management in a cluster • Even NoSQL folks want to have a SQL interface24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3 Shortest Path• Why is shortest path (SP) important? – SP is a good measure of how closely related things are – SP is a foundation to many other algorithms and metrics • Distance, diameter, radius, eccentricity • Closeness Centrality – In G, if John is closer to others than Mary, then John is more important because John can spread information quicker. • Betweenness Centrality – In G, if John locates on more shortest paths between other nodes than Mary, then John is more important because John can better control information flow • Many existing algorithms – Dijkstra, Bellman Ford, BFS, A*, Johnson, � • Proposed solution: SQL�Based Dijkstra (SBD)24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. http://en.wikipedia.org/wiki/Centrality 4 Performance of Finding Shortest Path in SQL• Twitter Graph • Performance on Desktop • Performance on ExadataNodes 41,652,230 Random Tests Run1 Run2 Run3 Run1 Run2 Run3Edges 1.4+ Billion 1: v766 v12345 2.84s 1.95s 1.91s 1.11s 1.06s 1.15sMax Out Degree 2,997,469 2: v766 v667023 2.41s 1.51s 1.44s 0.86s 0.85s 0.73s Max In Degree 770,155 3: v12 v667023 4.55s 4.46s 4.54s 6.61s 3.84s 3.82s Weights: integers in [1, 100] 4: v667023 v12 20.09s 1.55s 1.12s 3.91s 0.88s 0.84s5: v20 v13220032 8.62s 7.36s 7.42s 6.52s 5.85s 5.84s6: v32309592 v14293310 153.73s 3.93s 2.78s 2.12s 2.12s 2.42s7: v702013 v37083737 14.99s 1.85s 1.62s 1.0s 1.02s 1.08s8: v12041332 v37083590 1.55s 1.50s 1.03s 0.92s 0.92s 1.0s9: v57640358 v19945621 11.91s 0.77s 0.65s 0.56s 0.54s 0.53s10: v19945621v57640358 315.59s 93.44s 5.39s 4.05s 3.59s 3.76s• Hardware: 8GB RAM Desktop, 3-SATA disk, quad core • Hardware: Exadata, 128 cores, 128G SGA, 64G PGA (shared with 3 other databases) Test performed in Dec 2012 ~ Feb 201324 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5 Page Ranking• Page Ranking (PR) definition (from Google’s two founders) 4 3 PR(A) = (1-d) + d (PR(T1)/C(T1) + ... + PR(Tn)/C(Tn)) T2 where PR(A) is the Page Rank of page (node or vertex) A, d is a T1 damping factor, C(Ti) is the out degree of page Ti– PR is a variant of Eigenvector centrality which measures influence A of a node in a network – Intuitions: 1 1 • A vertex (page) is more important if there are more incoming edges T4 • A vertex is more important if there’re in�edges from more important vertexes T3 • Contribution of a vertex is diluted if out�degree is high– Proposed Solution: iteratively update Page Ranking of vertexes using SQL and parallel execution24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6 Performance of Page Ranking in SQL • Twitter Graph • Performance evaluation Nodes 41,652,230 • 50 iterations of PR calculation took less than 39 minutes Edges 1.4+ Billion • DOP = 128 Max Out Degree 2,997,469 • After 10 iterations, the average PR change is less than 0.003 Max In Degree 770,155 • Hardware: Exadata, 128 cores, 128G SGA, 64G PGA (shared with 3 Weights: not relevant other databases) 0.5 0.45 0.4 0.35 0.3 Avg | PR(v) – PRprev(v) | 0.25 0.2 0.15 0.1 0.05 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 Test performed in Dec 2012 ~ Feb 2013 Iterations24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7 Summary• Graph is a flexible, intuitive data modeling • W3C standards�based RDF Semantic Graph • Tools that work with Graph data • SQL�Based Graph Analytics• Oracle provides an efficient, secure, and scalable platform for – Managing graph data – Performing graph analytics24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8 24 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9 25 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 0 </div> </article> </div> </div> </div> <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.6.1/jquery.min.js" crossorigin="anonymous" referrerpolicy="no-referrer"></script> <script> var docId = '88583eeede4055880eaa869e7028e3fc'; var endPage = 1; var totalPage = 250; var pfLoading = false; window.addEventListener('scroll', function () { if (pfLoading) return; var $now = $('.article-imgview .pf').eq(endPage - 1); if (document.documentElement.scrollTop + $(window).height() > $now.offset().top) { pfLoading = true; endPage++; if (endPage > totalPage) return; var imgEle = new Image(); var imgsrc = "//data.docslib.org/img/88583eeede4055880eaa869e7028e3fc-" + endPage + (endPage > 3 ? ".jpg" : ".webp"); imgEle.src = imgsrc; var $imgLoad = $('<div class="pf" id="pf' + endPage + '"><img src="/loading.gif"></div>'); $('.article-imgview').append($imgLoad); imgEle.addEventListener('load', function () { $imgLoad.find('img').attr('src', imgsrc); pfLoading = false }); if (endPage < 5) { adcall('pf' + endPage); } } }, { passive: true }); if (totalPage > 0) adcall('pf1'); </script> <script> var sc_project = 11552861; var sc_invisible = 1; var sc_security = "b956b151"; </script> <script src="https://www.statcounter.com/counter/counter.js" async></script> </html>