Mining Big Data with RDF Graph Technology

Mining Big Data with RDF Graph Technology:

Xavier Lopez, Ph.D. Director, Product Mgmt. Zhe Wu, Ph.D. Consulting Member Technical Staff

1 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. "THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISION. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE."

. Big Data in Action . Introducing Oracle Spatial and Graph . Using RDF for Big Data and Mining . Tools: Oracle and 3rd party . Demo . Summary

DECIDE ACQUIRE

Analyze all your data, at once

ANALYZE ORGANIZE

Oracle R Spatial Oracle Enterprise Analytics MapReduce

Oracle Graph SQL Data Mining Analytics Analytics Parallel Processing Engine

XML Relational OLAP Spatial Data Layer Graph RDF Media

• Highlights existing graph capabilities in Oracle Spatial – Network Data Model graph since Oracle 10gR1 – RDF semantic graph since Oracle 10gR2

• Addresses increasing market demand for graph database capability – Social Network Graph database popularity – Multimodal and integrated transportation, utility and communications networks

• Network Data Model graph – Manages logical / spatial networks in database – Persists link/node structure, connectivity and direction – Supports constraints at link and node level – Logically partitioning network graphs for scalability

• RDF Semantic graph – Enterprise class RDF Graph Database – Scales to petabytes of triples – by exploiting Exadata, RAC, SQL*Loader , Parallelism, Label Security – W3C standards: RDFS, OWL2 RL, OWL2 EL, SPARQL 1.1, RDB2RDF, RDFa, SKOS – SQL, PL/SQL APIs and Java APIs (Jena/Sesame)

• Basic structure is a “triple” :Jack – [subject]  [predicate]  [object] subject • RDF can be serialized into XML – Namespaces, for example :has_Best_Friend • Schemas need not be specified in advance

predicate – No schema changes to cope with foreign vocabularies • RDF data is fully expressible as RDBMS data :Tony – RDBMS data is also fully expressible as RDF object

• Enterprise metadata framework for enterprise data & public cloud Apps

• Semantic terms link instance data with other resources and apps

RDF Semantic Metadata • Linked resources enable interoperability between apps

Instance Data

• Unified content metadata model for public clouds • Validate semantic and structural consistency

. Find related content & relations by navigating connected entities . “Reason” across entities

. Analyze content using integrated metadata - Blogs, wikis, video - Calendars, IM, voice

Oracle RDF Semantic Graph

• Integrate, archive, analyze, and act on data in social media and public clouds • Leverage enterprise content and domain vocabularies to provide context • Developing better analysis and predictions from Big Data sources

Oracle RDF Semantic Graph

• Native RDF Database • Supports SPARQL 1.1, SPARQL/SQL, GeoSPARQL • Jena, Sesame, & Joseki Web Services • W3C standards: RDFS, OWL 2 RL, OWL 2 EL, SKOS, RDF, RDB2RDF, SPARQL 1.1, RDFa • Scales with hardware – petabytes of triples • Works with OBIEE, Oracle BPM, Oracle Advanced Analytics • Exploits: Exadata, RAC, Parallelism, Label Security

• Integrated access to structured and RDF data – Join SPARQL results with relational data – Embed SPARQL query in SQL – Apply rich SQL operators (such as aggregates) on triples

• Semantic indexing (of unstructured content) – Generate RDF triples with 3rd party NLP and entity/concept extraction tools – Index generated RDF graph (1 named graph per doc) – Perform incremental updates of RDF graph as new documents entered

• Query text, spatial, temporal data using SPARQL -- GeoSPARQL

• Scales to 100s of billions of triples (petabytes) and more - Scales linearly with Oracle database and hardware - No limitations as with other in-memory approaches

• Fast loading of triples – Incremental and bulk loading

• Parallelism is exploited – Load, Query, Inference

• Fast access by persisting asserted and inferred triples

17 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Billions of Triples,Terabytes of Storage High Performance and Scalability • Lehigh Univ. Benchmark (25K) tested – 3.4 Billion triples in the model – 2.7 Billion triples inferred – Storage space including indexes: over 1 Terabytes – Load speed: 109 minutes on Sun M8000 – Inference speed: 160 minutes – Query throughput: 0.5 Billion in 9 minutes – Sun M8000, 512GB RAM, 2TB Flash array, 128 Threads – Balanced hardware is key!

– Navigation and visualization

– Graph creation and manipulation

– Reporting and charting

– Exploring & analyzing

– Automated discovery (mining) & predictive analysis

Ontology Engineering Reasoners NLP Entity Extractors

Open Source Frameworks Standards

Joseki Sesame

Applications SI / Consulting

Cytoscape Graph Visualizer Tom Sawyer’s Perspective

Open Source Commercial Product

• Powerful BI dashboards – Visually appealing – 100% thin client • Across all styles of analysis – R-OLAP, M-OLAP, Scorecards, Reporting, Collaboration, Actions • Across all data sources – Federated data access – Share, collaborate, & publish

• Open source language • Statistical computing and chart for graph data • Produces publication quality plots • Highly extensible with open source R packages

Problem Classificaon Sample Problem Anomaly Detecon Given demographic data about a set of customers, idenfy customer purchasing behavior that is significantly different from the norm

Associaon Rules Find the items that tend to be purchased together and specify their relaonship – market basket analysis

Clustering Segment demographic data into clusters and rank the probability that an individual will belong to a given cluster

Feature Extracon Given demographic data about a set of customers, group the aributes into general characteriscs of the customers

F1 F2 F3 F4

26 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. New functions in Oracle Database Release 12.1 • Open Geospatial Consortium (OGC) GeoSPARQL • Native SPARQL 1.1 query support – 40+ new query functions/operators: IF, COALESCE, STRBEFORE, REPLACE, ABS, – Aggregates: COUNT, SUM, MIN, MAX, AVG, GROUP_CONCAT, SAMPLE – Subqueries – Value Assignment: BIND, GROUP BY Expressions, SELECT Expressions – Negation: NOT EXISTS, MINUS – Improved Path Searching with Property Paths

• RDF views on relational tables (through RDB2RDF) – RDF views can be created on a set of relational tables and/or views – SPARQL queries access data from both a relational and RDF store – Allows filtering of data in a relational store based upon ontology – Support RDF view creation using • Direct Mapping: simple and straightforward to use • R2RML Mapping: customizations allowed

• Inference – Native OWL 2 EL inference support – User defined inferencing • Allows generation of new RDF resources • Temporal reasoning, Spatial reasoning • Web service callouts – Ladder Based Inference • Fine grained security for inference graph – Performance optimization for user defined rules – Integration with TrOWL*, an external OWL 2 reasoner • TrOWL is a transformation based, tractable reasoner for OWL 2

Applying RDF Graph Analysis with:

• Oracle Advanced Analytics • Oracle Business Intelligence EE

• Demo 1: Using OBIEE on RDF • Data: Data.gov Finance Data in RDF format

• Demo 2: Using Oracle R Enterprise on RDF Data • Data: CIA World Factbook in RDF format

SPARQL Gateway • Provide navigation, XML transformation

Joseki Web Service • Provide standard compliant WS endpoint Jena Adapter • Provide Java APIs

Oracle Database • Scalable, performant, secure RDBMS (Relational, XML, RDF, Spatial, Media, …)

• Oracle brings enterprise-class RDF semantic graph data management • Scalable, Secure, and High Performance: load, query, inference features • Supports W3C Semantic Standards • Works with structured and unstructured data • Enterprise-class Oracle tools can now mine insight from semantic data – OBIEE – Oracle Data Mining – Oracle R Enterprise • Supported by leading 3rd party semantics tools

Oracle RDF

[email protected] [email protected]

oracle.com

Date/Time Title Location Sunday, Sept 30 User Group Forum: Oracle Business Intelligence 11g 3:30pm-4:30pm Moscone West - 2006 Maps: Making Yourself Spatially Aware

Monday, Oct 1 10:45am-11:45am General Session: What’s Next for Oracle Database? Moscone North – Hall D Conference Session: Mining Big Data with RDF Graph 12:15pm-1:15pm Moscone South – 200 Technology: Discovering What You Didn’t Know General Session: Building and Managing a Private 3:15pm-4:15pm Moscone South – 103 Oracle Database Cloud BOF: Adding Maps and Location to Analytic Marriott Marquis – Club 7:15pm-8:00pm Applications: Real-World Experiences Room

47 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Spatial and Graph at OOW 2012 Sessions Date/Time Title Location Tuesday, Oct 2 JavaOne Session: Leveraging W3C Linked Data for Loosely Parc 55 - 8:30am-9:30am Coupled Application Integrations Embarcadero Conference Session: Oracle Spatial and Graph: Faster, Moscone South - 10:15am-11:15am Bigger, Better 2-D and 3-D Spatial Solutions 306 Conference Session: Integrating Oracle Database with a Social Marriott Marquis – 10:15am-11:15am Network Foothill F Wednesday, Oct 3 Oracle Exadata, Oracle Exalogic, Oracle Exalycs, and Big Data Wesn San Francisco 11:45am - 12:45pm Soluons in the Public Sector - City (Features panelists from US Census Bureau, New York City) Conference Session: Mash Up All Things Location with Marriott Marquis – 1:15pm-2:15pm Android, iPad, HTML5 Web Maps, and Spatial Club Room

. Special offer – Get certified on Oracle Spatial onsite! Test Fest @ Oracle PartnerNetwork Exchange Free for OPN Exchange participants

Date/Time Event Locaon

Monday – Thursday Test Fest @ OPN Exchange – Marrio Marquis, exam mes Cerficaon Tesng onsite for Oracle Spaal and many other Juniper Room available Oracle technologies (2 hour slots) •Pre-register for an exam slot at hp://www.oracle.com/opnexchange/learn/test-fest •Walk-ins available on a first-come, first-served basis •Proof of OPN membership and PearsonVue ID required

Date/Time Demo Locaon

Monday-Wednesday Spaal Capabilies in Oracle Spaal and Graph Moscone South, Le S-059 Monday-Wednesday Semanc RDF Graph Technologies in Oracle Spaal and Moscone South, Le Graph S-060 Monday-Wednesday Empowering Business to Drive Process Modeling Moscone South, Right S-234 Monday-Wednesday Oracle Event Processing Moscone South, Right S-230 Monday -Thursday Analyze Big Data Moscone North, Upper Lobby – N-013

• Represents Graph relationships as triples in • Support for SPARQL and mixed SPARQL compressed, partitioned tables and SQL queries • Supports quads/named graphs • Patented Table Function Rewrite converts • RDFS, OWL and user-defined inferencing, SPARQL to SQL Parallel execution and incremental modes • Packaged with enhanced Open Source • Kernel and SQL-layer enhancements for Tools: Jena, Sesame, Joseki Web Services optimized indexing and access of triple data • W3C standards RDF, RDFS, OWL2 RL, • Partitioning and local indexing by OWL2EL, SKOS, RDB2RDF, RDFa application table • Support for Oracle Exadata, RAC, • Scales to 100’s of billions of triples SQL*Loader direct path load, Parallelism, • Access from OBI EE, Oracle BPM, Oracle Oracle Label Security Advanced Analytic (R and ODM) through Oracle’s SPARQL gateway

• W3C Compliant Semantic Database – RDF, RDFS, OWL 2, SKOS, SPARQL, … • SPARQL Query – Query execution planning for optimal performance – Allow multiple RDF graphs in SPARQL queries • Inferencing – Pre-computed inferences (forward chaining) – Incorporates higher order logic and user defined rules – Supports incremental inferencing for higher availability • Jena Adapter for easy loading, querying and inferencing • Security: Fine-Grained Access Control (for each triple)

52 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. RDF Semantic Graph New Features • Named Graph: • Scalable, meaningfully grouping of triples simplifies development • Reduces the cost /improves performance of load/query/inference • RDF Views on Relational Tables • No duplication of data and storage • Simplifies semantic discovery workflows on relational schema and RDF graphs • Analytic operations and tools: • Supports SPARQL 1.1 Path Expressions, • Derives more info faster w/ graph analytics, Oracle Advanced Analytics, Oracle R Enterprise • User-provided inference gives users more ctrl over functions & optimization of inference process and results generated.

53 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. RDF Semantic Graph New Features • Document Indexing Performance Enhancements • Efficient processing of large document workloads, faster document operations w/ partitioning, parallel loading of entities extracted from documents • XML Schema, Text and Spatial Data Types: • Filter queries written in SPARQL or SQL w/ XML schema, text, spatial attributes. • Faster, more selective queries using keywords, geography/distance, XML types. • W3C and OGC Standards, Open Source • Conforms to the latest W3C and OGC standards open source frameworks • 45% improvement in query performance w/ Jena Adapter mid-tier caching • Scalable querying of distributed SPARQL endpoints.

Oracle Big Data Appliance Oracle RDF Oracle Advanced (Extract and annotate text as RDF) Semantic Graph Analytics

InfiniBand InfiniBand

Unstructured Documents RDF Models

Stream Acquire Organize Analyze & Visualize