– Amazon Neptune

Technical Evangelist @ AWS [email protected]

•

• Amazon Neptune

• Amazon Neptune •

& IT

Challenges Building Apps- with Highly Connected Data

RDF ( PROPERTY GRAPH) (RESOURCE DESCRIPTION FRAMEWORK)

Apache TinkerPop™ W3C Gremlin Traversal Language SPARQL Query Language

(/)

• Vertex / User User

• Edge name: FRIEND name: • Bill Since 11/29/16 Sarah

•

• Apache TinkerPop

• Gremlin

Amazon Neptune Tinkerpop Gremlin 3.3.0(20178) Gremlin

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TinkerPop User //Connect to Neptune and receive a remote graph, g. name: Bill user1 = g.addVertex (id, 1, label, "User", "name", "Bill"); user2 = g.addVertex (id, 2, label, "User", "name", "Sarah"); FRIEND ... user1.addEdge("FRIEND", user2, id, 21); User name: Sarah

Gremlin (Apache TinkerPop 3.3)

name: rdf:type contacts:User; Bill contact:name: ”Bill” .

• (IRIs)

IRI

• IRI • RDFRDFXML • IRI“”

FRIEND contacts:friend #1 #2 (IRI) . 2

User @prefix contacts: . name: Bill rdf:type contacts:User; contact:name: ”Bill” .

contacts:friend . FRIEND

User rdf:type contacts:User; name: contact:name: ”Sarah” . Sarah

RDF (Turtle Serialization)

Customers Order

CompanyName: OrderDate: Acme 8/1/2017 … PURCHASED …

HAS_DETAILS Product Order ProductName: HAS_PRODUCT Details “Echo” … UnitPrice: $179.99 …

SUPPLIES Supplier

CompanyName: “Amazon” …

Echo

SELECT distinct c.CompanyName FROM customers AS c JOIN orders AS o ON /* Join the customer from the order */ (c.CustomerID = o.CustomerID) JOIN order_details AS od /* Join the order details from the order */ ON (o.OrderID = od.OrderID) JOIN products as p /* Join the products from the order details */ ON (od.ProductID = p.ProductID) WHERE p.ProductName = ’Echo'; /* Find the product named ‘Echo’ */

PREFIX sales_db: SELECT distinct ?comp_name WHERE { ?customer ?order ; #customer graph pattern ?comp_name . #orders graph pattern ?order ?order_d . #order details graph pattern ?order_d ?product . #products graph pattern ?product “Echo” . }

* : http://www.playnexacro.com/index.html#show:article

Echo

/* All products named ”Echo” */ g.V().hasLabel(‘Product’).has('name',’Echo') .in(’HAS_PRODUCT') /* Traverse to order details */ .in(‘HAS_DETAILS’) /* Traverse to order */ .in(’HAS_ORDER’) /* Traverse to Customer */ .values(’CompanyName’).dedup() /* Unique Company Name */

• gremlinpython pip3 install gremlinpython --user

• from gremlin_python import statics from gremlin_python.structure.graph import Graph from gremlin_python.process.graph_traversal import __ from gremlin_python.process.strategies import * from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection graph = Graph() g = graph.traversal().withRemote(DriverRemoteConnection('ws://your-neptune-endpoint:8182/gremlin','g')) print(© 2017, g.VAmazon().limit(2). Web Services, Inc. ortoList its Affiliates.()) All rights reserved. AMAZON NEPTUNE

Apache TinkerPop 3 GremlinSPARQL & W3C RDF 6

Multi-AZACID 15 (TLS)

Bulk load from S3

Database Mgmt.

• VPC

• (AZs)

• AZ

• VPCAmazon Neptune

36 AZ 1 AZ 2 AZ 3 Amazon S3 Amazon (119) Neptune

Storage Storage Storage Storage Storage Storage Storage 10 GB Monitoring Node Node Node Node Node Node

64 TB Amazon S3

? () 64 () 64 ()

AZ 1 AZ 2 AZ 3 AZ 1 AZ 2 AZ 3

Amazon Amazon Neptune Neptune

Caching Caching

AZ 1 AZ 2 AZ 3 Primary PrimaryPrimaryPrimary Primary PrimaryPrimary Cluster NodeRead Read NodeNodeMaster Node NodeNode and Instance Replica Replica Monitoring • Node

•

• ( ) •

•

Database Failure App Failure Detection DNS Propagation Running

Replica-Aware App Running

Recovery

15- 20 sec 3 - 10 sec

Gremlin • UTF-8 • CSV RFC 4180 CSV

RDF • UTF-8 • (https://www.w3.org/TR/n-triples/) N -Triples (ntriples) • (https://www.w3.org/TR/n-quads/) N-Quads (nquads) • (https://www.w3.org/TR/rdf-syntax-grammar/) RDF/XML (rdfxml) • (https://www.w3.org/TR/turtle/) N-Quads (turtle)

curl -X POST \ -H 'Content-Type: application/json' \ http://your-neptune-endpoint:8182/loader -d ' { "source" : "s3://bucket-name/object-key-name", "format" : "format", "iamRoleArn" : "arn:aws:iam::account-id:role/role-name", "region" : "region", "failOnError" : "FALSE" © 2017, Amazon Web Services, Inc. or its Affiliates. All rights }'reserved.

“ Amazon Neptune”

- Tim Vanderham, CTO , Thomson Reuters Tax & Accounting

Apache TinkerPop 3 GremlinSPARQL & W3C RDF 6

• Amazon Neptune • Amazon Neptune • Amazon Neptune Tools Repo • Amazon Neptune Samples Repo