Getting Started with Apache® Ignite™ as a Distributed

VALENTIN KULICHENKO Lead Architect GridGain Systems, Inc.

© 2018 GridGain Systems, Inc. Agenda

• Apache Ignite as a Distributed Database • Connectivity • • Data Manipulation Language • Collocated and Non-collocated Joins • Under the Hood • Durable Memory Architecture • Native Persistence • Data Storage Configuration • Data Regions • Data Eviction and Replacement • Performance Tips • Demo • Q&A

© 2017 GridGain Systems, Inc. What is Apache Ignite?

Financial Telco Travel & E-Commerce Pharma & IoT Services Logistics Healthcare

SQL Key/Value Transactions Compute Services Streaming ML

Memory-Centric Storage

Ignite Native Persistence Third-Party Persistence (Flash, SSD, Intel 3D XPoint) (RDBMS, HDFS, NoSQL)

© 2017 GridGain Systems, Inc. Ignite as a Distributed Database

© 2017 GridGain Systems, Inc. Distributed SQL

Cross-platform .NET C++ BI Tools Compatibility

SELECT, UPDATE, DDL, DML Support JDBC ODBC SQL API INSERT, MERGE, DELETE, CREATE and ALTER

Apache Ignite Cluster

Indexes in DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY Dynamic RAM or Disk Scaling

Server Node Server Node Server Node

© 2017 GridGain Systems, Inc. Connectivity

• JDBC // Register JDBC driver. • ODBC Class.forName("org.apache.ignite.IgniteJdbcThinDriver"); // Open the JDBC connection. • REST Connection conn = DriverManager.getConnection("jdbc:ignite:thin://192.168.0.50"); • Java, .NET and C++ APIs

./sqlline.sh --color=true --verbose=true -u jdbc:ignite:thin://127.0.0.1/

© 2017 GridGain Systems, Inc. Data Definition Language

• CREATE/DROP TABLE CREATE TABLE `city` ( `ID` INT(11), • CREATE/DROP INDEX `Name` CHAR(35), `CountryCode` CHAR(3), • ALTER TABLE `District` CHAR(20), `Population` INT(11), • Changes Durability PRIMARY KEY (`ID`, `CountryCode`) ) WITH "template=partitioned, backups=1, affinityKey=CountryCode"; • Ignite Native Persistence

https://apacheignite-sql.readme.io/docs/ddl

© 2017 GridGain Systems, Inc. Data Manipulation Language

• ANSI-99 specification

• Fault-tolerant and consistent SELECT country.name, city.name, MAX(city.population) as max_pop FROM country JOIN city ON city.countrycode = country.code • INSERT, UPDATE, DELETE WHERE country.code IN ('USA','RUS','CHN') • SELECT GROUP BY country.name, city.name ORDER BY max_pop DESC LIMIT 3; • JOINs • Subqueries

https://apacheignite-sql.readme.io/docs/dml

© 2017 GridGain Systems, Inc. Collocated Joins

Ignite Node

Toronto Montreal 2 Canada Ottawa Calgary 1

Ignite Node 3 2

Mumbai India New Delhi 1. Initial Query 2. Query execution over local data 3. Reduce multiple results in one

© 2017 GridGain Systems, Inc. Non-Collocated Joins

Ignite Node

Toronto Mumbai 2 Canada Calgary 1

Montreal 3 Mumbai Ottawa

4 2 Ignite Node

Montreal India Ottawa 1. Initial Query 2. Query execution (local + remote data) New Delhi 3. Potential data movement 4. Reduce multiple results in one

© 2017 GridGain Systems, Inc. Under the Hood

© 2017 GridGain Systems, Inc. Durable Memory

Automatic Defragmentation Ignite Cluster

Predictable memory Off-heap Removes DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY consumption noticeable GC pauses

Fully Transactional Stores Superset (Write-Ahead Log) of Data

Instantaneous Server Node Server Node Server Node Restarts

© 2017 GridGain Systems, Inc. Ignite Native Persistence

Server Node

1. Update

2. Persist Write-Ahead Log

3. Ack

RAM 4. Checkpointing Partition File 1

Partition File N

© 2017 GridGain Systems, Inc. Data Storage Configuration

© 2017 GridGain Systems, Inc. Single Data Region

Default Data Region (4 GB)

Cache 1 Cache 2

Cache 3 Cache 4

© 2017 GridGain Systems, Inc. Multiple Data Regions

DATA_REGION_128_GB

DATA_REGION_4_GB Large Cache 1

Small Cache

Large Cache 2

© 2017 GridGain Systems, Inc. Enabling Data Eviction

• Eviction Modes: • DISABLED • RANDOM_LRU • RANDOM_2_LRU

© 2017 GridGain Systems, Inc. Enabling Native Persistence

• WAL Modes: • DEFAULT (full sync) • LOG_ONLY • BACKGROUND • NONE

© 2017 GridGain Systems, Inc. Performance Tips

• Leave memory space for heap and OS • Use production ready SSDs • Have separate physical disk for WAL • Advanced data storage configuration: • Change page size • Enable write throttling • Increase checkpoint buffer size

https://apacheignite.readme.io/docs/durable-memory-tuning

© 2017 GridGain Systems, Inc. Demo

© 2017 GridGain Systems, Inc. Resources • GridGain in-memory computing resources – Learning: https://www.gridgain.com/resources/in-memory-computing-resources – In-Memory Computing Summit: https://www.imcsummit.org/ – Local Events: https://www.gridgain.com/company/news/events – Customer Stories: https://www.gridgain.com/customers/customers-and-industries • Getting started with Apache Ignite: – https://ignite.apache.org/ – Getting Started doc: https://apacheignite.readme.io/docs/getting-started – GridGain Web Console (free): https://www.gridgain.com/resources/manage – Apache Ignite User Forum: http://apache-ignite-users.70518.x6.nabble.com/ – Stack Overflow: https://stackoverflow.com/questions/tagged/ignite • Getting started with GridGain: – https://www.gridgain.com/ – https://www.gridgain.com/resources/download

© 2018 GridGain Systems, Inc. HAPPY VALENTIN’S DAY!

VALENTIN KULICHENKO Lead Architect GridGain Systems, Inc.

© 2018 GridGain Systems, Inc.