
Getting Started with Apache® Ignite™ as a Distributed Database VALENTIN KULICHENKO Lead Architect GridGain Systems, Inc. © 2018 GridGain Systems, Inc. Agenda • Apache Ignite as a Distributed Database • Connectivity • Data Definition Language • Data Manipulation Language • Collocated and Non-collocated Joins • Under the Hood • Durable Memory Architecture • Native Persistence • Data Storage Configuration • Data Regions • Data Eviction and Page Replacement • Performance Tips • Demo • Q&A © 2017 GridGain Systems, Inc. What is Apache Ignite? Financial Telco Travel & E-Commerce Pharma & IoT Services Logistics Healthcare SQL Key/Value Transactions Compute Services Streaming ML Memory-Centric Storage Ignite Native Persistence Third-Party Persistence (Flash, SSD, Intel 3D XPoint) (RDBMS, HDFS, NoSQL) © 2017 GridGain Systems, Inc. Ignite as a Distributed Database © 2017 GridGain Systems, Inc. Distributed SQL Cross-platform Java .NET C++ BI Tools Compatibility SELECT, UPDATE, DDL, DML Support JDBC ODBC SQL API INSERT, MERGE, DELETE, CREATE and ALTER Apache Ignite Cluster Indexes in DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY Dynamic RAM or Disk Scaling Server Node Server Node Server Node © 2017 GridGain Systems, Inc. Connectivity • JDBC // Register JDBC driver. • ODBC Class.forName("org.apache.ignite.IgniteJdbcThinDriver"); // Open the JDBC connection. • REST Connection conn = DriverManager.getConnection("jdbc:ignite:thin://192.168.0.50"); • Java, .NET and C++ APIs ./sqlline.sh --color=true --verbose=true -u jdbc:ignite:thin://127.0.0.1/ © 2017 GridGain Systems, Inc. Data Definition Language • CREATE/DROP TABLE CREATE TABLE `city` ( `ID` INT(11), • CREATE/DROP INDEX `Name` CHAR(35), `CountryCode` CHAR(3), • ALTER TABLE `District` CHAR(20), `Population` INT(11), • Changes Durability PRIMARY KEY (`ID`, `CountryCode`) ) WITH "template=partitioned, backups=1, affinityKey=CountryCode"; • Ignite Native Persistence https://apacheignite-sql.readme.io/docs/ddl © 2017 GridGain Systems, Inc. Data Manipulation Language • ANSI-99 specification • Fault-tolerant and consistent SELECT country.name, city.name, MAX(city.population) as max_pop FROM country JOIN city ON city.countrycode = country.code • INSERT, UPDATE, DELETE WHERE country.code IN ('USA','RUS','CHN') • SELECT GROUP BY country.name, city.name ORDER BY max_pop DESC LIMIT 3; • JOINs • Subqueries https://apacheignite-sql.readme.io/docs/dml © 2017 GridGain Systems, Inc. Collocated Joins Ignite Node Toronto Montreal 2 Canada Ottawa Calgary 1 Ignite Node 3 2 Mumbai India New Delhi 1. Initial Query 2. Query execution over local data 3. Reduce multiple results in one © 2017 GridGain Systems, Inc. Non-Collocated Joins Ignite Node Toronto Mumbai 2 Canada Calgary 1 Montreal 3 Mumbai Ottawa 4 2 Ignite Node Montreal India Ottawa 1. Initial Query 2. Query execution (local + remote data) New Delhi 3. Potential data movement 4. Reduce multiple results in one © 2017 GridGain Systems, Inc. Under the Hood © 2017 GridGain Systems, Inc. Durable Memory Automatic Defragmentation Ignite Cluster Predictable memory Off-heap Removes DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY consumption noticeable GC pauses Fully Transactional Stores Superset (Write-Ahead Log) of Data Instantaneous Server Node Server Node Server Node Restarts © 2017 GridGain Systems, Inc. Ignite Native Persistence Server Node 1. Update 2. Persist Write-Ahead Log 3. Ack RAM 4. Checkpointing Partition File 1 Partition File N © 2017 GridGain Systems, Inc. Data Storage Configuration © 2017 GridGain Systems, Inc. Single Data Region <property name="dataStorageConfiguration"> <bean class="org.apache.ignite.configuration.DataStorageConfiguration"> <property name="defaultDataRegionConfiguration"> <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> <property name="maxSize" value="#{4L * 1024 * 1024 * 1024}"/> <!-- 4 GB. --> </bean> </property> </bean> </property> Default Data Region (4 GB) Cache 1 Cache 2 Cache 3 Cache 4 © 2017 GridGain Systems, Inc. Multiple Data Regions <property name="dataStorageConfiguration"> <bean class="org.apache.ignite.configuration.DataStorageConfiguration"> <property name="dataRegionConfigurations"> <list> <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> <property name="name" value="DATA_REGION_4_GB"/> <property name="maxSize" value="#{4L * 1024 * 1024 * 1024}"/> <!-- 4 GB. --> </bean> <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> <property name="name" value="DATA_REGION_128_GB"/> <property name="maxSize" value="#{128L * 1024 * 1024 * 1024}"/> <!-- 128 GB. --> </bean> </list> </property> </bean> </property> DATA_REGION_128_GB DATA_REGION_4_GB Large Cache 1 Small Cache Large Cache 2 © 2017 GridGain Systems, Inc. Enabling Data Eviction <property name="dataStorageConfiguration"> <bean class="org.apache.ignite.configuration.DataStorageConfiguration"> <property name="defaultDataRegionConfiguration"> <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> <property name="maxSize" value="#{4L * 1024 * 1024 * 1024}"/> <!-- 4 GB. --> <property name="pageEvictionMode" value="RANDOM_LRU"/> <property name="evictionThreshold" value="0.9"/> </bean> </property> </bean> </property> • Eviction Modes: • DISABLED • RANDOM_LRU • RANDOM_2_LRU © 2017 GridGain Systems, Inc. Enabling Native Persistence <property name="dataStorageConfiguration"> <bean class="org.apache.ignite.configuration.DataStorageConfiguration"> <property name="defaultDataRegionConfiguration"> <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> <property name="maxSize" value="#{4L * 1024 * 1024 * 1024}"/> <!-- 4 GB. --> <property name="persistenceEnabled" value="true"/> </bean> </property> <property name="walMode" value="LOG_ONLY"/> <property name="checkpointFrequency" value="60000"/> <!-- 60 sec. --> </bean> </property> • WAL Modes: • DEFAULT (full sync) • LOG_ONLY • BACKGROUND • NONE © 2017 GridGain Systems, Inc. Performance Tips • Leave memory space for heap and OS • Use production ready SSDs • Have separate physical disk for WAL • Advanced data storage configuration: • Change page size • Enable write throttling • Increase checkpoint buffer size https://apacheignite.readme.io/docs/durable-memory-tuning © 2017 GridGain Systems, Inc. Demo © 2017 GridGain Systems, Inc. Resources • GridGain in-memory computing resources – Learning: https://www.gridgain.com/resources/in-memory-computing-resources – In-Memory Computing Summit: https://www.imcsummit.org/ – Local Events: https://www.gridgain.com/company/news/events – Customer Stories: https://www.gridgain.com/customers/customers-and-industries • Getting started with Apache Ignite: – https://ignite.apache.org/ – Getting Started doc: https://apacheignite.readme.io/docs/getting-started – GridGain Web Console (free): https://www.gridgain.com/resources/manage – Apache Ignite User Forum: http://apache-ignite-users.70518.x6.nabble.com/ – Stack Overflow: https://stackoverflow.com/questions/tagged/ignite • Getting started with GridGain: – https://www.gridgain.com/ – https://www.gridgain.com/resources/download © 2018 GridGain Systems, Inc. HAPPY VALENTIN’S DAY! VALENTIN KULICHENKO Lead Architect GridGain Systems, Inc. © 2018 GridGain Systems, Inc..
Details
-
File Typepdf
-
Upload Time-
-
Content LanguagesEnglish
-
Upload UserAnonymous/Not logged-in
-
File Pages22 Page
-
File Size-