What Is Apache Drill ?

What Is Apache Drill ?

Efficient Big Data Exploration with SQL and Apache Drill Jonatan Kazmierczak Java User Group Switzerland, Zürich, 07.02.2017 About author Jonatan.Kazmierczak (at) gmail (dot) com senior consultant at Atos Consulting Switzerland creator of Class Visualizer top rated participant in contests in programming and data science: HackerRank, TopCoder, Google Code Jam working with Java and SQL for 20 years -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- About author – cont. first rank in Java www.hackerrank.com/leaderboard/java/practice/level/1/filter/country=Switzerland/page/1 www.hackerrank.com/jonatan_k -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Agenda Introduction Demo: starting with Drill Technical details Demo: deep dive into Drill Summary, Q & A -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Introduction Computers – before www.amibay.com/showthread.php?71410-Atari-65XE-BOX-XC12-BOX-2-Quickshots -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Data – before -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Data – now -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Computers – now 32GB RAM 3TB RAM -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- What is Apache Drill ? low latency distributed schema-free SQL query engine for large-scale datasets designed to scale to several thousands of nodes and query petabytes of data at the speeds required by BI/Analytics environments -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Demo: starting with Drill Technical details Basic info Website drill.apache.org Current version 1.9.0 Query language SQL:2003 Interfaces shell, web console, JDBC/ODBC, REST API, Java API, C++ API -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Supported data sources and formats RDBMS NoSQL FS -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Features Dynamic schema discovery Flexible data model In-memory data processing (whenever possible) Extensible architecture Distributed and embedded mode -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Distributed setup Node 1 Node 3 ZooKeeper Drillbit Drillbit Node 2 Client Drillbit -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Sample query select * from dfs.demo.Ácountries.csvÁ storage plugin workspace table / view / file / document -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Config – storage plugins -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Query execution Drillbit SQL Parser Optimizer Parallelizer Executor Storage Plugin Client Foreman SQL query parse SQL query logical plan optimize logical plan physical plan parallelize physical plan execution tree (fragments) execute fragments fetch data data results results Client Foreman SQL Parser Optimizer Parallelizer Executor Storage Plugin -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Drill inside over 0x2000 classes -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Demo: deep dive into Drill Summary Advantages Easy to start working with Concept of SQL-on-Anything Using standard SQL -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Disadvantages Partially implemented or unfinished features Lacks in documentation -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Use cases Data exploration Data transformation BI / Data analytics Applicable Not applicable -- SQL and Apache Drill -- Jonatan Kazmierczak -- JUG CH 2017 -- Questions Thank you Jonatan.Kazmierczak (at) gmail (dot) com Son-of-God.info.

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    27 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us