Evaluaon of distributed open source soluons in CERN database use cases

HEPiX, spring 2015 Kacper Surdy IT-DB-DBF M. Grzybek, D. L. Garcia, Z. Baranowski, L. Canali, E. Grancher

Movaons

• propose beer suited soluon for big volumes of read-only data • open new possibilies for data analysis • offload Oracle in data warehouse workloads • our users ask for this!

3 Hadoop evaluaon

• One interface (SQL) • Different use cases from CERN • LHC log, Controls data, Atlas jobs manager (PANDA) • Different query engines • Different approaches for storing data

4 Query engines • SQL -> MapReduce jobs Cloudera Impala • Custom SQL planner and executor • Uses Hive metastore Apache SQL • Spark lightweight threading execuon • Can use Hive metastore

5 Data formats Data storage format and compression type make a difference • data volume • IO vs. CPU bound throughput • benefits of columnar storage • paroning

6 Data formats CSV • human readable

7 ACCLOG 8 days - file size comparison 1800 GB

1600 GB

1400 GB 1240 GB 1200 GB

1000 GB

800 GB 649 GB 600 GB

400 GB 238 GB 200 GB 109 GB

0 GB CSV SequenceFile Avro Parquet no compression snappy Oracle

8 Data formats CSV • human readable SequenceFile • flat set of binary stored (key, value) pairs

9 ACCLOG 8 days - file size comparison 1800 GB

1600 GB 1545 GB

1400 GB 1240 GB 1200 GB

1000 GB

800 GB 649 GB 600 GB

400 GB 238 GB 265 GB 200 GB 109 GB 117 GB

0 GB CSV SequenceFile Avro Parquet no compression snappy bzip2 Oracle

10 Data formats CSV • human readable SequenceFile • flat set of binary stored (key, value) pairs Apache Avro • binary serializing standard

11 ACCLOG 8 days - file size comparison 1800 GB

1600 GB 1545 GB

1400 GB 1240 GB 1200 GB

1000 GB

800 GB 649 GB 600 GB 542 GB

400 GB 265 GB 238 GB 226 GB 171 GB 200 GB 109 GB 117 GB

0 GB CSV SequenceFile Avro Parquet no compression snappy bzip2 Oracle

12 ACCLOG 8 days - union query execuon me 2000 s 1800 s 1800 s

1600 s

1400 s

1200 s

1000 s

800 s 682 s 572 s 600 s

400 s 216 s 200 s 113 s 118 s

0 s SequenceFile Avro Parquet no compression snappy bzip2

13 Data formats CSV • human readable SequenceFile • flat set of binary stored (key, value) pairs Apache Avro • binary serializing standard Parquet • binary columnar-storage

14 ACCLOG 8 days - file size comparison 1800 GB

1600 GB 1545 GB

1400 GB 1240 GB 1200 GB

1000 GB

800 GB 649 GB 600 GB 542 GB 558 GB

400 GB 265 GB 288 GB 238 GB 226 GB 171 GB 200 GB 109 GB 117 GB

0 GB CSV SequenceFile Avro Parquet no compression snappy bzip2 Oracle

15 ACCLOG 8 days - union query execuon me 2000 s 1800 s 1800 s

1600 s

1400 s

1200 s

1000 s

800 s 682 s 572 s 600 s

400 s 328 s 216 s 200 s 113 s 118 s 117 s

0 s SequenceFile Avro Parquet no compression snappy bzip2

16 Potenal columnar store benefit

Ntuples data exeucon comparison

500 467.31 450

400

350

300

250

200 Execuon me [s] 150 117 100

50 23.39 0 Parquet Avro Oracle

17 Scalability - Impala

ACCLOG data 300

250

200

150 Measured Ideal

Execuon me [s] 100

50

0 4 8 12 16 Number of nodes

18 Scalability - Hive

ACCLOG data – Hive scalability 600

500

400

300 Measured Ideal

Execuon me [s] 200

100

0 4 8 12 16 Number of nodes

19 Scalability – Spark SQL

ACCLOG data 200 180 160 140

120 100 Measured 80 Ideal 60

Execuon me [s] 40 20 0 4 8 12 16 Number of nodes

20 Summary

• Data format choice is a key • Heterogeneous data in the same place • No maer what interface -> it scales

21