DAY 1 DAY 3 Course Overview Message Processing with Apache Kafka  What is Hadoop and HDFS?  What is Apache Kafka?  What is Data Ingestion?  Apache Kafka Overview  Use cases for SQOOP, FLUME and KAFKA  Scaling Apache Kafka Data ingestion from RDBMS using SQOOP  Apache Kafka Cluster Architecture  What is SQOOP  Apache Kafka Command Line Tools  How SQOOP works – Map only Job Kafka Cluster Basics  SQOOP Tools  Working with Kafka Command line tool  List-databases  Create and manage topics  List-tables  Starting Producer  Eval  Starting Consumer  import DAY 4 DAY 2 Integrating Apache Flume and Apache Kafka SQOOP Tools – continue  Overview  Import-all-tables  Use Cases  Export  Configuration – Kafka as Source, Sink and Channel  Job SPARK Streaming with Kafka  Merge  Spark Streaming basics SQOOP use cases  Connecting Apache kafka with Spark Streaming Capturing Data with Apache Flume  Process Apache Kafka messages with Apache Spark  What is Apache Flume? streaming  Basic Flume Architecture  Using Kafka as a direct Data source  Flume Sources DAY 5  Flume Sinks Twitter Data Analysis with Kafka  Flume Channels  Getting data with Kafka from twitter Consumer and Producer Java API  Flume Configuration Getting Started with Flume  Create custom consumer  Various Flume Source configuration  Create custom producer  Various Flume Sink configuration  Running custom producer and Consumer Flume Configuration to get data from TWITTER