Building a Data Pipeline with Pentaho – from Ingest to Analytics

Building a Data Pipeline with Pentaho – From Ingest to Analytics

Bruce Berry Senior Training and Development Specialist, Global Learning September 2018

Evolution of Business Intelligence

Pentaho Tools

Hands-on Demonstration: Data Source to Dashboard

§ Challenge: Combining data from applications or source systems, databases, and files for reporting purposes

§ Solution: ‒ ETL (Extract, Transform, Load) ‒ Data warehouse/lake ‒ Reporting

§ Challenge: Provide reporting tools to non-technical business users

§ Solution: ‒ ETL (Extract, Transform, Load) ‒ Data warehouse/lake ‒ Data model (known reporting need) ‒ Self-service reporting

§ Challenge: Multi-dimensional analytics and visualizing data

§ Solution: ‒ ETL ‒ Data Warehouse/Lake ‒ OLAP Cube (known reporting needs) ‒ Self-service analytics, dashboarding

§ OLAP provides users a multi-dimensional, aggregated view of data

§ OLAP cube ‒ Measures (sales; quantity) ‒ Dimensions ‒ Hierarchies (geography; time) ‒ Levels (country>state>city; year>quarter>month)

§ Challenge: Reporting needs are not known or predefined

§ Solution: ‒ ETL ‒ OLAP cube (“on the fly”) ‒ Analytics, Dashboarding

§ Challenge: Incorporate cloud-based and streaming data in reporting and analytics

§ Solution: ‒ ETL ‒ Blend data ‒ Data service ‒ OLAP cube (“on the fly”) ‒ Analytics, dashboarding ‒ Machine learning, predictive analytics

§ Pentaho Data Integration (PDI) ‒ ETL ‒ Blend data ‒ Data service ‒ Data model or OLAP cube (“on the fly”) ‒ Machine learning ‒ Predictive analytics

§ Schema Workbench ‒ OLAP cube

§ Pentaho Analyzer ‒ Self-service analytics, visualizations

§ CTools ‒ Community Data Access ‒ Community Dashboard Editor ‒ Dashboarding

§ Metadata Editor ‒ Data model

§ Interactive Reports ‒ Self-service reporting

§ Overview ‒ In this guided demonstration, we will: ‒ Review a Pentaho Data Integration (PDI) transformation that obtains data on energy generation and usage around the world, prepares the data for analytics by building a data model (cube), and publishes the data to the repository as a data service ‒ Review a PDI job that runs the transformation and publishes the cube to the repository so it can be used for analytics ‒ Use Analyzer to analyze and visualize the data ‒ View an interactive dashboard that presents several views of the data

ACCESS STREAMLINE VISUALIZE DELIVER When & Where All Enterprise Information Delivery & Report Information Users Need It Data Sources In Any Style

CLOUD PDI-Transformation ANALYZER CTOOLS • Obtain worldwide • Prepare and publish • Analyze and visualize • Deliver the data in an energy data data service the data interactive dashboard

PDI-Job • Run transformation • Create and publish data model/cube

§ Hitachi Vantara Web Site ‒ https://www.hitachivantara.com ‒ Innovate with Data and Analytics ‒ https://www.hitachivantara.com/en-us/solutions/data-analytics.html ‒ Pentaho Data Integration ‒ https://www.hitachivantara.com/en-us/products/big-data-integration- analytics/pentaho-data-integration.html ‒ Pentaho Business Analytics ‒ https://www.hitachivantara.com/en-us/products/big-data-integration- analytics/pentaho-business-analytics.html

§ Training ‒ https://www.hitachivantara.com/en-us/services/training- certification/training/pentaho.html ‒ Pentaho Data Integration ‒ Pentaho Data Integration Fundamentals (DI1000) ‒ Pentaho Data Integration Advanced (DI1500) ‒ Business Analytics ‒ Business Analytics User Console (BA1000) ‒ Business Analytics Report Designer (BA2000) ‒ Business Analytics Data Modeling (BA3000)

§ Training ‒ CTools ‒ CTools Fundamentals (CT1000) ‒ CTools Advanced (CT1500)

1. From the Schedule screen on the app, select your session.

2. Open the Training Survey.