<<

Performance Dashboards: Measuring, Monitoring, and Managing Your Business Wayne W. Eckerson

• WAYNE W. ECKERSON is the Director of Research and Services for The Warehousing Institute. • Eckerson has 17 years of industry experience and has covered data warehousing and since 1995. • Eckerson is the author of many in- depths reports, a columnist for several business and technology magazines, and a noted speaker and consultant. • Eckerson has recently written a book titled, Performance Dashboards: Measuring, Monitoring, and Managing Your Business (Wiley & Sons, 2005). • He can be reached at weckerson@ tdwi.org

© Copyright TDWI, 2006 Slide 2 Course Agenda 1. Evolution of Performance Dashboards 2. Why Performance Dashboards? 3. What are Performance Dashboards? 4. Architecting Performance Dashboards 5. Case Studies 7. Costs of Deployment 8. How to Build Effective Metrics 9. How to Design Effective Dashboard Screens 10. Criteria for Evaluating Dashboard Products

© Copyright TDWI, 2006 Slide 3 Appendices • A - Performance Dashboard Trends • B- Readiness Assessment • C - How to Ensure Adoption • D - Performance Dashboard Market Segmentation • E – Sample Metrics Report

© Copyright TDWI, 2006 Slide 4 Evolution of Performance Dashboards

© Copyright TDWI, 2006 Slide 5 The Business Challenge Decision makers suffer from…

• too much data…. • too little … • delivered too late… to make effective decisions.

© Copyright TDWI, 2006 Slide 6 Evolution of a Solution The search for the perfect “business insight system”: • 1980s – Executive information systems (EIS) – Decision support systems (DSS) • 1990s – Data warehousing (DW) – Business intelligence (BI) • 2000s – Dashboards and scorecards – Performance management • 2010+??

© Copyright TDWI, 2006 Slide 7 Two Metaphors

Dashboard Performance Chart

Performance Dashboard

© Copyright TDWI, 2006 Slide 8 Two Disciplines

Business Corporate Intelligence Performance Management STRATEGY

e 2 z ls . i a B P Wisdom g o u s F d la te G e o g n , v r e a s ti I e ts tr e n n c , lu e it a P S a c s ia s ents . p t ts la Ev Review, , Refine V In iv , n Act 1 , , a e M s n s M s o , io e y , d s iv g T e is t e a l c t rg s M je a e b tr t Data Plans O S s Rules and Models Integrated Data A ct

Knowledge io W e n 3

s I/D c n .

Analytical Tools R , B a s

e D d M

t v e rm r

a o

D s is c fo

a i r ni

u io s e bo

t j i h

a n o P s t o

d s n a

s D r

Information A , /

/ A

Data Warehouses t n c

a

A

l

y .

z 4

DATA REFINERY e EXECUTION Performance Dashboards

© Copyright TDWI, 2006 Slide 9 Corporate Performance Management STRATEGY

e ls 2 iz a o B . g s u P e G e F d l t , v o g a a s ti r e n tr e n e t lu e c s S a c a , . n s P 1 V I I ts la , , s n , n n s re it T s io e u ia a , s iv s t r s t a iv g i c e e e M je s ts b M O Integrated A c Data g, R tio in W n rt o e s o ls

r vi , p o

k s D e o 3

fl io t .

o e )

w n c , R I e Mo st s y B s

, , is r ( u

u D i e o n j A o s

i l n u i h

s i

d s e Q e

c s y r t r l o

u ts , a a

/A

s n r

t ,

W / s A A

c io ta

n a n

s D a

l

y 4. A 4.

z e

EXECUTION

© Copyright TDWI, 2006 Slide 10 Waves of Software Automation

Æ Effectiveness 0 0 0 2 Mgmt M (Strategy P C Execution) , M M R C C S Æ

5 e , , 9 Cross-Functional c n r r 9 o o i e 1 t t n Value Chains F a n g i s e (Customers, Supplies, Products)) m a e C l o p t l t a l u m s S a m a e A g C C g M Æ a k s 0 r Front-Office c 9 e 9 (Sales, Service, Marketing) a l 1 P b

a P n

R E

E T I 0 / e 0 Back Office r 0 2 (Manufacturing, Finance, , a - w 5 t 8 Procurement, Logistics) f 9 o 1 S Efficiency Business Activities Operations

© Copyright TDWI, 2006 Slide 11 Why Performance Dashboards?

© Copyright TDWI, 2006 Slide 12 Status of Performance Dashboards

No plans, 17%

Deployed, 51% Under development 33%

From Wayne Eckerson, “Development Techniques for Creating Analytic Applications,” TDWI, 2005.

© Copyright TDWI, 2006 Slide 13 Business Week Cover Story - February 13, 2006

© Copyright TDWI, 2006 Slide 14 Tactical Drivers Resonates with users • Monitors status of several areas on one screen • Graphical view of key metrics • Alerts users to exception conditions • Click to analyze and drill to detail • Customized views based on role • Personalized views based on interest • No training required!

© Copyright TDWI, 2006 Slide 15 Tactical Drivers (cont.) Rich data • Blends data from multiple sources • Both detailed and aggregated • Both historical and real-time Empowers workers • Focuses users on what’s really important • Shows them how their contributions count • Motivates with goals, competition, & incentives • Drives proactive intervention

© Copyright TDWI, 2006 Slide 16 Strategic Drivers Aligns the business • Everyone uses the same data • Everyone uses the same metrics • Everyone works toward the same strategy Improves communication • Tool for communicating strategy • Managers & staff - collaboration • Among departments - coordination Improves visibility and compliance • Fewer surprises

© Copyright TDWI, 2006 Slide 17 Strategic Drivers - The “Five Cs” • Communicate • Compare • Collaborate • Coordinate • Congratulate

© Copyright TDWI, 2006 Slide 18 Agent of Organizational Change

© Copyright TDWI, 2006 Slide 19 Charting a Course

Goal

Direction without a Performance Dashboard

Direction with a Performance Dashboard

© Copyright TDWI, 2006 Slide 20 What are Performance Dashboards?

© Copyright TDWI, 2006 Slide 21 The “Three Threes” • Three Applications • Three Layers • Three Types

© Copyright TDWI, 2006 Slide 22 Three Applications

Monitoring Analysis Collaboration

Purpose Convey performance Analyze exceptions and Collaborate, plan, status & trends at a find root cause and ACT glance Elements - Multi-paneled screens - Drill down/up -Telephone - Graphical metrics (i.e. hierarchies -Meetings dials, gauges, symbols) - Pivot and swap out - Email (notification) - Charts and tables dimensions - Annotations - Status, trend, and - Drill through to - Threaded threshold indicators operational data discussions - Color-coded, - Time series, - Recommended conditional formatting segmentation, predictive, analysis, actions - Alerts: Web-based, and other analyses - Publish to server email, other - Reporting -Workflow - Triggers, Updates

© Copyright TDWI, 2006 Slide 23 Three Layers of Information

Graphical Abstracted Data Monitoring Graphs, Symbols, Charts

Summarized Dimensional Data Analysis Dimensions, hierarchies, “slice/dice” Planning Detailed, Operational Data Reporting DW queries, Operational reports

Performance Dashboard Plans, models, forecasts, updates

© Copyright TDWI, 2006 Slide 24 Dashboard Usage “Our executives will drill one or two levels down before they call someone who can fix the problem, while our managers will often drill three or four layers down before they make a call.”

– Thomas Tomlinson, director of BI for Bull Moose Tube, a steel manufacturer in Chesterfield, MO.

© Copyright TDWI, 2006 Slide 25 Dashboards vs Scorecards • Distinct? • Synonymous? •Both?

Rule of thumb: Use whatever term business users prefer!

© Copyright TDWI, 2006 Slide 26 Dashboards vs Scorecards Dashboard Scorecard

Purpose Measures current activity Charts progress Users Executives, managers, staff Executives, managers, staff Updates “Right time” feeds Periodic snapshots Data Events Summaries Queries Run against remote systems Run against local Display Charts Symbols

Dashboards and scorecards are visual interfaces for monitoring business performance

© Copyright TDWI, 2006 Slide 27 Three Types

Operational Tactical Strategic

Focus Monitor operations Optimize process Execute strategy Emphasis Monitoring Analysis Collaboration Users Supervisors+ Managers+ Executives+ Scope Operational Departmental Enterprise Information Detailed Detailed/Summary Summary Updates Intra-day Daily/Weekly Monthly/Quarterly “Looks like a…” “Dashboard” “BI Portal” “Scorecard”

© Copyright TDWI, 2006 Slide 28 Pretenders to the throne

– Too Flat “A prettified

– Too Isolated “Another spreadmart”

– Too Manual “Not scalable or sustainable”

– Too Cheap “You get what you pay for!”

© Copyright TDWI, 2006 Slide 29 How Do You Architect a Performance Dashboard?

© Copyright TDWI, 2006 Slide 30 Three Architectures

Business Architecture BI Architecture Data Architecture

© Copyright TDWI, 2006 Slide 31 Business architecture

Stakeholders Investors Board Workforce Customers Suppliers Regulators Business Architecture

Strategy Mission Vision Values Goals Objectives Strategy Map

Tactics Assets People Knowledge Plans Process Projects

Semantics Terms Definitions Rules Education Governance

Metrics Leading Lagging Diagnostic

Displays Dashboard BI Portal Scorecard Technical Architecture

Applications Monitoring Analysis Management

Data Stores ODS Memory cache Data marts Reports Documents

Integration Custom API EAI EII Direct queries ETL Manual

Data Sources Legacy Systems Packaged Apps Web pages Files Surveys Text Performance Dashboard

© Copyright TDWI, 2006 Slide 32 BI Architecture

Business Architecture

Monitoring Layer Dashboards, scorecards, KPIs, Alerts r Analysis Layer Dimensions, hierarchies, “slice/dice”

Reporting Layer Planning Laye

Management and operational reports Plans, models, forecasts Integrated BI Capabilities

BI Platform (Analytic Server) - Common services, object model, API, file formats, etc. Data Delivery Architecture

© Copyright TDWI, 2006 Slide 33 Data Architecture – Quicken Loans

Packaged Phone Mainframe AS/400 Web Applications System

Messaging Backbone (Enterprise Application Integration software) e c i b v r We Se

ETL Operational ETL Data Warehouse Real-time Store Data Store (2 days of data) (7 years of data) (2 months of data)

250 OLAP Cubes 100 OLAP Cubes Cubes (7 years of data) Cubes (2 weeks of data)

Reporting & Operational and Analysis Tools Tactical Dashboards

© Copyright TDWI, 2006 Slide 34 Direct Query Architecture

Data Packaged Phone Data Marts Mainframe Reports Warehouse Applications System

Query engine Pros: Screen elements - Deploy quickly linked directly to - Low cost individual queries Cons - No depth, limited drill down - No dimensions - Hard-wired queries

© Copyright TDWI, 2006 Slide 35 Query and Cache Architecture

Data Packaged Phone Data Marts Mainframe Reports Spreadsheets Warehouse Applications System

Query Engine Pros: In-memory or disk cache - Deploy quickly - Fast response Queries populate a - Rapid navigation queryable cache Cons - Static data sets

© Copyright TDWI, 2006 Slide 36 BI Semantic Layer

Data Packaged Phone Data Marts Mainframe Reports Spreadsheets Warehouse Applications System

ODBC ODBC ODBC

Query Engine Semantic SemanticSemantic Semantic Semantic Layer LayerLayerLayer Layer BI tools provide Pros: - Abstract query objects query objects that - Dimensionalized views represent a Cons in business terms - Generic ODBC connections for users. -Primarily historical data in DW

© Copyright TDWI, 2006 Slide 37 Federated Query Architecture

Data Packaged Phone Data Marts Mainframe Reports Spreadsheets Warehouse Applications System

An EII tool Pros: dynamically Distributed Query Engine/EII - Multiple sources Semantic Layer - Semantic layer abstraction queries data from - Quick to deploy multiple sources - Prototype before you persist Cons to populate screen - No history elements. - Data quality issues - Complexity

© Copyright TDWI, 2006 Slide 38 Data Mart Architecture

Data Packaged Phone Data Marts Mainframe Reports Spreadsheets Warehouse Applications System

ETL Dashboard Dimensionalized Data Mart queries its own (OLAP or ) Pros: - Multiple sources persistent data Query engine and semantic layer - Dimensional model mart loaded in - Historical context batch. - Fast complex queries Cons - No right time data - Non-integrated?

© Copyright TDWI, 2006 Slide 39 Event-driven Architecture

Workflow Operational Outputs Engine Systems

Triggers SQL/Stored Procedures Alerts

Operational Dashboard Data Capture, Data Aggregation, Metrics Management, Event Detection, Rule Processing, Agents/Triggers

Enterprise Service Bus Historical Context

Operational Operational Operational Data Inputs System System System Warehouse

© Copyright TDWI, 2006 Slide 40 “Manual” Architecture

Use when…. • Data doesn’t exist • Strategy is short-term • Want to prototype the KPIs • Executives can’t wait But don’t be fooled… • Permanent prototypes • No scale, depth, value • Reputations on the line!

© Copyright TDWI, 2006 Slide 41 Performance Dashboard Case Studies

© Copyright TDWI, 2006 Slide 42 Operational Dashboard Case - Quicken Loans

• The largest U.S.-based online lender – $12 billion in loans in 2004, 2,500 employees – Sells mortgages via call center and Web • Web Call Center in Livonia, Michigan – 500+ “mortgage lenders” on one giant floor – Disruptions costs millions of dollars an hour

© Copyright TDWI, 2006 Slide 43 Situation • Company philosophy/culture – What gets measured, gets approved – Leverage “velocity as a competitive problem” • Information systems – pre-2002 – Reports run off operational systems • Run slowly, Deliver obsolete data – Disjointed data for historical analysis • Took three weeks to do 18-24 month analysis – Executives and users very frustrated! • Negative view of data warehousing/OLAP

© Copyright TDWI, 2006 Slide 44 Solution • Right-time data warehousing architecture – 1 year at $1 million – Trickle fed OLAP cubes – Existing ESB • Different dashboards for different users – Dashboard ticker – mortgage specialists – Kanban reports – Sales managers, TV monitors – Managerial dashboards – Call center managers – Analytical dashboards/BI tools – Analysts • Metrics – Phone statistics, Number and quality of leads, Sales pipeline, Web traffic, Commissions, products mix

© Copyright TDWI, 2006 Slide 45 Quicken Loans Architecture

Packaged Phone Mainframe AS/400 Web Applications System

Messaging Backbone (Enterprise Application Integration software) e c i b v r We Se

ETL Operational ETL Data Warehouse Real-time Store Data Store (2 days of data) (7 years of data) (2 months of data)

250 OLAP Cubes 100 OLAP Cubes Cubes (7 years of data) Cubes (2 weeks of data)

Reporting & Operational and Analysis Tools Tactical Dashboards

© Copyright TDWI, 2006 Slide 46 Dashboard Ticker

Personal and group metrics updated daily

Personal and group metrics updated instantly

Personal forecasts updated every 15 minutes

© Copyright TDWI, 2006 Slide 47 Kanban Report

Metric 1 Metric 2 Metric 3 Metric 4 Metric 5 Metric 6 Metric 7 Metric 8 Metric 9 Employee 1 Employee 2 Employee 3 Employee 4 Employee 5 Employee 6 Employee 7 Employee 8 Employee 9 Employee 10 Employee 11 Employee 12 Employee 13 Employee 14 Employee 15 Employee 16 Employee 17

© Copyright TDWI, 2006 Slide 48 Managerial Dashboard

© Copyright TDWI, 2006 Slide 49 Tactical Dashboard Case International Truck and Engine • $9.7 billion manufacturer of trucks, buses, diesel engines, and parts based in Illinois • Key business issues: – Market reality: Global competition, new regulations, emerging markets – Goals: 1) $15b in revenues 2) reduced costs, 3) improved quality, 4) reduced risk

© Copyright TDWI, 2006 Slide 50 Situation

• Finance department goals in 2001: – Provide access to financial information any time – Focus on analysis rather than data collection – Deliver proactive rather than reactive analysis – Use financial data as a predictive tool for decisions • Programs – Accelerate closing of books – Standardize company’s information infrastructure – Replace legacy systems with packaged applications – Implement a Web-based “reporting portal.”

© Copyright TDWI, 2006 Slide 51 KBI Portal • Purpose – Deliver actionable information to financial analysts • Scope – Spans 32 source systems across five divisions – 130 key business indicators, updated daily – Supports 500 financial executives, managers, and analysts • Upshot – Bridges gulf between finance and operations – Replaces hodge-podge of paper reports – Saves analysts time creating custom reports – Shuts down dozens of reporting systems

© Copyright TDWI, 2006 Slide 52 Architecture

Purpose Data

Run the business Source Systems Transactional

ETLTools

Gather all data in one place. Transactional Keep a copy for future reuse. Staging Area Database Data Warehouse Transactional & Integrate data for easy loading into Star Schema Database lightly summarized OLAP cubes. ETLTools

Moderately Store data dimensionally to support OLAP Cubes fast queries and easy navigation. summarized

Web Server

Display key metrics so they can be Highly viewed at a glance. Reporting Portal summarized

© Copyright TDWI, 2006 Slide 53 Monitoring Layer

© Copyright TDWI, 2006 Slide 54 Analysis Layer

© Copyright TDWI, 2006 Slide 55 Detail Transaction Layer

‘Click’‘Click’anyany VIN VIN to to expand expand fullfull page page order/build order/build report report

© Copyright TDWI, 2006 Slide 56 Personalization

© Copyright TDWI, 2006 Slide 57 Metadata

© Copyright TDWI, 2006 Slide 58 Strategic Dashboard Case Hewlett-Packard TSG

• HP Technology Services Group – Provides consulting, support services, and software globally for HP – $12 billion division of Hewlett Packard • Situation – Dozens of overlapping reporting systems with inconsistent metrics – No consistent means of measuring regional and business unit performance against company objectives and holding individuals accountable

© Copyright TDWI, 2006 Slide 59 Solution • HP Performance Measurement and Management System (PMMS) – Executive scorecard (LIBRA) deployed to EMEA region in 2001, then globally thereafter – Cascaded down multiple levels in each region – Implemented unified reporting system underneath (MUSE) • Upshot: $26 million cost-savings in 3 years on $1m expenditure – $8.6 million – Shut down dozens of report systems – $10.6 million – Reduced time spent looking for reports – $1.3 million - Training users on BI tools, etc. – Raised customer satisfaction scores, lowered missed service-level commitments, correlate to revenue

© Copyright TDWI, 2006 Slide 60 PMMS Architecture

Value Content Data

“Balanced” Scorecards 12 OLAP cubes updated monthly with 100MB of data Knowledge “LIBRA” “Unbalanced” scorecards (OLAP)

2,500 OLAP cubes updated daily to monthly with 200 GB Information “MUSE” Scorecard Reports of data, spread across four (OLAP) data centers

Operational Systems Data Warehouses 40 data sources, only Data SOURCE DATA exception conditions captured Budgets, Surveys (, Spreadsheets, Files) Data Marts Volume

© Copyright TDWI, 2006 Slide 61 Monitoring Layer

© Copyright TDWI, 2006 Slide 62 Analysis Layer

© Copyright TDWI, 2006 Slide 63 Analysis Layer (cont.)

© Copyright TDWI, 2006 Slide 64 Reporting Layer

© Copyright TDWI, 2006 Slide 65 Rohm & Haas • Company – $8 billion global chemicals manufacturer • Impetus – CFO restructures finance to improve efficiency – Eliminate spreadmarts – offer consistent metrics • Time and cost – First iteration: 12 months, $500k – Subsequent dashboards (12): $100k • Tools – Custom built – SAP Portal, SAP Web Application Developer – Runs against SAP BW with Hyperion data imported

© Copyright TDWI, 2006 Slide 66 © Copyright TDWI, 2006 Slide 67 © Copyright TDWI, 2006 Slide 68 Railroad Company • Impetus – Automate daily performance report (paper) • 120 page report, 45 measures, 4 levels, all locations • Reduce time spent analyzing data • Time and Cost – First iteration: 7 months and $500k – Current view: 1.5 years and $1M • Tools – Existing: Teradata, Essbase, Alphablox, ESRI – New: Treemap software (~$2k)

© Copyright TDWI, 2006 Slide 69 © Copyright TDWI, 2006 Slide 70 © Copyright TDWI, 2006 Slide 71 © Copyright TDWI, 2006 Slide 72 IBM • Impetus - 1998 – Dueling spreadsheets in weekly sales meetings • Time and cost – First iteration: 6 months, $200k – Became basis of BI and EDW initiative • 25,000 users, thousands of reports, 40 dashboards • Tools – Lotus Notes (email, disconnected, unstructured) – Web portal for mid-level managers

© Copyright TDWI, 2006 Slide 73 End to End Business Process

Dashboard designed to allow management by exception. Requires agreement across the enterprise on thresholds.

© Copyright TDWI, 2006 Slide 74 Simple text based inputs. Updated as required.

© Copyright TDWI, 2006 Slide 75 Contact Information Wayne Eckerson

Director of Research and Services The Data Warehousing Institute 70 Martins Lane, Hingham, MA USA 781 740-9504 [email protected]

© Copyright TDWI, 2006 Slide 76