INTRODUCTION TO THE SQLBI METHODOLOGY Draft 1.0 – September 20, 2008 Alberto Ferrari ([email protected]) Marco Russo ([email protected]) www.sqlbi.com INTRODUCTION .......................................................................................................................................................... 3 ARCHITECTURE OF A BI SOLUTION .............................................................................................................................. 5 CLASSIFICATION OF BI SOLUTIONS .................................................................................................................................... 5 ACTORS..................................................................................................................................................................... 6 User / Customer .................................................................................................................................................. 6 BI analyst ............................................................................................................................................................ 7 Microsoft BI suite ................................................................................................................................................ 7 ARCHITECTURE ............................................................................................................................................................ 8 Components of a BI Solution ............................................................................................................................... 8 SOURCE OLTP DATABASE ....................................................................................................................................................... 9 CONFIGURATION DATABASE................................................................................................................................................... 11 STAGING AREA .................................................................................................................................................................. 13 DATA WAREHOUSE ............................................................................................................................................................. 14 DATA MARTS .................................................................................................................................................................... 15 OLAP CUBES .................................................................................................................................................................... 15 REPORTS .......................................................................................................................................................................... 16 CLIENT TOOLS.................................................................................................................................................................... 17 OPERATIONAL DATA STORE ................................................................................................................................................... 17 COMPLETE ARCHITECTURE ..................................................................................................................................................... 18 DE-COUPLING SINGLE STEPS ................................................................................................................................................... 19 The Kimball methodology ...................................................................................................................................20 FACT AND DIMENSIONS ........................................................................................................................................................ 20 STAR SCHEMA, SNOWFLAKE SCHEMA ....................................................................................................................................... 21 JUNK DIMENSION ............................................................................................................................................................... 23 DEGENERATE DIMENSIONS..................................................................................................................................................... 24 SLOWLY CHANGING DIMENSION TYPE I, II, III .............................................................................................................................. 24 BRIDGE TABLES (OR FACTLESS FACT TABLES) ............................................................................................................................... 25 SNAPSHOT VS. TRANSACTION FACT TABLES ................................................................................................................................. 26 UPDATING FACTS AND DIMENSION ........................................................................................................................................... 27 NATURAL AND SURROGATE KEYS ............................................................................................................................................. 28 Why and when we use Kimball methodology ......................................................................................................29 The Inmon Methodology ....................................................................................................................................30 Why and when we use Inmon methodology .......................................................................................................31 BUILDING THE ARCHITECTURE ........................................................................................................................................32 Designing Relational Data ..................................................................................................................................32 USAGE OF SCHEMAS ............................................................................................................................................................ 32 USAGE OF VIEWS ................................................................................................................................................................ 32 MIRROR OLTP .................................................................................................................................................................. 35 STAGING .......................................................................................................................................................................... 38 DATA WAREHOUSE .............................................................................................................................................................. 39 DATA MART ..................................................................................................................................................................... 39 OLAP CUBES..................................................................................................................................................................... 41 CONFIGURATION DATABASE ................................................................................................................................................... 42 LOG DATABASE................................................................................................................................................................... 42 DISK OPTIMIZATION ............................................................................................................................................................. 43 Introduction to the SQLBI Methodology – Draft 1.0 – send feedback to dedicated forum on www.sqlbi.com 2 Introduction This paper is an introduction to a methodology for the implementation of advanced BI solutions using Microsoft SQL Server, SQL Server Analysis Services and – more generally – the Microsoft suite for Business Intelligence. It is difficult to clearly describe what the paper is about. It is much easier to say what it is not about and what you are supposed to know in order to get the most out of this paper. It does not describe what SSAS is, but sometimes it will try to show how it internally works. We will extensively use SSAS but will never spend time describing what you will do with a single click of the mouse. You are supposed to have a medium knowledge of the usage of SSAS and the ability to navigate easily through all its windows and properties. Instead, sometimes it will be necessary to go deep into its calculation technique in order to understand why a specific way leads to better results when compared to another one. It is not a paper about SQL Server Integration Services. Sometimes you will see screenshots of SSIS packages briefly described in order to make concepts clearer, but we will not spend time describing each feature of SSIS. It does not describe SQL at all. The book is full of SQL statements that you will need to immediately understand because we will provide no description of the SQL Syntax. For a BI analyst, SQL should be easy to read as English is. It is not an introductory or generic paper on BI solutions. You are supposed to be a BI analyst who knows the basis of BI and wants to discover something more about using the full power of SSAS to give better solution