DOCSLIB.ORG

Building a Data Mining Model Using Data Warehouse and OLAP Cubes IST 734 SS Chung

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Building a Data Mining Model Using Data Warehouse and OLAP Cubes IST 734 SS Chung Cleveland State University Building a Data Mining Model using Data Warehouse and OLAP Cubes IST 734 SS Chung 14 Sunnie S Chung IST 734 Build a Data Mining Model using Data Warehouse and OLAP cubes Contents 1. Abstract: ................................................................................................................................................ 3 2. Introduction: .......................................................................................................................................... 3 3. Adventure works database: ................................................................................................................... 4 4. Getting familiar with Sql Server Analysis Services (SSAS) tools and various datamining algorithms 5 4.1. Microsoft Association Algorithm .................................................................................................. 6 4.2. Microsoft Clustering Algorithm .................................................................................................... 9 4.3. Microsoft Time Series Algorithm ................................................................................................ 12 4.4. Microsoft Decision Trees Algorithm .......................................................................................... 16 5. Star schema: ........................................................................................................................................ 19 5.1. Fact Tables and Dimension Tables .................................................................................................. 19 5.2. Steps in Star Schema Design: .......................................................................................................... 19 6. Cube AND MDX: ............................................................................................................................... 19 6.1. Cube: ........................................................................................................................................... 19 6.2. Dimensions: ................................................................................................................................ 20 6.3. Measure & Measure Groups ....................................................................................................... 20 6.4. Steps in Building and Deploying a cube: .................................................................................... 20 7. Quering On Cubes Using MDX-Examples ......................................................................................... 26 7.1. Queries: ....................................................................................................................................... 26 7.2. Screenshots: ................................................................................................................................ 27 8. DataMining and DMX ....................................................................................................................... 32 8.1. Business Scenario1: .................................................................................................................... 32 8.2. Business Scenario2: .................................................................................................................... 36 Sunnie S Chung IST 734 8.3. Business Scenario3: .................................................................................................................... 38 9. Conclusion: .......................................................................................................................................... 41 10. References: ..................................................................................................................................... 41 1. Abstract: The purpose of this project is learn the application of various database and datamining concpets and its application in current business using Adventure works database and various database tools.We begin by designing a star schema and building a DataWarehouse OLAP cube, for Sales Analysis using Sql Server Analysis Services (SSAS) and query the cube using MultiDimensional eXpressions language(MDX) to find the current business trends.Next we create data mining structure using Data Mining Extensions(DMX) and find the hidden pattern and predict things that are helpful in improving the business and its growth. Some of the data mining techniques that will be considering to accomplish the above tasks are Microsoft Association Mining, Cluster analysis, Time series and Naïve- Bayes Algorithm. 2. Introduction: A data warehouse is a centralized repository that stores data from multiple information sources and transforms them into a common, multidimensional data model for efficient querying and analysis.OLAP and Data Mining are two complementary technologies for Business Intelligence.Online Analytical Processing (OLAP) is a technology that is used to organize large business databases and support business intelligence. OLAP is a database technology that has been optimized for querying and reporting, instead of processing transactions OLAP databases are divided into one or more cubes, and each cube is organized and designed by a cube administrator to fit the way that you retrieve and analyze data. OLAP is used for decision-support systems to analyze aggregated information for sales, finance, budget, and many other types of applications. OLAP is about aggregating measures based on dimensionhierarchies and storing these precalculated aggregations in a special data structure. With the help of preaggregations and special indexes, you can query aggregated data and get decision-support query results back in real time OLAP provides us with a very good view of what is happening, but can not predict what will happen in the future or why it is happening.This part is done by datamining.Data Mining is a combination of discovering techniques + prediction techniques. Sunnie S Chung IST 734 The sequence of steps that will be followed in this project is 1. Understand the Adventure works database which will be used in this project(fully understand the transactional data available in the database). 2. Getting familiar with Sql Server Analysis Services (SSAS) tools and various datamining algorithms such as Clustering,Association,Time Series etc in the SSAS tool 3. The next step will be to come up with a list of questions: what questions need to be answered, what metrics will help business managers monitor and grow the business. 4. Based on the business questions that need to be answered, data staging layer in a star schema format will be designed. 5. Then the cube will be built to extract data from the Star schema staging layer and we perform our data mining on the cube. 6. Perform Datamining in the Adventure works database to find hidden patterns and information using DMX and MDX. 3. Adventure works database: In this section we will try to understand the Adventureworks database that will be used as a part of the project.We try to understand scope of the business its various components and products etc. 3.1. Business Overview: Adventure Works Cycles is a large multinational bicycle manufacturer, with headquarters located in Bothell, Washington. The company has approximately 300 employees, 29 of which are sales representatives. The primary distribution channel for Adventure Works Cycles through the retail stores of their resellers. These resellers are located in Australia, Canada, France, Germany, the United Kingdom, and the United States. Adventure Works Cycles also sells to individual customers worldwide by means of the Internet. Adventure Works Cycles has five major product offerings: Bikes – Three primary bike product lines: Mountain, Road, and Touring. Accessories – Examples include helmets and water bottles. Clothing – Examples include jerseys and biking shorts. Components – Examples include bottom brackets and frames. Services – Examples include premium service and standard service. Sunnie S Chung IST 734 The version of Adventure works database that will be used in this project is Adventure works 2012 4. Getting familiar with SQL Server Analysis Services (SSAS) tools and various data mining algorithms In this section we will learn and understand some data mining algorithms and its applications that will be used as a part of project later. Sunnie S Chung IST 734 4.1. Microsoft Association Algorithm The Microsoft Association algorithm is an association algorithm provided by Analysis Services that is useful for recommendation engines. A recommendation engine recommends products to customers based on items they have already bought, or in which they have indicated an interest. The Microsoft Association algorithm is also useful for market basket analysis. Association models are built on datasets that contain identifiers both for individual cases and for the items that the cases contain. A group of items in a case is called an itemset . An association model consists of a series of itemsets and the rules that describe how those items are grouped together within the cases. The rules that the algorithm identifies can be used to predict a customer's likely future purchases, based on the items that already exist in the customer's shopping cart. Sunnie S Chung IST 734 Sunnie S Chung IST 734 Sunnie S Chung IST 734 4.2. Microsoft Clustering Algorithm The Microsoft Clustering algorithm is a segmentation algorithm provided by Analysis Services. The algorithm uses iterative techniques to
Load more

Recommended publications
  • MCSA SQL Server 2016
    Microsoft MCSA: SQL Server Pre-course Reading R-481-01 Version 1.0 https://firebrand.training Pre-course Reading So you are taking a course about SQL from Firebrand. As you may already know there are three different tracks for studying SQL 2016 offered at Firebrand: . MCSASQLDD - a track for database developers . MCSASQLDA - a track for database administrators . MCSASQLBID - a track for business intelligence developers The track for Database Development will entail writing code in Transact SQL ranging in complexity from a simple Select statement to retrieve the data stored in a table or tables to creating more complex programming objects like Stored Procedures, Triggers, Functions and the like. The Database Administration track covers items like how to create logins and users, do backup and restore as well as more complex items like index management and creating an Azure database out in the cloud. The Business Intelligence track includes developing ways to move data into a data warehouse and then using that data warehouse as a data source for models that your business users can run reports against. In this document there will be sections for each of the tracks including some links to some helpful websites that can help you prepare for your upcoming Firebrand course. Before we break down into the different subsections for the different tracks, let’s start off with a little history about Microsoft SQL Server. Many years ago Microsoft bought a product that was called Sybase and it became Microsoft SQL Server. It was a multiuser relational database which means it provided a way to hold data in tables that were related to each other and allowed multiple users to access that data simultaneously.
    [Show full text]
  • Oracle White Paper June 2009
    An Oracle White Paper June 2009 Oracle Data Mining 11g Competing on In-Database Analytics Oracle White Paper— Oracle Data Mining 11g: Competing on In-Database Analytics Disclaimer The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Oracle White Paper— Oracle Data Mining 11g: Competing on In-Database Analytics Executive Overview............................................................................. 1 In-Database Data Mining .................................................................... 1 Key Benefits .................................................................................... 3 Introduction ......................................................................................... 4 Oracle Data Mining ......................................................................... 4 Data Mining Deep Dive ....................................................................... 6 Oracle Data Mining for Data Analysts ............................................... 15 Oracle Data Mining for Applications Developers............................... 16 Competing on In-Database Analytics................................................ 18 Beyond a Tool; Enabling
    [Show full text]
  • Data Mining with Microsoft SQL Server 2008 / Jamie Maclennan, Bogdan Crivat, Zhaohui Tang
    Maclennan ffirs.tex V3 - 10/04/2008 3:27am Page ii Maclennan ffirs.tex V3 - 10/04/2008 3:27am Page i Data Mining with Microsoft SQL Server2008 Maclennan ffirs.tex V3 - 10/04/2008 3:27am Page ii Maclennan ffirs.tex V3 - 10/04/2008 3:27am Page iii Data Mining with Microsoft SQL Server2008 Jamie MacLennan ZhaoHui Tang Bogdan Crivat Wiley Publishing, Inc. Maclennan ffirs.tex V3 - 10/04/2008 3:27am Page iv Data Mining with MicrosoftSQL Server2008 Published by Wiley Publishing, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright 2009 by Wiley Publishing, Inc., Indianapolis, Indiana Published by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-0-470-27774-4 Manufactured in the United States of America 10987654321 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4355, or online at www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose.
    [Show full text]
  • Modelo Tese MGI / MEGI
    MODELO ZeEN Uma abordagem minimalista para o desenho de data warehouses Miguel Nuno da Silva Gomes Rodrigues Gago Dissertação apresentada como requisito parcial para obtenção do grau de Mestre em Estatística e Gestão de Informação Dissertation presented as partial requirement for obtaining the Master’s degree in Statistics and Information Management ii TÍTULOTÍTULO Subtítulo Subtítulo Nome completo do Candidato Nome completo do Candidato Dissertação / Trabalho de Projeto / Relatório de Dissertação / Trabalho de Projeto / Relatório de Estágio apresentada(o)Estágio apresentada como requisito(o) como parcial requisito para obtenção parcial do para grauobtenção de Mestre do emgrau Gestão de Mestre de Informação em Estatística e Gestão de Informação Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa MODELO ZeEN Uma abordagem minimalista para o desenho de data warehouses por Miguel Nuno da Silva Gomes Rodrigues Gago Dissertação apresentada como requisito parcial para a obtenção do grau de Mestre em Estatística e Gestão de Informação, Especialização em Gestão dos Sistemas e Tecnologias de Informação Orientador: Prof. Dr. Miguel de Castro Neto Março 2013 iii Ao meu Pai, o Engenheiro Armando Rodrigues Gago, que me ensinou a procurar sempre mais além. iv Agradecimentos À minha Mãe Maria Ondina, À minha Mulher Luísa, pelo tempo que lhes subtraí e por acreditarem sempre em mim. Ao Prof. Dr. Miguel de Castro Neto, por me ter incutido confiança em desenvolver esta dissertação na área da Business Intelligence. v Il semble que la perfection soit atteinte, non quand il n'y a plus rien à ajouter mais quand il n'y a plus rien à retrancher.
    [Show full text]
  • SQL Server 2012 Tutorials – Analysis Services Data Mining
    SQL Server 2012 Tutorials: Analysis Services - Data Mining SQL Server 2012 Books Online Summary: Microsoft SQL Server Analysis Services makes it easy to create sophisticated data mining solutions. The step-by-step tutorials in the following list will help you learn how to get the most out of Analysis Services, so that you can perform advanced analysis to solve business problems that are beyond the reach of traditional business intelligence methods. Category: Step-by-Step Applies to: SQL Server 2012 Source: SQL Server Books Online (link to source content) E-book publication date: June 2012 Copyright © 2012 by Microsoft Corporation All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any means without the written permission of the publisher. Microsoft and the trademarks listed at http://www.microsoft.com/about/legal/en/us/IntellectualProperty/Trademarks/EN-US.aspx are trademarks of the Microsoft group of companies. All other marks are property of their respective owners. The example companies, organizations, products, domain names, email addresses, logos, people, places, and events depicted herein are fictitious. No association with any real company, organization, product, domain name, email address, logo, person, place, or event is intended or should be inferred. This book expresses the author’s views and opinions. The information contained in this book is provided without any express, statutory, or implied warranties. Neither the authors, Microsoft Corporation, nor its resellers, or distributors will be held liable for any damages caused or alleged to be caused either directly or indirectly by this book.
    [Show full text]
  • Predictive Analysis in Microsoft SQL Server 2012 Gain Intuitive and Comprehensive Predictive Insight
    Predictive Analysis in Microsoft SQL Server 2012 Gain intuitive and comprehensive predictive insight automatically grouping similar Top Features Complete customers together. Test multiple data-mining models Inform decisions with intuitive and simultaneously with statistical scores comprehensive predictive insight Forecasting of error and accuracy and confirm available to all users. Predict sales and inventory amounts their stability with cross-validation. and learn how they are interrelated Rich and Innovative Algorithms to foresee bottlenecks and improve Build multiple, incompatible mining Benefit from many rich and performance. models within a single structure; innovative data-mining algorithms apply model analysis over filtered to support common business Data Exploration data; query against structure data to problems promptly and accurately. Analyze profitability across present complete information, all customers or compare customers enabled by enhanced mining Market Basket Analysis who prefer different brands of the structures. Discover which items tend to be same product to discover new Combine the best of both worlds by bought together to create opportunities. blending optimized near-term recommendations on-the-fly and to predictions (ARTXP) and stable long- determine how product placement Unsupervised Learning term predictions (ARIMA) with can directly contribute to your Identify previously unknown Better Time Series Support. bottom line. relationships between various elements of your business to better Discover the relationship between Churn Analysis inform your decisions. items that are frequently purchased Anticipate customers who may be together by using Shopping Basket considering canceling their service Website Analysis Analysis and generate interactive Understand how people use your forms for scoring new cases by and identify benefits that will keep them from leaving.
    [Show full text]
  • SQL Server Analysis Services (SSAS)?
    What is SQL Server Analysis Services (SSAS)? SQL Server Analysis Services (SSAS) is the On-Line Analytical Processing (OLAP) Component of SQL Server. SSAS allows you to build multidimensional structures called Cubes to pre-calculate and store complex aggregations, and also to build mining models to perform data analysis to identify valuable information like trends, patterns, relationships etc. within the data using Data Mining capabilities of SSAS, which otherwise could be really difficult to determine without Data Mining capabilities. SSAS comes bundled with SQL Server and you get to choose whether or not to install this component as part of the SQL Server Installation. What is OLAP? How is it different from OLTP? OLAP stands for On-Line Analytical Processing. It is a capability or a set of tools which enables the end users to easily and effectively access the data warehouse data using a wide range of tools like Microsoft Excel, Reporting Services, and many other 3rd party business intelligence tools. OLAP is used for analysis purposes to support day-to-day business decisions and is characterized by less frequent data updates and contains historical data. Whereas, OLTP (On- Line Transactional Processing) is used to support day-to-day business operations and is characterized by frequent data updates and contains the most recent data along with limited historical data based on the retention policy driven by business needs. What is a Data Source? What are the different data sources supported by SSAS? A Data Source contains the connection information used by SSAS to connect to the underlying database to load the data into SSAS during processing.
    [Show full text]
  • A Query Language for Analyzing Networks
    A Query Language for Analyzing Networks Anton Dries Siegfried Nijssen Luc De Raedt K.U.Leuven, Celestijnenlaan 200A, Leuven, Belgium {anton.dries,siegfried.nijssen,luc.deraedt}@cs.kuleuven.be ABSTRACT to analyze the network in order to discover new knowledge With more and more large networks becoming available, and use that knowledge to improve the network. mining and querying such networks are increasingly impor- Discovering new knowledge in databases (also known as tant tasks which are not being supported by database models KDD) typically involves a process in which multiple oper- and querying languages. This paper wants to alleviate this ations are repeatedly performed on the data, and as new situation by proposing a data model and a query language insights are gained, the data is being transformed and ex- for facilitating the analysis of networks. Key features in- tended. As one example consider a bibliographical network clude support for executing external tools on the networks, with authors, papers, and citations such as that gathered by flexible contexts on the network each resulting in a different Citeseer or Google Scholar. Important problems in such bib- graph, primitives for querying subgraphs (including paths) liographical networks include: entity resolution [3], which is and transforming graphs. The data model provides for a concerned with detecting which nodes (authors or papers) closure property, in which the output of every query can be refer to the same entity, and collective classification [18], stored in the database and used for further querying. where the task is to categorize papers according to their subject. This type of analysis typically requires one to call multiple tools and to perform a lot of transformations on Categories and Subject Descriptors the data.
    [Show full text]
  • Inductive Databases and Constraint-Based Data Mining Sašo Džeroski • Bart Goethals • Panþe Panov Editors
    Inductive Databases and Constraint-Based Data Mining Sašo Džeroski • Bart Goethals • Panþe Panov Editors Inductive Databases and Constraint-Based Data Mining 1 C Editors Sašo Džeroski Panče Panov Jožef Stefan Institute Jožef Stefan Institute Dept. of Knowledge Technologies Dept. of Knowledge Technologies Jamova cesta 39 Jamova cesta 39 SI-1000 Ljubljana SI-1000 Ljubljana Slovenia Slovenia [email protected] [email protected] Bart Goethals University of Antwerp Mathematics and Computer Science Dept. Middelheimlaan 1 B-2020 Antwerpen Belgium [email protected] ISBN 978-1-4419-7737-3 e-ISBN 978-1-4419-7738-0 DOI 10.1007/978-1-4419-7738-0 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2010938297 © Springer Science+Business Media, LLC 2010 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connec- tion with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) Preface This book is about inductive databases and constraint-based data mining, emerging research topics lying at the intersection of data mining and database research.
    [Show full text]
  • SPARQL-ML: Knowledge Discovery for the Semantic Web University Of
    University of Zurich Department of Informatics SPARQL-ML: Diploma Thesis December 18, 2007 Knowledge Discovery for the Semantic Web Andre´ Locher of Bratsch VS, Switzerland Student-ID: 03-706-405 [email protected] Advisor: Christoph Kiefer Prof. Abraham Bernstein, PhD Department of Informatics University of Zurich http://www.ifi.uzh.ch/ddis Acknowledgements I would like to thank Christoph Kiefer and Prof. Abraham Bernstein for giving me the oppor- tunity to write this thesis and for their valuable input, which always made me strive for more. Additionally I want to give a big thank you to Claudia von Bastian for her moral support during the last six months and of course for proofreading. I would also like to use this opportunity to thank my parents for supporting my studies and allowing me to go all the way. Abstract Machine learning as well as data mining has been successfully applied to automatically or semi- automatically create Semantic Web data from plain data. Only little work has been done so far to explore the possibilities of machine learning to induce models from existing Semantic Web data. The interlinked structure of Semantic Web data allows to include relations between entities in addition to attributes of entities of propositional data mining techniques. It is, therefore, a perfect match for Statistical Relational Learning methods (SRL), which combine relational learning with statistics and probability theory. This thesis presents SPARQL-ML, a novel approach to perform data mining tasks for knowl- edge discovery in the Semantic Web. Our approach is based on SPARQL and allows the use of sta- tistical relational learning methods, such as Relational Probability Trees and Relational Bayesian Classifiers, as well as traditional propositional learning methods.
    [Show full text]
  • Mining Model Content
    © 2014 IJIRT | Volume 1 Issue 4 | ISSN : 2349-6002 MINING MODEL CONTENT Aastha Trehan, Ritika Grover, Prateek Puri Dronacharya College Of Engineering, Gurgaon Abstract- This paper highlights about the mining model content used in data Microsoft Generic Content Tree Viewer, provided in SQL Server Data mining. The mining model is complete after you have designed and processed a Tools (SSDT), and then switch to one of the custom viewers to see how the mining model using data from the underlying mining structure and information is interpreted and displayed graphically for each model contains mining model content. You can use this content to make predictions or analyze your data. This paper describes in-depth the structure of mining model type. You can also create queries against the mining model content by using content, nodes in the model content, mining model content by algorithm type any client that supports the MINING_MODEL_CONTENT schema rowset. and tools for viewing and querying mining model content. Index Terms- Mining Model Conent, Mining Model, Mining Content Nodes, II. STRUCTURE OF MINING MODEL CONTENT Tools The content of each model is presented as a series of nodes. A node is an object within a mining model that contains metadata and information about I. INTRODUCTION a portion of the model. Nodes are arranged in a hierarchy. The exact Mining model content includes metadata about the model, statistics about arrangement of nodes in the hierarchy, and the meaning of the hierarchy, the data, and patterns discovered by the mining algorithm. Depending on depends on the algorithm that you used. For example, if you create a the algorithm that was used, the model content may include regression decision trees model, the model can contain multiple trees, all connected to formulas, the definitions of rules and itemsets, or weights and other the model root; if you create a neural network model, the model may statistics.
    [Show full text]
  • Normalization Rules
    APPENDIX Normalization Rules Normalization is the process of removing data redundancy by implementing normalization rules. There are five degrees of normal forms, from the first normal form through the fifth normal form, as described in this appendix. First Normal Form The following are the characteristics of first normal form (1NF): • There must not be any repeating columns or groups of columns. An example of a repeating column is a customer table with Phone Number 1 and Phone Number 2 columns. Using “table (column, column)” notation, an example of a repeating group of columns is Order Table (Order ID, Order Date, Product ID, Price, Quantity, Product ID, Price, Quantity). Product ID, Price, and Quantity are the repeating group of columns. • Each table must have a primary key (PK) that uniquely identifies each row. The PK can be a composite, that is, can consist of several columns, for example, Order Table (Order ID, Order Date, Customer ID, Product ID, Product Name, Price, Quantity). In this notation, the underlined columns are the PKs; in this case, Order ID and Product ID are a composite PK. Second Normal Form The following are the characteristics of second normal form (2NF): • It must be in 1NF. • When each value in column 1 is associated with a value in column 2, we say that column 2 is dependant on column 1, for example, Customer (Customer ID, Customer Name). Customer Name is dependant on Customer ID, noted as Customer ID ➤ Customer Name. 505 506 APPENDIX ■ NORMALIZATION RULES • In 2NF, all non-PK columns must be dependent on the entire PK, not just on part of it, for example, Order Table (Order ID, Order Date, Product ID, Price, Quantity).
    [Show full text]