The Data Warehouse Lifecycle Toolkit
Total Page:16
File Type:pdf, Size:1020Kb
The Data Warehouse Lifecycle Toolkit Second Edition The Data Warehouse Lifecycle Toolkit Second Edition Ralph Kimball Margy Ross Warren Thornthwaite Joy Mundy Bob Becker Wiley Publishing, Inc. The Data Warehouse Lifecycle Toolkit, Second Edition Published by Wiley Publishing, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright 2008 by Ralph Kimball, Margy Ross, Warren Thornthwaite, Joy Mundy and Bob Becker Published by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-0-470-14977-5 Manufactured in the United States of America 10987654321 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4355, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Website is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Website may provide or recommendations it may make. Further, readers should be aware that Internet Websites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services or to obtain technical support, please contact our Customer Care Department within the U.S. at (800) 762-2974, outside the U.S. at (317) 572-3993 or fax (317) 572-4002. Library of Congress Cataloging-in-Publication Data: The data warehouse lifecycle toolkit / Ralph Kimball... [et al.]. -- 2nd ed. p. cm. Includes index. ISBN 978-0-470-14977-5 (paper/website) 1. Data warehousing. I. Kimball, Ralph. QA76.9.D37D38 2007 005.74--dc22 2007040691 Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or vendor mentioned in this book. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. About the Authors The authors’ professional careers have followed remarkably similar paths. Each author has focused on data warehousing and business intelligence (DW/BI) consulting and education for more than fifteen years. Most worked together at Metaphor Computer Systems, a pioneering decision support vendor, in the 1980s. All the authors are members of the Kimball Group and teach for Kimball University. They contribute regularly to Intelligent Enterprise magazine and other industry publications; most have previously written books in the Toolkit series. Ralph Kimball founded the Kimball Group. Since the mid 1980s, he has been the DW/BI industry’s thought leader on the dimensional approach and trained more than 10,000 IT professionals. Ralph has his Ph.D. in Electrical Engineering from Stanford University. Margy Ross is President of the Kimball Group. She has focused exclusively on DW/BI since 1982 with an emphasis on business requirements analysis and dimensional modeling. Margy graduated with a BS in Industrial Engineering from Northwestern University. Warren Thornthwaite began his DW/BI career in 1980. After managing Metaphor’s consulting organization, he worked for Stanford University and WebTV. Warren holds a BA in Communications Studies from the University of Michigan and an MBA from the University of Pennsylvania’s Wharton School. Joy Mundy has focused on DW/BI systems since 1992 with stints at Stanford, Web TV, and Microsoft’s SQL Server product development organization. Joy graduated from Tufts University with a BA in Economics, and from Stanford University with an MS in Engineering Economic Systems. Bob Becker has helped clients across a variety of industries with their DW/BI challenges and solutions since 1989, including extensive work with health care organizations. Bob has a BSB in Marketing from the University of Minnesota’s School of Business. v Credits Executive Editor Vice President and Executive Robert Elliott Publisher Joseph B. Wikert Development Editor Sara Shlaer Project Coordinator, Cover Lynsey Osborn Production Editor Debra Banninger Proofreader Nancy Carrasco Copy Editor Kim Cofer Indexer Editorial Manager Melanie Belkin Mary Beth Wakefield Anniversary Logo Design Production Manager Richard Pacifico Tim Tate Cover Image Vice President and Executive Steve Allen/Getty Images Group Publisher Richard Swadley vii Acknowledgments First, thanks to our students, clients, readers, and former colleagues for sup- porting, teaching, and influencing us. One of the authors recently received a fortune cookie that read, ‘‘You learn most when teaching others.’’ We couldn’t agree more. Our Kimball University students have pushed us to provide pre- cise, specific guidance and kept us on our toes with their questions. Similarly, the challenges faced by our Kimball Group consulting clients have become our challenges, and have kept us grounded in reality. Finally, ex-colleagues have contributed to our thinking about the concepts in this book, including Laura Reeves who participated as a co-author of the first edition of the Lifecycle Toolkit. Beginning with our associates from the early days at Metaphor, through Red Brick, Stanford University, DecisionWorks Consulting, InfoDynamics, and Microsoft, we’ve learned lots from each of you. Thanks to the Wiley team for making this book a reality. Bob Elliott’s subtle, yet persistent prodding got the project off the ground. Sara Shlaer did a wonderful job editing our text with an incredible amount of patience, tenacity, and attention to detail. Deb Banninger and the behind-the-scenes folks worked tirelessly to deliver a quality product. We’ve enjoyed working with all of you. Finally, thanks to our spouses, partners, and children for putting up with the demands of our careers, while supporting us unconditionally. You’ve suffered through late nights and missed vacations alongside us. Thanks to Julie Kimball, Sara Kimball Smith, and Brian Kimball, Scott and Katie Ross, Elizabeth Wright, Tony Navarrete, and Pam, Elisa, and Jenna Becker. We couldn’t have done it without you! ix Contents at a Glance Chapter 1 Introducing the Kimball Lifecycle 1 Chapter 2 Launching and Managing the Project/Program 15 Chapter 3 Collecting the Requirements 63 Chapter 4 Introducing the Technical Architecture 109 Chapter 5 Creating the Architecture Plan and Selecting Products 179 Chapter 6 Introducing Dimensional Modeling 233 Chapter 7 Designing the Dimensional Model 287 Chapter 8 Designing the Physical Database and Planning for Performance 327 Chapter 9 Introducing Extract, Transformation, and Load 369 Chapter 10 Designing and Developing the ETL System 425 Chapter 11 Introducing Business Intelligence Applications 473 Chapter 12 Designing and Developing Business Intelligence Applications 505 Chapter 13 Deploying and Supporting the DW/BI System 541 Chapter 14 Expanding the DW/BI System 579 xi Contents Acknowledgments ix Introduction xxxi Chapter 1 Introducing the Kimball Lifecycle 1 Lifecycle History Lesson 1 Lifecycle Milestones 3 Program/Project Planning 4 Program/Project Management 4 Business Requirements Definition 5 Technology Track 5 Technical Architecture Design 5 Product Selection and Installation 6 Data Track 6 Dimensional Modeling 6 Physical Design 6 ETL Design and Development 7 Business Intelligence Application Track 7 BI Application Design 7 BI Application Development 7 Deployment 7 Maintenance 8 Growth 8 Using the Lifecycle Roadmap 8 Lifecycle Navigation Aids 9 Lifecycle Vocabulary Primer 9 Data Warehouse versus Business Intelligence 10 ETL System 11 Business Process Dimensional Model 12 Business Intelligence Applications 13 Conclusion 14 xiii xiv Contents Chapter 2 Launching and Managing the Project/Program 15 Define the Project 16 Assess Your Readiness for DW/BI 16 Strong Senior Business Management Sponsor(s) 16 Compelling Business Motivation 17 Feasibility 17 Factors Not Considered Readiness Deal Breakers 18 Address Shortfalls and Determine