SQLCAT’s Guide to: BI and Analytics Microsoft SQLCAT Team Summary: This e-book is a collection of some of the more popular technical content that was available on the old SQLCAT.COM site. It covers SQL Server technology ranging from SQL Server 2005 to SQL Server 2012. However, this is not all the content that was available on SQLCAT.COM. To see additional content from that site you can follow the SQLCAT blog which will point to additional content. For more comprehensive content on SQL Server, see the MSDN library. Category: Guide & Reference Applies to: SQL Server Business Intelligence and Analytics Source: SQLCAT Blog E-book publication date: September 2013 Copyright © 2012 by Microsoft Corporation All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any means without the written permission of the publisher. Microsoft and the trademarks listed at http://www.microsoft.com/about/legal/en/us/IntellectualProperty/Trademarks/EN- US.aspx are trademarks of the Microsoft group of companies. All other marks are property of their respective owners. The example companies, organizations, products, domain names, email addresses, logos, people, places, and events depicted herein are fictitious. No association with any real company, organization, product, domain name, email address, logo, person, place, or event is intended or should be inferred. This book expresses the author’s views and opinions. The information contained in this book is provided without any express, statutory, or implied warranties. Neither the authors, Microsoft Corporation, nor its resellers, or distributors will be held liable for any damages caused or alleged to be caused either directly or indirectly by this book. Contents Section 1: Administration ............................................................................................................................. 5 Running Microsoft SQL Server 2008 Analysis Services on Windows Server 2008 vs. Windows Server 2003 and Memory Preallocation: Lessons Learned .................................................................................. 6 Scripts for Stress Testing Analysis Services using ASCMD ...................................................................... 13 A Solution for Collecting Analysis Services Performance Data From Many Sources For Performance Analysis ................................................................................................................................................... 14 Compress Many-to-Many C# Utility ........................................................................................................ 15 Create a Processing Log Script Project .................................................................................................... 16 Powershell Scripts for SQL Server 2008 Analysis Services DMVs ........................................................... 17 Sample Robocopy Script to customer synchronize Analysis Services databases ................................... 18 Section 2: Analysis Services Performance ................................................................................................... 22 Analysis Services Synchronization Best Practices ................................................................................... 23 Microsoft SQL Server 2008 Analysis Services Consolidation Best Practices ........................................... 28 Analysis Services Distinct Count Optimization Using Solid State Devices .............................................. 45 Excel, Cube Formulas, Analysis Services, Performance, Network Latency, and Connection Strings...... 57 Reintroducing Usage-Based Optimization in SQL Server 2008 Analysis Services ................................... 65 Sections 3: Analysis Services Scaleout ........................................................................................................ 72 Analysis Services Load Balancing Solution .............................................................................................. 73 SSAS Monitoring Scripts For Management Data Warehouse ................................................................. 74 ASLB Toolkit ............................................................................................................................................ 75 Section 4: Real World Scenarios ................................................................................................................. 77 Edgenet Realizes the Power of Dynamic IT and Self-Service BI .............................................................. 78 Section 5: Reporting Services ...................................................................................................................... 85 Reporting Services Scale-Out Deployment Best Practices ...................................................................... 87 Report Server Catalog Best Practices ...................................................................................................... 96 Reporting Services Performance Optimizations ................................................................................... 102 Reporting Services Scale-Out Architecture ........................................................................................... 110 SQL Server Reporting Services Disaster Recovery Case Study .............................................................. 116 Scaling Up Reporting Services 2008 vs. Reporting Services 2005: Lessons Learned ............................ 125 Reporting Services Performance in SharePoint Integrated Mode in SQL Server 2008 R2 ................... 139 Deploying a Business Intelligence Solution Using SharePoint 2007, SQL Server 2008 Reporting Services, and PerformancePoint Monitoring Server 2007 with Kerberos ........................................................... 149 Section 6: SQL Server Integration Services ............................................................................................... 159 Increasing Throughput of Pipelines by Splitting Synchronous Transformations into Multiple Tasks .. 160 Moving Large Amounts of Data Between Oracle and SQL Server: Findings and Observations............ 166 SSIS Package For Collecting SSAS DMV Data......................................................................................... 170 The “Balanced Data Distributor” for SSIS ............................................................................................. 171 Section 7: SQL Top 10 ............................................................................................................................... 174 Top 10 Performance and Productivity Reasons to Use SQL Server 2008 for Your Business Intelligence Solutions ............................................................................................................................................... 175 Top 10 SQL Server Integration Services Best Practices ......................................................................... 187 Analysis Services Query Performance Top 10 Best Practices ............................................................... 196 Section 1: Administration Running Microsoft SQL Server 2008 Analysis Services on Windows Server 2008 vs. Windows Server 2003 and Memory Preallocation: Lessons Learned Introduction SQL Server 2005 Service Pack 2 introduced a new advanced configuration setting for Analysis Services that enables an administrator to specify that a certain percentage of memory be allocated to Analysis Services when the service starts (memory preallocation). This configuration setting was introduced because the Microsoft Windows Server 2003 operating system did not scale well with many small virtual memory allocations due to memory fragmentation and inefficiencies in retrieving information already located in memory. With the use of this configuration setting with 64- bit versions of Windows Server 2003 on multiprocessor computers (specifically computers with 16, 32, and 64 cores), the SQL Customer Advisor Team (SQL CAT) observed significant performance gains during Analysis Services processing and querying when memory use substantially exceeds 3 GB. Indeed, when Unisys and Microsoft achieved a record-setting benchmark for Analysis Services cube processing, the memory preallocation configuration setting for Analysis Services was set to 20 rather than to the default setting of 0 (resulting in 20% of physical memory being allocated to Analysis Services). For more information about this benchmark, see Data Warehouse Performance: Unisys and Microsoft Achieve Record-setting Benchmark. The memory manager in Windows Server 2008 includes many performance and scalability enhancements, one of which is a change in the algorithm for scanning pages in memory. Additional memory manager improvements include the following: · Kernel virtual address space is dynamically allocated to improve scalability when a greater number of processors and larger memory configurations are used. The sizes of paged and non-paged memory pools are no longer fixed, so manually reconfiguring systems to prevent resource imbalances is not required. Previously, the sizes of some resources, such as memory pools, were set by registry keys or by SKU. · Kernel virtual address space is made more available in x86 architectures by more efficiently using space in kernel- mode stacks in recursive calls into the kernel. · When a kernel-mode component allocates more than a page of memory, the memory manager now uses the pool memory between