DOCSLIB.ORG

SQL Server Indexing: Using a Low-Selectivity BIT Column First Can Be the Best Strategy

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
SQL Server Indexing: Using a Low-Selectivity BIT Column First Can Be the Best Strategy SQLCAT’s Guide to: Relational Engine Microsoft SQLCAT Team Summary: This ebook is a collection of the most popular technical notes, tools and blogs authored by the SQLCAT team and posted to their blog over the course of several years. It covers SQL technology from 2005 to 2012. Category: Guide & Reference Applies to: SQL Server 2005 to 2012 Source: SQLCAT Blog E-book publication date: September 2013 Copyright © 2012 by Microsoft Corporation All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any means without the written permission of the publisher. Microsoft and the trademarks listed at http://www.microsoft.com/about/legal/en/us/IntellectualProperty/Trademarks/EN- US.aspx are trademarks of the Microsoft group of companies. All other marks are property of their respective owners. The example companies, organizations, products, domain names, email addresses, logos, people, places, and events depicted herein are fictitious. No association with any real company, organization, product, domain name, email address, logo, person, place, or event is intended or should be inferred. This book expresses the author’s views and opinions. The information contained in this book is provided without any express, statutory, or implied warranties. Neither the authors, Microsoft Corporation, nor its resellers, or distributors will be held liable for any damages caused or alleged to be caused either directly o Contents Section 1: Administration ............................................................................................................................. 5 DBCC Checks and Terabyte-Scale Databases ............................................................................................ 6 Scheduling Sub-Minute Log Shipping in SQL Server 2008 ...................................................................... 12 Tuning Backup Compression Part 2 ........................................................................................................ 15 Restart SQL Audit Policy and Job ............................................................................................................ 25 SQL DMVStats Toolkit ............................................................................................................................. 26 Section 2: Database Design ......................................................................................................................... 27 SQL Server Partition Management Tool ................................................................................................. 47 Character data types versus number data types: are there any performance benefits? ...................... 28 The Many Benefits of Money…Data Type! ............................................................................................. 38 How many files should a database have? - Part 1: OLAP workloads ...................................................... 43 Section 3: Fast-track .................................................................................................................................... 47 Lessons Learned and Findings from a Large Fast-Track POC .................................................................. 49 Section 4: Performance ............................................................................................................................... 70 Top Tips for Maximizing the Performance & Scalability of Dynamics AX 2009 systems on SQL Server 2008 ........................................................................................................................................................ 71 Introduction: ......................................................................................................................................... 102 Conclusions: .......................................................................................................................................... 111 Top SQL Server 2005 Performance Issues for OLTP Applications ......................................................... 112 Table-Valued Functions and tempdb Contention ................................................................................. 114 Resolving PAGELATCH Contention on Highly Concurrent INSERT Workloads ...................................... 130 SQL Server Indexing: Using a Low-Selectivity BIT Column First Can Be the Best Strategy ..................... 85 Tuning the Performance of Backup Compression in SQL Server 2008 ................................................... 71 Maximizing Throughput with TVPs ....................................................................................................... 130 Bulk Loading Data into a Table with Concurrent Queries ..................................................................... 140 Section 5: Real World Scenarios ............................................................................................................... 146 Lessons Learned from Benchmarking a Tier 1 Core Banking ISV Solution - Temenos T24 ................... 147 Section 6: Replication ............................................................................................................................... 155 Initializing a Transactional Replication Subscriber from an Array-Based Snapshot ............................. 156 Upgrading Replication from SQL Server 2000 32-Bit to SQL Server 2008 64-Bit without re-initialization .............................................................................................................................................................. 167 Section 7: Service Broker .......................................................................................................................... 168 SQL Server Service Broker: Maintaining Identity Uniqueness Across Database Copies ....................... 169 Section 8: Troubleshooting ....................................................................................................................... 172 Diagnosing Transaction Log Performance Issues and Limits of the Log Manager ................................ 173 Eliminating Deadlocks Caused By Foreign Keys with Large Transactions ............................................. 180 Resolving scheduler contention for concurrent BULK INSERT .............................................................. 187 Response Time Analysis using Extended Events ................................................................................... 191 Memory Error Recovery in SQL Server 2012 ........................................................................................ 192 Section 9: SQL Top 10 ............................................................................................................................... 195 Top 10 Hidden Gems in SQL 2008 R2 .................................................................................................... 196 Top 10 SQL Server 2008 Features for the Database Administrator (DBA) ........................................... 209 Top 10 SQL Server 2008 Features for ISV Applications ......................................................................... 221 Top 10 Hidden Gems in SQL Server 2005 ............................................................................................. 196 Top 10 Best Practices for Building a Large Scale Relational Data Warehouse ..................................... 230 Storage Top 10 Best Practices ............................................................................................................... 196 Top 10 Best Practices for SQL Server Maintenance for SAP ................................................................. 234 Section 1: Administration DBCC Checks and Terabyte-Scale Databases 1. Overview: Historically, the SQL Server product team and the support organization have recommended that all maintenance plans for Microsoft® SQL Server® include running DBCC CHECKDB on a regular basis to detect and correct potential corruption in the database. In the case of mission-critical applications such as trading systems, medical records, and banking operations, many customers have interpreted this recommendation as requiring even a daily DBCC check to ensure that no errors are present before performing a backup operation. However, since the SQL Server 6.5 days, in which a daily DBCC CHECKDB may have been a best practice, there have been several trends - both challenges and opportunities -- that make it worth reconsidering the way that DBCC should fit into SQL Server database maintenance plans, especially for large databases. Challenges: Many mission-critical databases have grown to massive size. It is not unusual to find databases in the 5-20 terabyte range on SQL Server, in which a DBCC operation requires many hours to complete. This is usually incompatible with a daily maintenance window, if feasible at all on a 24x7 system. Every new version of SQL Server has expanded the range of logical checks performed by DBCC so that it has become more time-consuming to execute. For example, SQL Server 2005 introduced data purity checks to validate that data present in columns adheres to valid ranges for the data type. SQL Server 2005 also introduced checks that verify the integrity of indexed views. Opportunities: Enterprise-class storage subsystems and the widespread use of RAID, especially in mission-critical settings, have made the storage tier much less prone to physical corruption. SQL Server has introduced new mechanisms to detect the
Load more

Recommended publications
  • PL/SQL Multi-Diagrammatic Source Code Visualization
    SQLFlow: PL/SQL Multi-Diagrammatic Source Code Visualization Samir Tartir Ayman Issa Department of Computer Science University of the West of England (UWE), University of Georgia Coldharbour Lane, Frenchay, Bristol BS16 1QY Athens, Georgia 30602 United Kingdom USA Email: [email protected] Email: [email protected] ABSTRACT: A major problem in software introducing new problems to other code that was perfectly maintenance is the lack of a well-documented source code working before. Therefore, educating programmers about in software applications. This has led to serious the behaviors of the target code before start modifying it difficulties in software maintenance and evolution. In is not an easy task. Source code visualization is one of the particular, for those developers who are faced with the heavily used techniques in the literature to overcome this task of fixing or modifying a piece of code they never lack of documentation problem, and its consequent even knew existed before. Database triggers and knowledge-sharing problem [7, 9]. procedures are parts of almost every application, are typical examples of badly documented software Therefore, this research aims at investigating the current components being the hidden back end of software database source code visualization tools and identifying applications. Source code visualization is one of the the main open issues for research. Consequently, the heavily used techniques in the literature so as to mostly used PL/SQL database programming language has familiarize software developers with new source code, been selected for a flowcharting reverse engineering and compensate for the knowledge-sharing problem. prototype tool called "SQLFlow". SQLFlow has been Therefore, a new PL/SQL flowcharting reverse designed, implemented, and tested to parse the target engineering tool, SQLFlow, has been designed and PL/SQL code stored in database procedures, analyze its developed.
    [Show full text]
  • SQL Server 2017 on Linux Quick Start Guide | 4
    SQL Server 2017 on Linux Quick Start Guide Contents Who should read this guide? ........................................................................................................................ 4 Getting started with SQL Server on Linux ..................................................................................................... 5 Why SQL Server with Linux? ..................................................................................................................... 5 Supported platforms ................................................................................................................................. 5 Architectural changes ............................................................................................................................... 6 Comparing SQL on Windows vs. Linux ...................................................................................................... 6 SQL Server installation on Linux ................................................................................................................ 8 Installing SQL Server packages .................................................................................................................. 8 Configuration capabilities ....................................................................................................................... 11 Licensing .................................................................................................................................................. 12 Administering and
    [Show full text]
  • Microsoft and Cray to Unveil $25,000 Windows-Based Supercomputer
    AAll About Microsoft: l lCodeTracker A monthly look at Microsoft’s codenames and what they Areveal about the direction of the company. b o u t M i c r o s o f t : All About Microsoft CodeTracker Keeping track of Microsoft's myriad codenames is an (almost) full-time occupation. I know, as I spend a lot of my work hours tracking down the latest names in the hopes of being able to better keep tabs on what's coming next from the Redmondians. Each month, I'll be releasing an updated, downloadable version of the CodeTracker. I'll add new codenames -- arranged in alphabetical order by codename -- of forthcoming Microsoft products and technologies. I also will note timing changes (date slips, the release of a new test build, the disappearance of a planned deliverable) for entries that are already part of the Tracker. Once Microsoft releases the final version of a product or technology I've been tracking, I will remove it from the Tracker. In that way, the CodeTracker will remain focused on futures. (An aside about the Tracker: A question mark in place of an entry means I have insufficient information to hazard even an educated guess about a particular category.) If you have suggested new entries or corrections to existing ones, please drop me an e-mail at mjf at microsofttracker dot com. Thanks! Mary Jo Foley, Editor, ZDNet's "All About Microsoft" blog This Month's Theme: Big iron needs love, too If you went by nothing but blog and publication headlines, you might think mobile phones and slates are where all the innovation is these days.
    [Show full text]
  • Bag-Of-Features Image Indexing and Classification in Microsoft SQL Server Relational Database
    Bag-of-Features Image Indexing and Classification in Microsoft SQL Server Relational Database Marcin Korytkowski, Rafał Scherer, Paweł Staszewski, Piotr Woldan Institute of Computational Intelligence Cze¸stochowa University of Technology al. Armii Krajowej 36, 42-200 Cze¸stochowa, Poland Email: [email protected], [email protected] Abstract—This paper presents a novel relational database ar- data directly in the database files. The example of such an chitecture aimed to visual objects classification and retrieval. The approach can be Microsoft SQL Server where binary data is framework is based on the bag-of-features image representation stored outside the RDBMS and only the information about model combined with the Support Vector Machine classification the data location is stored in the database tables. MS SQL and is integrated in a Microsoft SQL Server database. Server utilizes a special field type called FileStream which Keywords—content-based image processing, relational integrates SQL Server database engine with NTFS file system databases, image classification by storing binary large object (BLOB) data as files in the file system. Microsoft SQL dialect (Transact-SQL) statements I. INTRODUCTION can insert, update, query, search, and back up FileStream data. Application Programming Interface provides streaming Thanks to content-based image retrieval (CBIR) access to the data. FileStream uses operating system cache for [1][2][3][4][5][6][7][8] we are able to search for similar caching file data. This helps to reduce any negative effects images and classify them [9][10][11][12][13]. Images can be that FileStream data might have on the RDBMS performance.
    [Show full text]
  • Socrates: the New SQL Server in the Cloud
    Socrates: The New SQL Server in the Cloud Panagiotis Antonopoulos, Alex Budovski, Cristian Diaconu, Alejandro Hernandez Saenz, Jack Hu, Hanuma Kodavalla, Donald Kossmann, Sandeep Lingam, Umar Farooq Minhas, Naveen Prakash, Vijendra Purohit, Hugh Qu, Chaitanya Sreenivas Ravella, Krystyna Reisteter, Sheetal Shrotri, Dixin Tang, Vikram Wakade Microsoft Azure & Microsoft Research ABSTRACT 1 INTRODUCTION The database-as-a-service paradigm in the cloud (DBaaS) The cloud is here to stay. Most start-ups are cloud-native. is becoming increasingly popular. Organizations adopt this Furthermore, many large enterprises are moving their data paradigm because they expect higher security, higher avail- and workloads into the cloud. The main reasons to move ability, and lower and more flexible cost with high perfor- into the cloud are security, time-to-market, and a more flexi- mance. It has become clear, however, that these expectations ble “pay-as-you-go” cost model which avoids overpaying for cannot be met in the cloud with the traditional, monolithic under-utilized machines. While all these reasons are com- database architecture. This paper presents a novel DBaaS pelling, the expectation is that a database runs in the cloud architecture, called Socrates. Socrates has been implemented at least as well as (if not better) than on premise. Specifically, in Microsoft SQL Server and is available in Azure as SQL DB customers expect a “database-as-a-service” to be highly avail- Hyperscale. This paper describes the key ideas and features able (e.g., 99.999% availability), support large databases (e.g., of Socrates, and it compares the performance of Socrates a 100TB OLTP database), and be highly performant.
    [Show full text]
  • Oracle Database Vault DBA Administrative Best Practices
    Oracle Database Vault DBA Administrative Best Practices ORACLE WHITE PAPER | MAY 2015 Table of Contents Introduction 2 Database Administration Tasks Summary 3 General Database Administration Tasks 4 Managing Database Initialization Parameters 4 Scheduling Database Jobs 5 Administering Database Users 7 Managing Users and Roles 7 Managing Users using Oracle Enterprise Manager 8 Creating and Modifying Database Objects 8 Database Backup and Recovery 8 Oracle Data Pump 9 Security Best Practices for using Oracle RMAN 11 Flashback Table 11 Managing Database Storage Structures 12 Database Replication 12 Oracle Data Guard 12 Oracle Streams 12 Database Tuning 12 Database Patching and Upgrade 14 Oracle Enterprise Manager 16 Managing Oracle Database Vault 17 Conclusion 20 1 | ORACLE DATABASE VAULT DBA ADMINISTRATIVE BEST PRACTICES Introduction Oracle Database Vault provides powerful security controls for protecting applications and sensitive data. Oracle Database Vault prevents privileged users from accessing application data, restricts ad hoc database changes and enforces controls over how, when and where application data can be accessed. Oracle Database Vault secures existing database environments transparently, eliminating costly and time consuming application changes. With the increased sophistication and number of attacks on data, it is more important than ever to put more security controls inside the database. However, most customers have a small number of DBAs to manage their databases and cannot afford having dedicated people to manage their database security. Database consolidation and improved operational efficiencies make it possible to have even less people to manage the database. Oracle Database Vault controls are flexible and provide security benefits to customers even when they have a single DBA.
    [Show full text]
  • Discovery Attender User Guide
    Welcome Sherpa Software's Discovery Attender is a software tool designed to automate the search and collection of electronically stored information across a variety of platforms. This dynamic solution sits behind the firewall and enables in-house talent to identify and gather responsive information in a timely, defensible manner. The Discovery Attender interface is designed to be intuitive and user friendly. The Welcome Screen (above) serves as the gateway to the product. From here you can directly open the PreSearch Tool or create a project. A project serves as a repository for a collection - related searches and associated result sets. Once a new pro- ject is created, the MAIN CONSOLE acts as the central hub to allow you to control the creation, processing, and organization of searches. From here, you can access a Search Wizard which guides you through the step-by- step process of creating a new search. The MAIN CONSOLE also provides access to the Result Management features to view, organize, export and report on the messages, attachments and files that are found during your custom searches. Where to Begin To effectively filter and produce data with Discovery Attender, you must first create a project . Once a project is created, follow the steps below to begin the process: Step Action Description 1 Create and Run A Search A setup wizard leads you through the process of selecting locations and criteria to include in your search. Once started, the chosen data stores are scanned to find items (called results) that match the selected criteria. Information and metadata from these results are stored in the search data- bases.
    [Show full text]
  • 2 Day + Performance Tuning Guide
    Oracle® Database 2 Day + Performance Tuning Guide 21c F32092-02 August 2021 Oracle Database 2 Day + Performance Tuning Guide, 21c F32092-02 Copyright © 2007, 2021, Oracle and/or its affiliates. Contributors: Glenn Maxey, Rajesh Bhatiya, Lance Ashdown, Immanuel Chan, Debaditya Chatterjee, Maria Colgan, Dinesh Das, Kakali Das, Karl Dias, Mike Feng, Yong Feng, Andrew Holdsworth, Kevin Jernigan, Caroline Johnston, Aneesh Kahndelwal, Sushil Kumar, Sue K. Lee, Herve Lejeune, Ana McCollum, David McDermid, Colin McGregor, Mughees Minhas, Valarie Moore, Deborah Owens, Mark Ramacher, Uri Shaft, Susan Shepard, Janet Stern, Stephen Wexler, Graham Wood, Khaled Yagoub, Hailing Yu, Michael Zampiceni This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S.
    [Show full text]
  • A Prolog-Based Proof Tool for Type Theory Taλ and Implicational Intuitionistic-Logic
    A Prolog-based Proof Tool for Type Theory TAλ and Implicational Intuitionistic-Logic L. Yohanes Stefanus University of Indonesia Depok 16424, Indonesia [email protected] and Ario Santoso∗ Technische Universit¨atDresden Dresden 01187, Germany [email protected] Abstract Studies on type theory have brought numerous important contributions to computer science. In this paper we present a GUI-based proof tool that provides assistance in constructing deductions in type theory and validating implicational intuitionistic-logic for- mulas. As such, this proof tool is a testbed for learning type theory and implicational intuitionistic-logic. This proof tool focuses on an important variant of type theory named TAλ, especially on its two core algorithms: the principal-type algorithm and the type inhabitant search algorithm. The former algorithm finds a most general type assignable to a given λ-term, while the latter finds inhabitants (closed λ-terms in β-normal form) to which a given type can be assigned. By the Curry{Howard correspondence, the latter algorithm provides provability for implicational formulas in intuitionistic logic. We elab- orate on how to implement those two algorithms declaratively in Prolog and the overall GUI-based program architecture. In our implementation, we make some modification to improve the performance of the principal-type algorithm. We have also built a web-based version of the proof tool called λ-Guru. 1 Introduction In 1902, Russell revealed the inconsistency in naive set theory. This inconsistency challenged the foundation of mathematics at that time. To solve that problem, type theory was introduced [1, 5]. As time goes on, type theory becomes widely used, it becomes a logician's standard tool, especially in proof theory.
    [Show full text]
  • An End-To-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning
    An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning Ji Zhangx, Yu Liux, Ke Zhoux , Guoliang Liz, Zhili Xiaoy, Bin Chengy, Jiashu Xingy, Yangtao Wangx, Tianheng Chengx, Li Liux, Minwei Ranx, and Zekang Lix xWuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, China zTsinghua University, China, yTencent Inc., China {jizhang, liu_yu, k.zhou, ytwbruce, vic, lillian_hust, mwran, zekangli}@hust.edu.cn [email protected]; {tomxiao, bencheng, flacroxing}@tencent.com ABSTRACT our model and improves efficiency of online tuning. We con- Configuration tuning is vital to optimize the performance ducted extensive experiments under 6 different workloads of database management system (DBMS). It becomes more on real cloud databases to demonstrate the superiority of tedious and urgent for cloud databases (CDB) due to the di- CDBTune. Experimental results showed that CDBTune had a verse database instances and query workloads, which make good adaptability and significantly outperformed the state- the database administrator (DBA) incompetent. Although of-the-art tuning tools and DBA experts. there are some studies on automatic DBMS configuration tuning, they have several limitations. Firstly, they adopt a 1 INTRODUCTION pipelined learning model but cannot optimize the overall The performance of database management systems (DBMSs) performance in an end-to-end manner. Secondly, they rely relies on hundreds of tunable configuration knobs. Supe- on large-scale high-quality training samples which are hard rior knob settings can improve the performance for DBMSs to obtain. Thirdly, there are a large number of knobs that (e.g., higher throughput and lower latency). However, only a are in continuous space and have unseen dependencies, and few experienced database administrators (DBAs) master the they cannot recommend reasonable configurations in such skills of setting appropriate knob configurations.
    [Show full text]
  • Oracle Performance Tuning Checklist
    Oracle Performance Tuning Checklist Griseous Wayne rear afire. Relaxing Rutger gloze fermentation. Good-natured Renault limb, his retaker wallowers capes sedately. This uses akismet to remove things like someone in sql, and best used in those rows and oracle performance tuning is used inside of system Before being stressed, you know this essentially uses cookies could do not mentioned reduce the nonclustered index fragmentation and oracle performance tuning advisor before purchasing additional encryption of both. Tips and features such features available in other tools to oracle performance tuning checklist. Tuning an Oracle database Use Automatic Workload Repository AWR reports to collect diagnostics data Review essential top-timed events for potential bottlenecks Monitor the excellent for different top SQL statement You separate use an AWR report on this base Make against that the SQL execution plans are optimal. In mind when building it out of oracle performance tuning checklist, dbas will also has been defined in presentation layer of db trace or removed from. Maybe show lazy loaded dataspace will be configured on a fetch them? Create a sql query statements automatically add something about obiee design efforts on file as a performance oracle tuning checklist. Each replicated and performance checklist be ready time as smart scan is. Optimal size values within this? Even more visible, and analytics experts. Do some pain points for last validation report back an performance oracle tuning checklist data, oracle golden gate installation can configure rman backups have. This oracle performance tuning checklist is never used as html does. Performance Tuning Checklist JIRA Performance Tuning Best Practices and.
    [Show full text]
  • Software License Agreement (EULA)
    Third-party Computer Software AutoVu™ ALPR cameras • angular-animate (https://docs.angularjs.org/api/ngAnimate) licensed under the terms of the MIT License (https://github.com/angular/angular.js/blob/master/LICENSE). © 2010-2016 Google, Inc. http://angularjs.org • angular-base64 (https://github.com/ninjatronic/angular-base64) licensed under the terms of the MIT License (https://github.com/ninjatronic/angular-base64/blob/master/LICENSE). © 2010 Nick Galbreath © 2013 Pete Martin • angular-translate (https://github.com/angular-translate/angular-translate) licensed under the terms of the MIT License (https://github.com/angular-translate/angular-translate/blob/master/LICENSE). © 2014 [email protected] • angular-translate-handler-log (https://github.com/angular-translate/bower-angular-translate-handler-log) licensed under the terms of the MIT License (https://github.com/angular-translate/angular-translate/blob/master/LICENSE). © 2014 [email protected] • angular-translate-loader-static-files (https://github.com/angular-translate/bower-angular-translate-loader-static-files) licensed under the terms of the MIT License (https://github.com/angular-translate/angular-translate/blob/master/LICENSE). © 2014 [email protected] • Angular Google Maps (http://angular-ui.github.io/angular-google-maps/#!/) licensed under the terms of the MIT License (https://opensource.org/licenses/MIT). © 2013-2016 angular-google-maps • AngularJS (http://angularjs.org/) licensed under the terms of the MIT License (https://github.com/angular/angular.js/blob/master/LICENSE). © 2010-2016 Google, Inc. http://angularjs.org • AngularUI Bootstrap (http://angular-ui.github.io/bootstrap/) licensed under the terms of the MIT License (https://github.com/angular- ui/bootstrap/blob/master/LICENSE).
    [Show full text]