Ztuned: Automated SQL Tuning Through Trial and (Sometimes) Error
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Oracle Database Vault DBA Administrative Best Practices
Oracle Database Vault DBA Administrative Best Practices ORACLE WHITE PAPER | MAY 2015 Table of Contents Introduction 2 Database Administration Tasks Summary 3 General Database Administration Tasks 4 Managing Database Initialization Parameters 4 Scheduling Database Jobs 5 Administering Database Users 7 Managing Users and Roles 7 Managing Users using Oracle Enterprise Manager 8 Creating and Modifying Database Objects 8 Database Backup and Recovery 8 Oracle Data Pump 9 Security Best Practices for using Oracle RMAN 11 Flashback Table 11 Managing Database Storage Structures 12 Database Replication 12 Oracle Data Guard 12 Oracle Streams 12 Database Tuning 12 Database Patching and Upgrade 14 Oracle Enterprise Manager 16 Managing Oracle Database Vault 17 Conclusion 20 1 | ORACLE DATABASE VAULT DBA ADMINISTRATIVE BEST PRACTICES Introduction Oracle Database Vault provides powerful security controls for protecting applications and sensitive data. Oracle Database Vault prevents privileged users from accessing application data, restricts ad hoc database changes and enforces controls over how, when and where application data can be accessed. Oracle Database Vault secures existing database environments transparently, eliminating costly and time consuming application changes. With the increased sophistication and number of attacks on data, it is more important than ever to put more security controls inside the database. However, most customers have a small number of DBAs to manage their databases and cannot afford having dedicated people to manage their database security. Database consolidation and improved operational efficiencies make it possible to have even less people to manage the database. Oracle Database Vault controls are flexible and provide security benefits to customers even when they have a single DBA. -
2 Day + Performance Tuning Guide
Oracle® Database 2 Day + Performance Tuning Guide 21c F32092-02 August 2021 Oracle Database 2 Day + Performance Tuning Guide, 21c F32092-02 Copyright © 2007, 2021, Oracle and/or its affiliates. Contributors: Glenn Maxey, Rajesh Bhatiya, Lance Ashdown, Immanuel Chan, Debaditya Chatterjee, Maria Colgan, Dinesh Das, Kakali Das, Karl Dias, Mike Feng, Yong Feng, Andrew Holdsworth, Kevin Jernigan, Caroline Johnston, Aneesh Kahndelwal, Sushil Kumar, Sue K. Lee, Herve Lejeune, Ana McCollum, David McDermid, Colin McGregor, Mughees Minhas, Valarie Moore, Deborah Owens, Mark Ramacher, Uri Shaft, Susan Shepard, Janet Stern, Stephen Wexler, Graham Wood, Khaled Yagoub, Hailing Yu, Michael Zampiceni This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. -
SAQE: Practical Privacy-Preserving Approximate Query Processing for Data Federations
SAQE: Practical Privacy-Preserving Approximate Query Processing for Data Federations Johes Bater Yongjoo Park Xi He Northwestern University University of Illinois (UIUC) University of Waterloo [email protected] [email protected] [email protected] Xiao Wang Jennie Rogers Northwestern University Northwestern University [email protected] [email protected] ABSTRACT 1. INTRODUCTION A private data federation enables clients to query the union of data Querying the union of multiple private data stores is challeng- from multiple data providers without revealing any extra private ing due to the need to compute over the combined datasets without information to the client or any other data providers. Unfortu- data providers disclosing their secret query inputs to anyone. Here, nately, this strong end-to-end privacy guarantee requires crypto- a client issues a query against the union of these private records graphic protocols that incur a significant performance overhead as and he or she receives the output of their query over this shared high as 1,000× compared to executing the same query in the clear. data. Presently, systems of this kind use a trusted third party to se- As a result, private data federations are impractical for common curely query the union of multiple private datastores. For example, database workloads. This gap reveals the following key challenge some large hospitals in the Chicago area offer services for a cer- in a private data federation: offering significantly fast and accurate tain percentage of the residents; if we can query the union of these query answers without compromising strong end-to-end privacy. databases, it may serve as invaluable resources for accurate diag- To address this challenge, we propose SAQE, the Secure Ap- nosis, informed immunization, timely epidemic control, and so on. -
An End-To-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning
An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning Ji Zhangx, Yu Liux, Ke Zhoux , Guoliang Liz, Zhili Xiaoy, Bin Chengy, Jiashu Xingy, Yangtao Wangx, Tianheng Chengx, Li Liux, Minwei Ranx, and Zekang Lix xWuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, China zTsinghua University, China, yTencent Inc., China {jizhang, liu_yu, k.zhou, ytwbruce, vic, lillian_hust, mwran, zekangli}@hust.edu.cn [email protected]; {tomxiao, bencheng, flacroxing}@tencent.com ABSTRACT our model and improves efficiency of online tuning. We con- Configuration tuning is vital to optimize the performance ducted extensive experiments under 6 different workloads of database management system (DBMS). It becomes more on real cloud databases to demonstrate the superiority of tedious and urgent for cloud databases (CDB) due to the di- CDBTune. Experimental results showed that CDBTune had a verse database instances and query workloads, which make good adaptability and significantly outperformed the state- the database administrator (DBA) incompetent. Although of-the-art tuning tools and DBA experts. there are some studies on automatic DBMS configuration tuning, they have several limitations. Firstly, they adopt a 1 INTRODUCTION pipelined learning model but cannot optimize the overall The performance of database management systems (DBMSs) performance in an end-to-end manner. Secondly, they rely relies on hundreds of tunable configuration knobs. Supe- on large-scale high-quality training samples which are hard rior knob settings can improve the performance for DBMSs to obtain. Thirdly, there are a large number of knobs that (e.g., higher throughput and lower latency). However, only a are in continuous space and have unseen dependencies, and few experienced database administrators (DBAs) master the they cannot recommend reasonable configurations in such skills of setting appropriate knob configurations. -
Oracle Performance Tuning Checklist
Oracle Performance Tuning Checklist Griseous Wayne rear afire. Relaxing Rutger gloze fermentation. Good-natured Renault limb, his retaker wallowers capes sedately. This uses akismet to remove things like someone in sql, and best used in those rows and oracle performance tuning is used inside of system Before being stressed, you know this essentially uses cookies could do not mentioned reduce the nonclustered index fragmentation and oracle performance tuning advisor before purchasing additional encryption of both. Tips and features such features available in other tools to oracle performance tuning checklist. Tuning an Oracle database Use Automatic Workload Repository AWR reports to collect diagnostics data Review essential top-timed events for potential bottlenecks Monitor the excellent for different top SQL statement You separate use an AWR report on this base Make against that the SQL execution plans are optimal. In mind when building it out of oracle performance tuning checklist, dbas will also has been defined in presentation layer of db trace or removed from. Maybe show lazy loaded dataspace will be configured on a fetch them? Create a sql query statements automatically add something about obiee design efforts on file as a performance oracle tuning checklist. Each replicated and performance checklist be ready time as smart scan is. Optimal size values within this? Even more visible, and analytics experts. Do some pain points for last validation report back an performance oracle tuning checklist data, oracle golden gate installation can configure rman backups have. This oracle performance tuning checklist is never used as html does. Performance Tuning Checklist JIRA Performance Tuning Best Practices and. -
Veritas: Shared Verifiable Databases and Tables in the Cloud
Veritas: Shared Verifiable Databases and Tables in the Cloud Lindsey Alleny, Panagiotis Antonopoulosy, Arvind Arasuy, Johannes Gehrkey, Joachim Hammery, James Huntery, Raghav Kaushiky, Donald Kossmanny, Jonathan Leey, Ravi Ramamurthyy, Srinath Settyy, Jakub Szymaszeky, Alexander van Renenz, Ramarathnam Venkatesany yMicrosoft Corporation zTechnische Universität München ABSTRACT A recent class of new technology under the umbrella term In this paper we introduce shared, verifiable database tables, a new "blockchain" [11, 13, 22, 28, 33] promises to solve this conundrum. abstraction for trusted data sharing in the cloud. For example, companies A and B could write the state of these shared tables into a blockchain infrastructure (such as Ethereum [33]). The KEYWORDS blockchain then gives them transaction properties, such as atomic- ity of updates, durability, and isolation, and the blockchain allows Blockchain, data management a third party to go through its history and verify the transactions. Both companies A and B would have to write their updates into 1 INTRODUCTION the blockchain, wait until they are committed, and also read any Our economy depends on interactions – buyers and sellers, suppli- state changes from the blockchain as a means of sharing data across ers and manufacturers, professors and students, etc. Consider the A and B. To implement permissioning, A and B can use a permis- scenario where there are two companies A and B, where B supplies sioned blockchain variant that allows transactions only from a set widgets to A. A and B are exchanging data about the latest price for of well-defined participants; e.g., Ethereum [33]. widgets and dates of when widgets were shipped. -
SQL Database Management Portal
APPENDIX A SQL Database Management Portal This appendix introduces you to the online SQL Database Management Portal built by Microsoft. It is a web-based application that allows you to design, manage, and monitor your SQL Database instances. Although the current version of the portal does not support all the functions of SQL Server Management Studio, the portal offers some interesting capabilities unique to SQL Database. Launching the Management Portal You can launch the SQL Database Management Portal (SDMP) from the Windows Azure Management Portal. Open a browser and navigate to https://manage.windowsazure.com, and then login with your Live ID. Once logged in, click SQL Databases on the left and click on your database from the list of available databases. This brings up the database dashboard, as seen in Figure A-1. To launch the management portal, click the Manage icon at the bottom. Figure A-1. SQL Database dashboard in Windows Azure Management Portal 257 APPENDIX A N SQL DATABASE MANAGEMENT PORTAL N Note You can also access the SDMP directly from a browser by typing https://sqldatabasename.database. windows.net, where sqldatabasename is the server name of your SQL Database. A new web page opens up that prompts you to log in to your SQL Database server. If you clicked through the Windows Azure Management Portal, the database name will be automatically filled and read-only. If you typed the URL directly in a browser, you will need to enter the database name manually. Enter a user name and password; then click Log on (Figure A-2). -
Oracle Database 2 Day + Performance Tuning Guide, 10G Release 2 (10.2) B28051-02
Oracle® Database 2 Day + Performance Tuning Guide 10g Release 2 (10.2) B28051-02 January 2008 Easy, Automatic, and Step-By-Step Performance Tuning Using Oracle Diagnostics Pack, Oracle Database Tuning Pack, and Oracle Enterprise Manager Oracle Database 2 Day + Performance Tuning Guide, 10g Release 2 (10.2) B28051-02 Copyright © 2006, 2008, Oracle. All rights reserved. Primary Author: Immanuel Chan Contributors: Karl Dias, Cecilia Grant, Connie Green, Andrew Holdsworth, Sushil Kumar, Herve Lejeune, Colin McGregor, Mughees Minhas, Valarie Moore, Deborah Owens, Mark Townsend, Graham Wood The Programs (which include both the software and documentation) contain proprietary information; they are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright, patent, and other intellectual and industrial property laws. Reverse engineering, disassembly, or decompilation of the Programs, except to the extent required to obtain interoperability with other independently created software or as specified by law, is prohibited. The information contained in this document is subject to change without notice. If you find any problems in the documentation, please report them to us in writing. This document is not warranted to be error-free. Except as may be expressly permitted in your license agreement for these Programs, no part of these Programs may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose. If the Programs are delivered to the United States Government or anyone licensing or using the Programs on behalf of the United States Government, the following notice is applicable: U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. -
Plan Stitch: Harnessing the Best of Many Plans
Plan Stitch: Harnessing the Best of Many Plans Bailu Ding Sudipto Das Wentao Wu Surajit Chaudhuri Vivek Narasayya Microsoft Research Redmond, WA 98052, USA fbadin, sudiptod, wentwu, surajitc, [email protected] ABSTRACT optimizer chooses multiple query plans for the same query, espe- Query performance regression due to the query optimizer select- cially for stored procedures and query templates. ing a bad query execution plan is a major pain point in produc- The query optimizer often makes good use of the newly-created indexes and statistics, and the resulting new query plan improves tion workloads. Commercial DBMSs today can automatically de- 1 tect and correct such query plan regressions by storing previously- in execution cost. At times, however, the latest query plan chosen executed plans and reverting to a previous plan which is still valid by the optimizer has significantly higher execution cost compared and has the least execution cost. Such reversion-based plan cor- to previously-executed plans, i.e., a plan regresses [5, 6, 31]. rection has relatively low risk of plan regression since the deci- The problem of plan regression is painful for production applica- sion is based on observed execution costs. However, this approach tions as it is hard to debug. It becomes even more challenging when ignores potentially valuable information of efficient subplans col- plans change regularly on a large scale, such as in a cloud database lected from other previously-executed plans. In this paper, we service with automatic index tuning. Given such a large-scale pro- propose a novel technique, Plan Stitch, that automatically and op- duction environment, detecting and correcting plan regressions in portunistically combines efficient subplans of previously-executed an automated manner becomes crucial. -
Exploring Query Re-Optimization in a Modern Database System
Radboud University Nijmegen Institute of Computing and Information Sciences Exploring Query Re-Optimization In a Modern Database System Master Thesis Computing Science Supervisor: Prof. dr. ir. Arjen P. Author: de Vries BSc Laurens N. Kuiper Second reader: Prof. dr. ir. Djoerd Hiemstra May 2021 Abstract The standard approach to query processing in database systems is to optimize before executing. When available statistics are accurate, optimization yields the optimal plan, and execution is as quick as it can be. However, when queries become more complex, the quality of statistics degrades, which leads to sub-optimal query plans, sometimes up to several orders of magnitude worse. Improving statistics for these queries is infeasible because the cardinality estimation error propagates exponentially through the plan. This problem can be alleviated through re-optimization, which is to progressively optimize the query, executing parts of the plan at a time, improving plan quality at the cost of additional overhead. In this work, re-optimization is implemented and simulated in DuckDB, a main-memory database system designed for analytical workloads, and evaluated on the Join Order Benchmark. Total plan cost of the benchmark is reduced by up to 44% using this method, and a reduction in end-to-end query latency of up to 20% is observed using only simple re-optimization schemes, showing the potential of this approach. Acknowledgements The past year has been a challenging and exciting year. From breaking my leg to starting a job, and finally finishing my thesis. I would like to thank everyone who supported me during this period, and helped me get here. -
Database Performance Study
Database Performance Study By Francois Charvet Ashish Pande Please do not copy, reproduce or distribute without the explicit consent of the authors. Table of Contents Part A: Findings and Conclusions…………p3 Scope and Methodology……………………………….p4 Database Performance - Background………………….p4 Factors…………………………………………………p5 Performance Monitoring………………………………p6 Solving Performance Issues…………………...............p7 Organizational aspects……………………...................p11 Training………………………………………………..p14 Type of database environments and normalizations…..p14 Part B: Interview Summaries………………p20 Appendices…………………………………...p44 Database Performance Study 2 Part A Findings and Conclusions Database Performance Study 3 Scope and Methodology The authors were contacted by a faculty member to conduct a research project for the IS Board at UMSL. While the original proposal would be to assist in SQL optimization and tuning at Company A1, the scope of such project would be very time-consuming and require specific expertise within the research team. The scope of the current project was therefore revised, and the new project would consist in determining the current standards and practices in the field of SQL and database optimization in a number of companies represented on the board. Conclusions would be based on a series of interviews with Database Administrators (DBA’s) from the different companies, and on current literature about the topic. The first meeting took place 6th February 2003, and interviews were held mainly throughout the spring Semester 2003. Results would be presented in a final paper, and a presentation would also be held at the end of the project. Individual summaries of the interviews conducted with the DBA’s are included in Part B. A representative set of questions used during the interviews is also included. -
Queryguard: Privacy-Preserving Latency-Aware Query Optimization for Edge Computing
QueryGuard: Privacy-preserving Latency-aware Query Optimization for Edge Computing Runhua Xu ∗, Balaji Palanisamy y and James Joshi z School of Computing and Information University of Pittsburgh, Pittsburgh, PA USA ∗[email protected], [email protected], [email protected] Abstract—The emerging edge computing paradigm has en- adopting distributed database techniques in edge computing abled applications having low response time requirements to brings new challenges and concerns. First, in contrast to meet the quality of service needs of applications by moving the cloud data centers where cloud servers are managed through computations to the edge of the network that is geographically closer to the end-users and end-devices. Despite the low latency strict and regularized policies, edge nodes may not have the advantages provided by the edge computing model, there are same degree of regulatory and monitoring oversight. This significant privacy risks associated with the adoption of edge may lead to higher privacy risks compared to that in cloud computing services for applications dealing with sensitive data. servers. In particular, when dealing with a join query, tra- In contrast to cloud data centers where system infrastructures are ditional distributed query processing techniques sometimes managed through strict and regularized policies, edge computing nodes are scattered geographically and may not have the same ship selected or projected data to different nodes, some of degree of regulatory and monitoring oversight. This can lead which may be untrusted or semi-trusted. Thus, such techniques to higher privacy risks for the data processed and stored at may lead to greater disclosure of private information within the edge nodes, thus making them less trusted.