DOCSLIB.ORG

Presto: the Definitive Guide

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Presto: the Definitive Guide Presto The Definitive Guide SQL at Any Scale, on Any Storage, in Any Environment Compliments of Matt Fuller, Manfred Moser & Martin Traverso Virtual Book Tour Starburst presents Presto: The Definitive Guide Register Now! Starburst is hosting a virtual book tour series where attendees will: Meet the authors: • Meet the authors from the comfort of your own home Matt Fuller • Meet the Presto creators and participate in an Ask Me Anything (AMA) session with the book Manfred Moser authors + Presto creators • Meet special guest speakers from Martin your favorite podcasts who will Traverso moderate the AMA Register here to save your spot. Praise for Presto: The Definitive Guide This book provides a great introduction to Presto and teaches you everything you need to know to start your successful usage of Presto. —Dain Sundstrom and David Phillips, Creators of the Presto Projects and Founders of the Presto Software Foundation Presto plays a key role in enabling analysis at Pinterest. This book covers the Presto essentials, from use cases through how to run Presto at massive scale. —Ashish Kumar Singh, Tech Lead, Bigdata Query Processing Platform, Pinterest Presto has set the bar in both community-building and technical excellence for lightning- fast analytical processing on stored data in modern cloud architectures. This book is a must-read for companies looking to modernize their analytics stack. —Jay Kreps, Cocreator of Apache Kafka, Cofounder and CEO of Confluent Presto has saved us all—both in academia and industry—countless hours of work, allowing us all to avoid having to write code to manage distributed query processing. We’re so grateful to have a high-quality open source distributed SQL engine to start from, enabling us to focus on innovating in new areas instead of reinventing the wheel for each new distributed data system project. —Daniel Abadi, Professor of Computer Science, University of Maryland, College Park Presto: The Definitive Guide SQL at Any Scale, on Any Storage, in Any Environment Matt Fuller, Manfred Moser, and Martin Traverso Beijing Boston Farnham Sebastopol Tokyo Presto: The Definitive Guide by Matt Fuller, Manfred Moser, and Martin Traverso Copyright © 2020 Matt Fuller, Martin Traverso, and Simpligility Technologies Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or [email protected]. Acquisition Editor: Jonathan Hassell Indexer: Potomac Indexing, LLC Development Editor: Michele Cronin Interior Designer: David Futato Production Editor: Elizabeth Kelly Cover Designer: Karen Montgomery Copyeditor: Sharon Wilkey Illustrator: Rebecca Demarest Proofreader: Piper Editorial April 2020: First Edition Revision History for the First Edition 2020-04-03: First release See http://oreilly.com/catalog/errata.csp?isbn=9781492044277 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Presto: The Definitive Guide, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly and Starburst. See our statement of editorial inde‐ pendence. 978-1-49208-403-7 [LSI] Table of Contents Foreword. xiii Preface. xv Part I. Getting Started with Presto 1. Introducing Presto. 3 The Problems with Big Data 3 Presto to the Rescue 4 Designed for Performance and Scale 5 SQL-on-Anything 6 Separation of Data Storage and Query Compute Resources 7 Presto Use Cases 7 One SQL Analytics Access Point 7 Access Point to Data Warehouse and Source Systems 8 Provide SQL-Based Access to Anything 9 Federated Queries 10 Semantic Layer for a Virtual Data Warehouse 10 Data Lake Query Engine 11 SQL Conversions and ETL 11 Better Insights Due to Faster Response Times 11 Big Data, Machine Learning, and Artificial Intelligence 12 Other Use Cases 12 Presto Resources 12 Website 12 Documentation 13 Community Chat 13 v Source Code, License, and Version 14 Contributing 14 Book Repository 15 Iris Data Set 15 Flight Data Set 16 A Brief History of Presto 16 Conclusion 17 2. Installing and Configuring Presto. 19 Trying Presto with the Docker Container 19 Installing from Archive File 20 Java Virtual Machine 20 Python 21 Installation 21 Configuration 22 Adding a Data Source 23 Running Presto 24 Conclusion 24 3. Using Presto. 25 Presto Command-Line Interface 25 Getting Started 25 Pagination 28 History 28 Additional Diagnostics 28 Executing Queries 29 Output Formats 30 Ignoring Errors 30 Presto JDBC Driver 30 Downloading and Registering the Driver 32 Establishing a Connection to Presto 32 Presto and ODBC 35 Client Libraries 35 Presto Web UI 35 SQL with Presto 36 Concepts 37 First Examples 37 Conclusion 40 vi | Table of Contents Part II. Diving Deeper into Presto 4. Presto Architecture. 43 Coordinator and Workers in a Cluster 43 Coordinator 45 Discovery Service 46 Workers 46 Connector-Based Architecture 47 Catalogs, Schemas, and Tables 48 Query Execution Model 48 Query Planning 53 Parsing and Analysis 54 Initial Query Planning 54 Optimization Rules 57 Predicate Pushdown 57 Cross Join Elimination 58 TopN 58 Partial Aggregations 59 Implementation Rules 60 Lateral Join Decorrelation 60 Semi-Join (IN) Decorrelation 61 Cost-Based Optimizer 62 The Cost Concept 62 Cost of the Join 64 Table Statistics 65 Filter Statistics 66 Table Statistics for Partitioned Tables 67 Join Enumeration 68 Broadcast Versus Distributed Joins 68 Working with Table Statistics 70 Presto ANALYZE 70 Gathering Statistics When Writing to Disk 71 Hive ANALYZE 71 Displaying Table Statistics 72 Conclusion 72 5. Production-Ready Deployment. 73 Configuration Details 73 Server Configuration 73 Logging 75 Node Configuration 76 JVM Configuration 77 Table of Contents | vii Launcher 77 Cluster Installation 79 RPM Installation 80 Installation Directory Structure 81 Configuration 82 Uninstall Presto 82 Installation in the Cloud 82 Cluster Sizing Considerations 83 Conclusion 84 6. Connectors. 85 Configuration 86 RDBMS Connector Example PostgreSQL 87 Query Pushdown 88 Parallelism and Concurrency 90 Other RDBMS Connectors 90 Security 92 Presto TPC-H and TPC-DS Connectors 92 Hive Connector for Distributed Storage Data Sources 93 Apache Hadoop and Hive 94 Hive Connector 95 Hive-Style Table Format 96 Managed and External Tables 97 Partitioned Data 98 Loading Data 100 File Formats and Compression 102 MinIO Example 103 Non-Relational Data Sources 104 Presto JMX Connector 104 Black Hole Connector 106 Memory Connector 107 Other Connectors 107 Conclusion 108 7. Advanced Connector Examples. 109 Connecting to HBase with Phoenix 109 Key-Value Store Connector Example: Accumulo 110 Using the Presto Accumulo Connector 113 Predicate Pushdown in Accumulo 115 Apache Cassandra Connector 117 Streaming System Connector Example: Kafka 118 Document Store Connector Example: Elasticsearch 120 viii | Table of Contents Overview 120 Configuration and Usage 121 Query Processing 121 Full-Text Search 122 Summary 122 Query Federation in Presto 122 Extract, Transform, Load and Federated Queries 129 Conclusion 129 8. Using SQL in Presto. 131 Presto Statements 132 Presto System Tables 134 Catalogs 136 Schemas 137 Information Schema 138 Tables 139 Table and Column Properties 141 Copying an Existing Table 142 Creating a New Table from Query Results 143 Modifying a Table 144 Deleting a Table 144 Table Limitations from Connectors 144 Views 145 Session Information and Configuration 146 Data Types 147 Collection Data Types 149 Temporal Data Types 150 Type Casting 154 SELECT Statement Basics 155 WHERE Clause 157 GROUP BY and HAVING Clauses 158 ORDER BY and LIMIT Clauses 159 JOIN Statements 160 UNION, INTERSECT, and EXCEPT Clauses 161 Grouping Operations 162 WITH Clause 164 Subqueries 165 Scalar Subquery 165 EXISTS Subquery 166 Quantified Subquery 166 Deleting Data from a Table 167 Conclusion 167 Table of Contents | ix 9. Advanced SQL. 169 Functions and Operators Introduction 169 Scalar Functions and Operators 170 Boolean Operators 171 Logical Operators 172 Range Selection with the BETWEEN Statement 173 Value Detection with IS (NOT) NULL 174 Mathematical Functions and Operators 174 Trigonometric Functions 175 Constant and Random Functions 176 String Functions and Operators 176 Strings and Maps 177 Unicode 178 Regular Expressions 179 Unnesting Complex Data Types 182 JSON Functions 183 Date and Time Functions and Operators 184 Histograms 186 Aggregate Functions 187 Map Aggregate Functions 187 Approximate Aggregate Functions 189 Window Functions
Load more

Recommended publications
  • Online User's Guide MFC-J775DW
    Online User's Guide MFC-J775DW © 2017 Brother Industries, Ltd. All rights reserved. Home > Table of Contents Table of Contents Before You Use Your Brother Machine ............................................................................................... 1 Applicable Models .......................................................................................................................................... 2 Definitions of Notes ........................................................................................................................................ 3 Notice - Disclaimer of Warranties (USA and Canada) ................................................................................... 4 Trademarks .................................................................................................................................................... 5 Important Note ............................................................................................................................................... 6 Introduction to Your Brother Machine................................................................................................. 7 Before Using Your Machine ........................................................................................................................... 8 Control Panel Overview ................................................................................................................................. 9 LCD Overview .............................................................................................................................................
    [Show full text]
  • Administration and Configuration Guide
    Red Hat JBoss Data Virtualization 6.4 Administration and Configuration Guide This guide is for administrators. Last Updated: 2018-09-26 Red Hat JBoss Data Virtualization 6.4 Administration and Configuration Guide This guide is for administrators. Red Hat Customer Content Services Legal Notice Copyright © 2018 Red Hat, Inc. This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries. Node.js ® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
    [Show full text]
  • IPS Signature Release Note V9.17.79
    SOPHOS IPS Signature Update Release Notes Version : 9.17.79 Release Date : 19th January 2020 IPS Signature Update Release Information Upgrade Applicable on IPS Signature Release Version 9.17.78 CR250i, CR300i, CR500i-4P, CR500i-6P, CR500i-8P, CR500ia, CR500ia-RP, CR500ia1F, CR500ia10F, CR750ia, CR750ia1F, CR750ia10F, CR1000i-11P, CR1000i-12P, CR1000ia, CR1000ia10F, CR1500i-11P, CR1500i-12P, CR1500ia, CR1500ia10F Sophos Appliance Models CR25iNG, CR25iNG-6P, CR35iNG, CR50iNG, CR100iNG, CR200iNG/XP, CR300iNG/XP, CR500iNG- XP, CR750iNG-XP, CR2500iNG, CR25wiNG, CR25wiNG-6P, CR35wiNG, CRiV1C, CRiV2C, CRiV4C, CRiV8C, CRiV12C, XG85 to XG450, SG105 to SG650 Upgrade Information Upgrade type: Automatic Compatibility Annotations: None Introduction The Release Note document for IPS Signature Database Version 9.17.79 includes support for the new signatures. The following sections describe the release in detail. New IPS Signatures The Sophos Intrusion Prevention System shields the network from known attacks by matching the network traffic against the signatures in the IPS Signature Database. These signatures are developed to significantly increase detection performance and reduce the false alarms. Report false positives at [email protected], along with the application details. January 2020 Page 2 of 245 IPS Signature Update This IPS Release includes Two Thousand, Seven Hundred and Sixty Two(2762) signatures to address One Thousand, Nine Hundred and Thirty Eight(1938) vulnerabilities. New signatures are added for the following vulnerabilities: Name CVE–ID
    [Show full text]
  • Mariadb / Mysql for Web Developers
    www.fromdual.com MariaDB / MySQL for Web Developers Web Developer Congress 2020, remote Oli Sennhauser Senior MariaDB & MySQL Consultant at FromDual GmbH https://www.fromdual.com/presentations 1 / 27 About FromDual GmbH www.fromdual.com Support Consulting remote-DBA Training 2 / 27 Contents www.fromdual.com MariaDB / MySQL for Web Developers ➢ Databases ➢ Connecting to the database ➢ Basic database queries (SELECT) ➢ Changing Data (DML) ➢ Transactions ➢ Error Handling and Debugging ➢ Joining Tables ➢ Indexing 3 / 27 What are databases for? www.fromdual.com ● Primarily: Relational DBMS (RDBMS) ● Storing Business Information: ● CRM, ERP, Accounting, Shop, Booking, etc. ● What are they NOT for (non optimal)? ● Logs → Files, Logstash ● Images, PDFs, huge texts → Filer, Solr ● Trash → Waste bin 4 / 27 Different types of databases www.fromdual.com ● Flat files, CSV, ISAM ● Hierarchical database ● Relational databases (RDBMS) ● Network databases ● Object Oriented databases (OODBMS) ● Object Relational DBMS (ORDBMS) ● Graph databases ● Column Stores (MariaDB CS) ● "Document" Stores (JSON, MongoDB) ● Wide Column Stores (Cassandra, HBase) 5 / 27 Common Relational DBMS www.fromdual.com ● MariaDB ● more in the Web-Client-Server field (LAMP) ● MySQL ● more in the Web-Client-Server field (LAMP) ● PostgreSQL ● more in the fat-Client-Server Business Software field ● SQLite ● Not a real "Client-Server-DBMS" → Library ● Embedded DBMS (Industry, Firefox, etc.) 6 / 27 Connection to the DBMS www.fromdual.com ● GUI (MySQL Workbench, HeidiSQL) ● CLI (mariadb,
    [Show full text]
  • Respecting the Block Interface – Computational Storage Using Virtual Objects
    Respecting the block interface – computational storage using virtual objects Ian F. Adams, John Keys, Michael P. Mesnier Intel Labs Abstract immediately obvious, and trade-offs need to be made. Unfor- tunately, this often means that the parallel bandwidth of all Computational storage has remained an elusive goal. Though drives within a storage server will not be available. The same minimizing data movement by placing computation close to problem exists within a single SSD, as the internal NAND storage has quantifiable benefits, many of the previous at- flash bandwidth often exceeds that of the storage controller. tempts failed to take root in industry. They either require a Enter computational storage, an oft-attempted approach to departure from the widespread block protocol to one that address the data movement problem. By keeping computa- is more computationally-friendly (e.g., file, object, or key- tion physically close to its data, we can avoid costly I/O. Var- value), or they introduce significant complexity (state) on top ious designs have been pursued, but the benefits have never of the block protocol. been enough to justify a new storage protocol. In looking We participated in many of these attempts and have since back at the decades of computational storage research, there concluded that neither a departure from nor a significant ad- is a common requirement that the block protocol be replaced dition to the block protocol is needed. Here we introduce a (or extended) with files, objects, or key-value pairs – all of block-compatible design based on virtual objects. Like a real which provide a convenient handle for performing computa- object (e.g., a file), a virtual object contains the metadata that tion.
    [Show full text]
  • Opera Mini Application for Android
    Opera Mini Application For Android Wat theologized his eternities goggling deathy, but quick-frozen Mohammed never hammer so unshakably. Fain and neverfringillid headline Tyrone sonever lambently. reapplied his proles! Tracie meows his bibulousness underdevelop someplace, but unrimed Ephrayim This application lies in early on this one knows of applications stored securely for example by that? Viber account to provide only be deactivated since then. Opera Mini is a super lightweight browser that loads web pages faster than what every other browser available. Opera Mini Browser Latest News Photos Videos on Opera. The Opera Mini for Android lets you do everything you any to online without wasting your fireplace plan It's stand fast safe mobile web browser that saves you tons of. Analysis of tomorrow with a few other. The mini application for opera android open multiple devices. Just with our site on a view flash drives against sim swap scammers? Thanks for better alternative software included in multitasking is passionate about how do you can browse, including sms charges may not part of mail and features. Other download option for opera mini Hospedajes Mirta. Activating it for you are you want. Opera mini 16 beta android app has a now released and before downloading the read or full review covering all the features here. It only you sign into your web page title is better your computer. The Opera Mini works the tender as tide original Opera for Android This app update features a similar appearance and functionality but thrive now displays Facebook. With google pixel exclusive skin smoothing makeover tool uses of your computer in total, control a light.
    [Show full text]
  • Sql Server System Tables
    Sql Server System Tables Thirteen and halcyon Bernd often purrs some knocking-shops alright or moit sanely. Fitz sculpts nuttily. Refulgent Mark fixings resistively, he interdicts his parasitology very absently. Why does pressure in sql developer to temp table mytable will be listed below list stored system tables in Snowflake via JDBC and has a many new database features. Definition You will be navigated to the data definition of CDS view. Sql server system as sql. He has to sql to use existing table id number of the each request is often simply need sql server system tables that are the conditions for any special feature. Tb database management studio provides various ways of sql server system tables and restore in this function by yourself! You can trick the script limits in the specific database user account reflect the Script Quotas page. Filegroups allowing us assume that sql server system dynamic sql server system tables. The sql server system tables? Generate DDL and DML SQL script output and execution. Even though it can be accessible in the sql server object explorer, but when we write the query it can be customized. Exploring system tables views SPs etc SQL Studies. Id and sql server from sql server system tables to get space for each execution plans manually. And grow addition, whenever a shower is detached or is shutdown, all rows associated with different database are removed. For is created in database role to fix the parameters in flow, sql server system tables and db_ddladmin, it has to. Thanks for sql server system tables have sql server system views and the part of.
    [Show full text]
  • Synthesis and Development of a Big Data Architecture for the Management of Radar Measurement Data
    1 Faculty of Electrical Engineering, Mathematics & Computer Science Synthesis and Development of a Big Data architecture for the management of radar measurement data Alex Aalbertsberg Master of Science Thesis November 2018 Supervisors: dr. ir. Maurice van Keulen (University of Twente) prof. dr. ir. Mehmet Aks¸it (University of Twente) dr. Doina Bucur (University of Twente) ir. Ronny Harmanny (Thales) University of Twente P.O. Box 217 7500 AE Enschede The Netherlands Approval Internship report/Thesis of: …………………………………………………………………………………………………………Alexander P. Aalbertsberg Title: …………………………………………………………………………………………Synthesis and Development of a Big Data architecture for the management of radar measurement data Educational institution: ………………………………………………………………………………..University of Twente Internship/Graduation period:…………………………………………………………………………..2017-2018 Location/Department:.…………………………………………………………………………………435 Advanced Development, Delft Thales Supervisor:……………………………………………………………………………R. I. A. Harmanny This report (both the paper and electronic version) has been read and commented on by the supervisor of Thales Netherlands B.V. In doing so, the supervisor has reviewed the contents and considering their sensitivity, also information included therein such as floor plans, technical specifications, commercial confidential information and organizational charts that contain names. Based on this, the supervisor has decided the following: o This report is publicly available (Open). Any defence may take place publicly and the report may be included in public libraries and/or published in knowledge bases. • o This report and/or a summary thereof is publicly available to a limited extent (Thales Group Internal). tors . It will be read and reviewed exclusively by teachers and if necessary by members of the examination board or review ? committee. The content will be kept confidential and not disseminated through publication or inclusion in public libraries and/or knowledge bases.
    [Show full text]
  • CA Network Flow Analysis Release Notes
    CA Network Flow Analysis Release Notes Release 9.1.3 This Documentation, which includes embedded help systems and electronically distributed materials, (hereinafter referred to as the “Documentation”) is for your informational purposes only and is subject to change or withdrawal by CA at any time. This Documentation may not be copied, transferred, reproduced, disclosed, modified or duplicated, in whole or in part, without the prior written consent of CA. This Documentation is confidential and proprietary information of CA and may not be disclosed by you or used for any purpose other than as may be permitted in (i) a separate agreement between you and CA governing your use of the CA software to which the Documentation relates; or (ii) a separate confidentiality agreement between you and CA. Notwithstanding the foregoing, if you are a licensed user of the software product(s) addressed in the Documentation, you may print or otherwise make available a reasonable number of copies of the Documentation for internal use by you and your employees in connection with that software, provided that all CA copyright notices and legends are affixed to each reproduced copy. The right to print or otherwise make available copies of the Documentation is limited to the period during which the applicable license for such software remains in full force and effect. Should the license terminate for any reason, it is your responsibility to certify in writing to CA that all copies and partial copies of the Documentation have been returned to CA or destroyed. TO THE EXTENT PERMITTED BY APPLICABLE LAW, CA PROVIDES THIS DOCUMENTATION “AS IS” WITHOUT WARRANTY OF ANY KIND, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT.
    [Show full text]
  • CADET: Computer Assisted Discovery Extraction and Translation
    CADET: Computer Assisted Discovery Extraction and Translation Benjamin Van Durme, Tom Lippincott, Kevin Duh, Deana Burchfield, Adam Poliak, Cash Costello, Tim Finin, Scott Miller, James Mayfield Philipp Koehn, Craig Harman, Dawn Lawrie, Chandler May, Max Thomas Annabelle Carrell, Julianne Chaloux, Tongfei Chen, Alex Comerford Mark Dredze, Benjamin Glass, Shudong Hao, Patrick Martin, Pushpendre Rastogi Rashmi Sankepally, Travis Wolfe, Ying-Ying Tran, Ted Zhang Human Language Technology Center of Excellence, Johns Hopkins University Abstract Computer Assisted Discovery Extraction and Translation (CADET) is a workbench for helping knowledge workers find, la- bel, and translate documents of interest. It combines a multitude of analytics together with a flexible environment for customiz- Figure 1: CADET concept ing the workflow for different users. This open-source framework allows for easy development of new research prototypes using a micro-service architecture based atop Docker and Apache Thrift.1 1 Introduction CADET is an integrated workbench for helping knowledge workers discover, extract, and translate Figure 2: Discovery user interface useful information. The user interface (Figure1) is based on a domain expert starting with a large information in structured form. To do so, she ex- collection of data, wishing to discover the subset ports the search results to our Extraction interface, that is most salient to their goals, and exporting where she can provide annotations to help train an the results to tools for either extraction of specific information extraction system. The Extraction in- information of interest or interactive translation. terface allows the user to label any text span using For example, imagine a humanitarian aid any schema, and also incorporates active learning worker with a large collection of social media to complement the discovery process in selecting messages obtained in the aftermath of a natural data to annotate.
    [Show full text]
  • Security Analysis of Firefox Webextensions
    6.857: Computer and Network Security Due: May 16, 2018 Security Analysis of Firefox WebExtensions Srilaya Bhavaraju, Tara Smith, Benny Zhang srilayab, tsmith12, felicity Abstract With the deprecation of Legacy addons, Mozilla recently introduced the WebExtensions API for the development of Firefox browser extensions. WebExtensions was designed for cross-browser compatibility and in response to several issues in the legacy addon model. We performed a security analysis of the new WebExtensions model. The goal of this paper is to analyze how well WebExtensions responds to threats in the previous legacy model as well as identify any potential vulnerabilities in the new model. 1 Introduction Firefox release 57, otherwise known as Firefox Quantum, brings a large overhaul to the open-source web browser. Major changes with this release include the deprecation of its initial XUL/XPCOM/XBL extensions API to shift to its own WebExtensions API. This WebExtensions API is currently in use by both Google Chrome and Opera, but Firefox distinguishes itself with further restrictions and additional functionalities. Mozilla’s goals with the new extension API is to support cross-browser extension development, as well as offer greater security than the XPCOM API. Our goal in this paper is to analyze how well the WebExtensions model responds to the vulnerabilities present in legacy addons and discuss any potential vulnerabilities in the new model. We present the old security model of Firefox extensions and examine the new model by looking at the structure, permissions model, and extension review process. We then identify various threats and attacks that may occur or have occurred before moving onto recommendations.
    [Show full text]
  • DBEAVER Universal DB Manager
    PG Conf 2017 PostgreSQL UI Tools DBEAVER universal DB manager Sergey Rider, Tech leader, developer Andrew Khitrin, PostgreSQL/Oracle expert OVERVIEW 2017 2015 2013 DBeaver Enterprise GitHub - a edition new home for 2012 Beaver met Beaver friends Hello 2011 big-big world Newborn Beaver PEOPLE COMMUNITY AUDIENCE • 5 major contributors • More than 500 thousands of end-users around the world • More than 30 regular (approx.) contributors • Users from more than 100 • More than 400 feature requests countries • Worldwide dev team: US, • Developers, DBAs, data Canada, Germany, Russia, analytics, content editors, China students and teachers FEATURES One tool fits all Data, metadata, SQL, maintenance Code (stored procedures, triggers, etc) Model (ER) Data export and migration …..and more POSTGRES PAIN POINTS FOR THE UI CLIENT • A lot of language (plpgsql, java, python, tcl) • Types mapping in plpgsql (bigint or int8 ?) • Source code 'generated part' in plsql • External profiler • Custom types POSTGRES PROBLEMS New hot features support (PG10 - partitions) A lot of important functionality depend on optional extension (debug,waits) Custom data types support (arrays, hstore, postgis, etc) JDBC issues (custom types, streaming, transactions) PLANS PG/SQL and PLSQL debugger Execution plan analyzer (and more plan vizual features) Performance metric and monitoring (we have a dream about the universal DB performance agent) Deep version control integration for the procedural language FOCUS PG/SQL debugger Execution plan analyzer Better support of PG10 OPEN SOURCE And we are open too You can take part • In development • In testing • In code review • ….. and other things LET’S DISCUSS Better understanding of PG community needs (regarding UI tools) Look through other GUI Discuss problems .
    [Show full text]