The Pragmatic Wisdom of Michael Stonebraker Editor: Michael L
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Big Data Velocity in Plain English
Big Data Velocity in Plain English John Ryan Data Warehouse Solution Architect Table of Contents The Requirement . 1 What’s the Problem? . .. 2 Components Needed . 3 Data Capture . 3 Transformation . 3 Storage and Analytics . 4 The Traditional Solution . 6 The NewSQL Based Solution . 7 NewSQL Advantage . 9 Thank You. 10 About the Author . 10 ii The Requirement The assumed requirement is the ability to capture, transform and analyse data at potentially massive velocity in real time. This involves capturing data from millions of customers or electronic sensors, and transforming and storing the results for real time analysis on dashboards. The solution must minimise latency — the delay between a real world event and it’s impact upon a dashboard, to under a second. Typical applications include: • Monitoring Machine Sensors: Using embedded sensors in industrial machines or vehicles — typically referred to as The Internet of Things (IoT) . For example Progressive Insurance use real time speed and vehicle braking data to help classify accident risk and deliver appropriate discounts. Similar technology is used by logistics giant FedEx which uses SenseAware technology to provide near real-time parcel tracking. • Fraud Detection: To assess the risk of credit card fraud prior to authorising or declining the transaction. This can be based upon a simple report of a lost or stolen card, or more likely, an analysis of aggregate spending behaviour, aligned with machine learning techniques. • Clickstream Analysis: Producing real time analysis of user web site clicks to dynamically deliver pages, recommended products or services, or deliver individually targeted advertising. Big Data: Velocity in Plain English eBook 1 What’s the Problem? The primary challenge for real time systems architects is the potentially massive throughput required which could exceed a million transactions per second. -
Ingres 9.3 Migration Guide
Ingres® 9.3 Migration Guide ING-93-MG-02 This Documentation is for the end user's informational purposes only and may be subject to change or withdrawal by Ingres Corporation ("Ingres") at any time. This Documentation is the proprietary information of Ingres and is protected by the copyright laws of the United States and international treaties. It is not distributed under a GPL license. You may make printed or electronic copies of this Documentation provided that such copies are for your own internal use and all Ingres copyright notices and legends are affixed to each reproduced copy. You may publish or distribute this document, in whole or in part, so long as the document remains unchanged and is disseminated with the applicable Ingres software. Any such publication or distribution must be in the same manner and medium as that used by Ingres, e.g., electronic download via website with the software or on a CD- ROM. Any other use, such as any dissemination of printed copies or use of this documentation, in whole or in part, in another publication, requires the prior written consent from an authorized representative of Ingres. To the extent permitted by applicable law, INGRES PROVIDES THIS DOCUMENTATION "AS IS" WITHOUT WARRANTY OF ANY KIND, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT. IN NO EVENT WILL INGRES BE LIABLE TO THE END USER OR ANY THIRD PARTY FOR ANY LOSS OR DAMAGE, DIRECT OR INDIRECT, FROM THE USER OF THIS DOCUMENTATION, INCLUDING WITHOUT LIMITATION, LOST PROFITS, BUSINESS INTERRUPTION, GOODWILL, OR LOST DATA, EVEN IF INGRES IS EXPRESSLY ADVISED OF SUCH LOSS OR DAMAGE. -
Oracle Vs. Nosql Vs. Newsql Comparing Database Technology
Oracle vs. NoSQL vs. NewSQL Comparing Database Technology John Ryan Senior Solution Architect, Snowflake Computing Table of Contents The World has Changed . 1 What’s Changed? . 2 What’s the Problem? . .. 3 Performance vs. Availability and Durability . 3 Consistecy vs. Availability . 4 Flexibility vs . Scalability . 5 ACID vs. Eventual Consistency . 6 The OLTP Database Reimagined . 7 Achieving the Impossible! . .. 8 NewSQL Database Technology . 9 VoltDB . 10 MemSQL . 11 Which Applications Need NewSQL Technology? . 12 Conclusion . 13 About the Author . 13 ii The World has Changed The world has changed massively in the past 20 years. Back in the year 2000, a few million users connected to the web using a 56k modem attached to a PC, and Amazon only sold books. Now billions of people are using to their smartphone or tablet 24x7 to buy just about everything, and they’re interacting with Facebook, Twitter and Instagram. The pace has been unstoppable . Expectations have also changed. If a web page doesn’t refresh within seconds we’re quickly frustrated, and go elsewhere. If a web site is down, we fear it’s the end of civilisation as we know it. If a major site is down, it makes global headlines. Instant gratification takes too long! — Ladawn Clare-Panton Aside: If you’re not a seasoned Database Architect, you may want to start with my previous articles on Scalability and Database Architecture. Oracle vs. NoSQL vs. NewSQL eBook 1 What’s Changed? The above leads to a few observations: • Scalability — With potentially explosive traffic growth, IT systems need to quickly grow to meet exponential numbers of transactions • High Availability — IT systems must run 24x7, and be resilient to failure. -
The Declarative Imperative Experiences and Conjectures in Distributed Logic
The Declarative Imperative Experiences and Conjectures in Distributed Logic Joseph M. Hellerstein University of California, Berkeley [email protected] ABSTRACT The juxtaposition of these trends presents stark alternatives. The rise of multicore processors and cloud computing is putting Will the forecasts of doom and gloom materialize in a storm enormous pressure on the software community to find solu- that drowns out progress in computing? Or is this the long- tions to the difficulty of parallel and distributed programming. delayed catharsis that will wash away today’s thicket of im- At the same time, there is more—and more varied—interest in perative languages, preparing the ground for a more fertile data-centric programming languages than at any time in com- declarative future? And what role might the database com- puting history, in part because these languages parallelize nat- munity play in shaping this future, having sowed the seeds of urally. This juxtaposition raises the possibility that the theory Datalog over the last quarter century? of declarative database query languages can provide a foun- Before addressing these issues directly, a few more words dation for the next generation of parallel and distributed pro- about both crisis and opportunity are in order. gramming languages. 1.1 Urgency: Parallelism In this paper I reflect on my group’s experience over seven years using Datalog extensions to build networking protocols I would be panicked if I were in industry. and distributed systems. Based on that experience, I present — John Hennessy, President, Stanford University [35] a number of theoretical conjectures that may both interest the database community, and clarify important practical issues in The need for parallelism is visible at micro and macro scales. -
The Ingres Next Initiative Enabling the Cloud, Mobile, and Web Journeys for Our Actian X and Ingres Database Customers
The Ingres NeXt Initiative Enabling the Cloud, Mobile, and Web Journeys for our Actian X and Ingres Database Customers Any organization that has leveraged technology to help run their business Benefits knows that the costs associated with on-going maintenance and support for Accelerate Cost Savings legacy systems can often take the upwards of 50% of the budget. Analysts and • Seize near-term cost IT vendors tend to see these on-going investments as sub-optimum, but for reductions across your IT and mission critical systems that have served their organizations well for decades business operations and tight budgets, , the risk to operations and business continuity make • Leverage existing IT replacing these tried and tested mission critical systems impractical. In many investments and data sources cases, these essential systems contain thousands of hours of custom developed business logic investment. • Avoid new CAPEX cycles when Products such as Actian Ingres and, more recently Actian X Relational retiring legacy systems Database Management Systems (RDBMS), have proven Actian’s commitment to helping our customers maximize their existing investments without having Optimize Risk Mitigation to sacrifice modern functionality. Ultimately, this balance in preservation and • Preserve investments in existing business logic while innovation with Ingres has taken us to three areas of development and taking advantage of modern deployment: continuation of heritage UNIX variants on their associated capabilities hardware, current Linux and Windows for on-premise and virtualized environments and most importantly, enabling migration to the public cloud. • Same database across all platforms Based on a recent survey of our Actian X and Ingres customer base, we are seeing a growing desire to move database workloads to the Cloud. -
Oracle Vs. Nosql Vs. Newsql Comparing Database Technology
Oracle vs. NoSQL vs. NewSQL Comparing Database Technology John Ryan Data Warehouse Solution Architect, UBS Table of Contents The World has Changed . 1 What’s Changed? . 2 What’s the Problem? . .. 3 Performance vs. Availability and Durability . 3 Consistecy vs. Availability . 4 Flexibility vs . Scalability . 5 ACID vs. Eventual Consistency . 6 The OLTP Database Reimagined . 7 Achieving the Impossible! . .. 8 NewSQL Database Technology . 9 VoltDB . 10 MemSQL . 11 Which Applications Need NewSQL Technology? . 12 Conclusion . 13 About the Author . 13 ii The World has Changed The world has changed massively in the past 20 years. Back in the year 2000, a few million users connected to the web using a 56k modem attached to a PC, and Amazon only sold books. Now billions of people are using to their smartphone or tablet 24x7 to buy just about everything, and they’re interacting with Facebook, Twitter and Instagram. The pace has been unstoppable . Expectations have also changed. If a web page doesn’t refresh within seconds we’re quickly frustrated, and go elsewhere. If a web site is down, we fear it’s the end of civilisation as we know it. If a major site is down, it makes global headlines. Instant gratification takes too long! — Ladawn Clare-Panton Aside: If you’re not a seasoned Database Architect, you may want to start with my previous articles on Scalability and Database Architecture. Oracle vs. NoSQL vs. NewSQL eBook 1 What’s Changed? The above leads to a few observations: • Scalability — With potentially explosive traffic growth, IT systems need to quickly grow to meet exponential numbers of transactions • High Availability — IT systems must run 24x7, and be resilient to failure. -
The Best Nurturers in Computer Science Research
The Best Nurturers in Computer Science Research Bharath Kumar M. Y. N. Srikant IISc-CSA-TR-2004-10 http://archive.csa.iisc.ernet.in/TR/2004/10/ Computer Science and Automation Indian Institute of Science, India October 2004 The Best Nurturers in Computer Science Research Bharath Kumar M.∗ Y. N. Srikant† Abstract The paper presents a heuristic for mining nurturers in temporally organized collaboration networks: people who facilitate the growth and success of the young ones. Specifically, this heuristic is applied to the computer science bibliographic data to find the best nurturers in computer science research. The measure of success is parameterized, and the paper demonstrates experiments and results with publication count and citations as success metrics. Rather than just the nurturer’s success, the heuristic captures the influence he has had in the indepen- dent success of the relatively young in the network. These results can hence be a useful resource to graduate students and post-doctoral can- didates. The heuristic is extended to accurately yield ranked nurturers inside a particular time period. Interestingly, there is a recognizable deviation between the rankings of the most successful researchers and the best nurturers, which although is obvious from a social perspective has not been statistically demonstrated. Keywords: Social Network Analysis, Bibliometrics, Temporal Data Mining. 1 Introduction Consider a student Arjun, who has finished his under-graduate degree in Computer Science, and is seeking a PhD degree followed by a successful career in Computer Science research. How does he choose his research advisor? He has the following options with him: 1. Look up the rankings of various universities [1], and apply to any “rea- sonably good” professor in any of the top universities. -
SQL Vs Nosql
SQL vs NoSQL By: Mohammed-Ali Khan Email: [email protected] Date: October 21, 2012 YORK UNIVERSITY Agenda ● History of DBMS – why relational is most popular? ● Types of DBMS – brief overview ● Main characteristics of RDBMS ● Main characteristics of NoSQL DBMS ● SQL vs NoSQL ● DB overview – Cassandra ● Criticism of NoSQL ● Some Predictions / Conclusions History of DBMS – why relational is most popular? ● 19th century: US Government needed reports from large datasets to conduct nation-wide census. ● 1890: Herman Hollerith creates the first automatic processing equipment which allowed American census to be conducted. ● 1911: IBM founded. ● 1960s: Efforts to standardize the database technology – US DoD: Conference on Data Systems Language (Codasyl). History of DBMS – why relational is most popular? ● 1968: IBM introduced IMS DBMS (a hierarchical DBMS). ● IBM employee Edgar Codd not satisfied, quoted as saying: – “Taking the old line view, that the burden of finding information should be placed on users...” ● Edgar Codd publishes a paper “A relational Model of Data for large shared Data Banks” – Key points: Independence of data from hardware/storage implementation – High level non-procedural language for data access – Pointers/keys (primary/secondary) History of DBMS – why relational is most popular? ● 1970s: Two database projects launched based on relational model – Ingres (Department of Defence Initiative) – System R (IBM) ● Ingres had its own query language called QUEL whereas System R used SQL. ● Larry Ellison, who worked at IBM, read publications of the System R group and eventually founded ORACLE. History of DBMS – why relational is most popular? ● 1980: IBM introduced SQL for the mainframe market. ● In a nutshell, American Government's requirements had a strong role in development of the relational DBMS. -
Issue Purchase Order for Ingres Relational Database Management System Software Support
BOARD MEETING DATE: December 5, 2014 AGENDA NO. 11 PROPOSAL: Issue Purchase Order for Ingres Relational Database Management System Software Support SYNOPSIS: The Ingres Relational Database Management System is used for the implementation of the Central Information Repository database. This database is used by most enterprise-level software applications at the SCAQMD and currently supports a suite of client/server and web-based applications known collectively as the Clean Air Support System (CLASS). The CLASS applications are used to support all of the SCAQMD core activities. Maintenance support for this software expires November 29, 2014. This action is to issue a three-year purchase order with Actian Corporation for a total amount of $564,967. Funds for this expense are included in the FY 2014-15 Budget and will be included in subsequent fiscal year budget requests. COMMITTEE: Administrative, November 14, 2014, Recommended for Approval RECOMMENDED ACTION: Authorize the Procurement Manager to issue a three-year purchase order with Actian Corporation (formerly Ingres Corporation) for Ingres Relational Database Management System software maintenance support, for the period of November 30, 2014 through November 29, 2017, for a total amount not to exceed $564,967. Barry R. Wallerstein, D.Env. Executive Officer JCM: MH:ZT:agg Background In November 2006, the SCAQMD entered into an annual support and maintenance agreement for Ingres Relational Database Management System (RDBMS) software. The RDBMS software runs on three database servers for production, development, and ad hoc reporting. The production server hosts the Central Information Repository database (DBCIR). This database supports a collection of more than 30 client/server and web-based applications known as the Clean Air Support System (CLASS). -
Devops and Dataops
Getting DataOps Right Andy Palmer, Michael Stonebraker, Nik Bates-Haus, Liam Cleary, and Mark Marinelli Beijing Boston Farnham Sebastopol Tokyo Getting DataOps Right by Andy Palmer, Michael Stonebraker, Nik Bates-Haus, Liam Cleary, and Mark Marinelli Copyright © 2019 O’Reilly Media. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more infor‐ mation, contact our corporate/institutional sales department: 800-998-9938 or [email protected]. Development Editor: Jeff Bleiel Proofreader: Charles Roumeliotis Acquisitions Editor: Rachel Roumeliotis Interior Designer: David Futato Production Editor: Katherine Tozer Cover Designer: Karen Montgomery Copyeditor: Octal Publishing, Inc. Illustrator: Rebecca Demarest June 2019: First Edition Revision History for the First Edition 2019-05-22: First Release The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Getting DataOps Right, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. -
Voltdb Achieves 3M Ops/Second Scaling Linearly on Telecom Benchmark
BENCHMARKS VoltDB Achieves 3M ops/second Scaling Linearly on Telecom Benchmark Executive Overview VoltDB is trusted globally by telecommunications software solution providers as the in-memory database of choice to power their mission-critical applications deployed at over 100 operators around the world. VoltDB was chosen for its performance and features to solve not only the current challenges, but also to support the rapid evolution of systems in the industry. Here is our benchmark that shows how the performance of VoltDB meets or exceeds the requirements of telecom systems. We showcase VoltDB’s high performance, low latency and linear scaling that are necessary to power revolutions in the industry such as 5G. In this benchmark, we tested VoltDB v8.3 in the cloud and observed the performance scale linearly with the number of servers and achieve over 3 million operations/sec- ond with latency consistently in single digits. VoltDB and the 5G Landscape The advent of 5G brings several key challenges to telecom software solution providers. In addition to the new hardware standards, this network evolution necessitates a transformation of the supporting IT functions of OSS and BSS as well. The opportunity to create additional revenues through new service creation generates pressure on the OSS and BSS functions to be agile and scalable to enable new use cases under ever-increasing load. This pressure on the systems, elevates the role of the supporting database from simply storing data to serving as an active real-time decisioning system. Simply put, VoltDB is a Telco-grade database that exactly meets the requirements of an active real-time decision- ing system such as one that might be needed to support 5G. -
Ingres 10.1 for Linux Quick Start Guide
Ingres 10S Quick Start Guide ING-1001-QSL-03 This Documentation is for the end user's informational purposes only and may be subject to change or withdrawal by Actian Corporation ("Actian") at any time. This Documentation is the proprietary information of Actian and is protected by the copyright laws of the United States and international treaties. It is not distributed under a GPL license. You may make printed or electronic copies of this Documentation provided that such copies are for your own internal use and all Actian copyright notices and legends are affixed to each reproduced copy. You may publish or distribute this document, in whole or in part, so long as the document remains unchanged and is disseminated with the applicable Actian software. Any such publication or distribution must be in the same manner and medium as that used by Actian, e.g., electronic download via website with the software or on a CD- ROM. Any other use, such as any dissemination of printed copies or use of this documentation, in whole or in part, in another publication, requires the prior written consent from an authorized representative of Actian. To the extent permitted by applicable law, ACTIAN PROVIDES THIS DOCUMENTATION "AS IS" WITHOUT WARRANTY OF ANY KIND, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT. IN NO EVENT WILL ACTIAN BE LIABLE TO THE END USER OR ANY THIRD PARTY FOR ANY LOSS OR DAMAGE, DIRECT OR INDIRECT, FROM THE USER OF THIS DOCUMENTATION, INCLUDING WITHOUT LIMITATION, LOST PROFITS, BUSINESS INTERRUPTION, GOODWILL, OR LOST DATA, EVEN IF ACTIAN IS EXPRESSLY ADVISED OF SUCH LOSS OR DAMAGE.