Databases Are Not Toasters

Total Page：16

File Type：pdf, Size：1020Kb

Databases are Not Toasters A Framework for Comparing Data Warehouse Appliances Omer Trajman (Vertica) Alain Crolotte (Teradata), David Steinhoff (ParAccel), Raghunath Nambiar (Hewlett-Packard), Meikel Poess (Oracle); Growth of Business Intelligence BI has been the top priority of CIOs for past three years BI platforms are growing at 8%+ for the next 4 years Gartner report "Forecast: Business Intelligence Platforms, Worldwide, 2007-2012.” Fundamentals of Business Intelligence More Data Faster Answers Define: Appliance “a data appliance consists of an integrated set of servers, storage, operating system(s), DBMS and software specifically pre-installed and pre-optimized for data warehousing.” -Wikipedia Fourteen Leading Appliances What are Vendors Promoting? Efficiency and Energy Efficiency Pre-installed Large Volumes/Capacity of Data Pre-configured / Fast Expandable in modular units deployment Single SKU Massively Parallel Single Vendor/Single Support Packaged database + application Specialized hardware Packaged database plus Specialized software hardware Special purpose or purpose built Bundled Solution vs. Appliance Self managing or self tuning Bundled pricing What are two Key Properties? Scalability of data and nodes Simplicity of setup What aren’t vendors promoting? Raw Speed of queries Price/Performance Existing Benchmarks Exist for… Raw Speed of queries Price/Performance What is an Appliance Benchmark? Scalability of data and nodes Simplicity of setup Databases are Not Toasters The authors would like to thank Karl Huppler, Michael Corwin, Kannan Govindarajan for participating in the survey. Omer Trajman – [email protected] Alain Crolotte – [email protected] David Steinhoff – [email protected] Raghunath Othayoth Nambiar – [email protected] Meikel Poess – [email protected].

Copy Link

Recommended publications

Data Warehouse Fundamentals for Storage Professionals – What You Need to Know EMC Proven Professional Knowledge Sharing 2011
Data Warehouse Fundamentals for Storage Professionals – What You Need To Know EMC Proven Professional Knowledge Sharing 2011 Bruce Yellin Advisory Technology Consultant EMC Corporation [email protected] Table of Contents Introduction ................................................................................................................................ 3 Data Warehouse Background .................................................................................................... 4 What Is a Data Warehouse? ................................................................................................... 4 Data Mart Defined .................................................................................................................. 8 Schemas and Data Models ..................................................................................................... 9 Data Warehouse Design – Top Down or Bottom Up? ............................................................10 Extract, Transformation and Loading (ETL) ...........................................................................11 Why You Build a Data Warehouse: Business Intelligence .....................................................13 Technology to the Rescue?.......................................................................................................19 RASP - Reliability, Availability, Scalability and Performance ..................................................20 Data Warehouse Backups .....................................................................................................26
[Show full text]
Magic Quadrant for Data Warehouse Database Management Systems
Magic Quadrant for Data Warehouse Database Management Systems Gartner RAS Core Research Note G00209623, Donald Feinberg, Mark A. Beyer, 28 January 2011, RV5A102012012 The data warehouse DBMS market is undergoing a transformation, including many acquisitions, as vendors adapt data warehouses to support the modern business intelligence and analytic workload requirements of users. This document compares 16 vendors to help you find the right one for your needs. WHAT YOU NEED TO KNOW Despite a troubled economic environment, the data warehouse database management system (DBMS) market returned to growth in 2010, with smaller vendors gaining in acceptance. As predicted in the previous iteration of this Magic Quadrant, 2010 brought major acquisitions, and several of the smaller vendors, such as Aster Data, Ingres and Vertica, took major strides by addressing specific market needs. The year also brought major market growth from data warehouse appliance offerings (see Note 1), with both EMC/Greenplum and Microsoft formally introducing appliances, and IBM, Oracle and Teradata broadening their appliance lines with new offerings. Although we believe that much of the growth was due to replacements of aging or performance-constrained data warehouse environments, we also think that the business value of using data warehouses for new applications such as performance management and advanced analytics has driven — and is driving — growth. All the vendors have stepped up their marketing efforts as the competition has grown. End-user organizations should ignore marketing claims about the applicability and performance capabilities of solutions. Instead, they should base their decisions on customer references and proofs of concept (POCs) to ensure that vendors’ claims will hold up in their environments.
[Show full text]
Government Contracting M&A Update
Government Contracting M&A Update “Market Intelligence for Business Owners” Q3 2013 Capstone Partners Investment Banking Advisors BOSTON | CHICAGO | LONDON | LOS ANGELES | PHILADELPHIA | SAN DIEGO | SILICON VALLEY Government Contracting Coverage Report MERGERS & ACQUISITIONS UPDATE With the nation’s attention focused on reducing government spending and sequestration, one would expect mergers & acquisitions in the government contracting space to come CAPSTONE PARTNERS LLC to a standstill. But such is not the case, with the number of acquisitions announced 200 South Wacker Drive through June totaling more than 250. 31st Floor Chicago, IL 60606 M&A Activity: Government Contractors www.capstonellc.com 1000 964 900 852 800 786 772 786 732 751 700 568 Ted Polk 600 521 Transactions Managing Director 500 of 398 (312) 674‐4531 400 [email protected] 300 256 Number 200 100 Lisa Tolliver 0 Director 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 YTD (312) 674‐4532 2013 [email protected] YTD 2013 through June 30, 2013 Source: Capital IQ, Capstone Partners LLC research While the year’s activity is currently on‐track to come in under the 2012 figure, that trend is reflective of what we are seeing in mergers and acquisitions in general. M&A activity across the board has been down in early 2013 compared to 2012, primarily the result of the market continuing to absorb the rash of transactions that were closed at the end of 2012 in anticipation of rising capital gains tax rates. But, while the number of closed transactions has slowed this year, M&A activity continues to be supported by strong market fundamentals, namely reasonably high transaction valuations; strategic acquirers with strong balance sheets; abundant private equity capital; an accessible and affordable debt market; and a modestly expanding U.S.
[Show full text]
Nexus User Guide (Pdf)
The Best Query Tool Works on all Systems When you possess a tool like Nexus, you have access to every system in your enterprise! The Nexus Query Chameleon is the only tool that works on all systems. Its Super Join Builder allows for the ERwin Logical Model to be loaded, and then Nexus shows tables and views visually. It then guides users to show what joins to what. As users choose the tables and columns they want in their report, Nexus builds the SQL for them with each click of the mouse. Nexus was designed for Teradata and Hadoop, but works on all platforms. Nexus even converts table structures between vendors, so querying and managing multi-vendor platforms is transparent. Even if you only work with one system, you will find that the Nexus is the best query tool you have ever used. If you work with multiple systems, you will be even more amazed. Download a free trial at www.CoffingDW.com. The Tera-Tom Video Series Lessons with Tera-Tom Teradata Architecture and SQL Video Series These exciting videos make learning and certification much easier Four ways to view them: 1. Safari (look up Coffing Studios) 2. CoffingDW.com (sign-up on our website) 3. Your company can buy them all for everyone to see (contact [email protected]) 4. YouTube – Search for CoffingDW or Tera-Tom. The Tera-Tom Genius Series The Tera-Tom Genius Series consists of ten books. Each book is designed for a specific audience, and Teradata is explained to the level best suited for that audience.
[Show full text]
TOP-OF-MIND TIME for IN-MEMORY DATABASES PERFORMANCE 2 In-Memory Databases Help Meet Need for IT Speed
VIRTUALIZATION CLOUD DEVELOPMENT APPLICATION IT HEALTH NETWORKING ARCHITECTURE STORAGE CENTER MANAGEMENT DATA BI/APPLICATIONS RECOVERY/COMPLIANCE DISASTER SECURITY 1 EDITOR’S NOTE Top-of-Mind Time for 2 IN-MEMORY DATABASES HELP In-Memory Databases MEET NEED FOR IT SPEED For all the promise they hold, in-memory databases and the process of implementing them require heavy investments in 3 IBM GIVES DB2 MORE GAS WITH company resources and skills. Is it worth it? IN-MEMORY ACCELERATOR 4 ADD-ON SOFTWARE TAKES ORACLE 12C IN NEW DIRECTION EDITOR’S NOTE 1 In-Memory’s Moment in the Database Sun In-memory databases used to be terri- 2014 report that in-memory databases could Home tory for niche technology vendors and equally provide “transformational performance im- niche applications. But today vendors of all da- provements” in operational and analytical ap- Editor’s Note tabase stripes—SQL, NoSQL, NewSQL—now plications. But in a video posted on YouTube offer in-memory technology, some as stand- the following month, Rosen said the heavily In-Memory Databases Help alone products and others as add-ons to disk- hyped technology also has “the potential to be Meet Need for IT based database management systems. That the next failed silver bullet from IT.” Challenges Speed includes relational database market leaders Or- abound, he cautioned, including data migration acle, IBM and Microsoft as well as business ap- issues and the proliferation of data silos that IBM Gives DB2 plications bigwig SAP with its HANA system. make it hard to do real-time analytics. More Gas With In an interview with SearchDataManage- This guide explores in-memory database In-Memory Accelerator ment’s Jack Vaughan, data management consul- trends and offers advice to help you get started tant William McKnight said that as the price of on deciding whether the technology is right for Add-On Software RAM declines, “memory in a lot of ways is be- your organization.
[Show full text]
Next Generation Data Warehouse Platforms
fourth quarter 2009 TDWI besT pracTIces reporT Next geNeratioN Data Warehouse Platforms By Philip Russom www.tdwi.org Research Sponsors Aster Data Systems HP IBM Infobright Kognitio Microsoft Oracle/Intel Sybase Teradata fourth QuArtEr 2009 TDWI besT pracTIces reporT Next geNeratioN Data Warehouse Platforms By Philip Russom Table of Contents Research Methodology and Demographics . 3 Introduction to Next Generation Data Warehouse Platforms . 4 Definitions of Terms and Concepts. 4 Why Care about Data Warehouse Platforms Now? . 5 The Evolving State of Data Warehouse Platforms . 6 Technology Drivers for New Generations of Data Warehouses . 6 Business Drivers for New Generations of Data Warehouses . 9 Your Data Warehouse Today and Tomorrow. 10 Quantifying Data Warehouse Generations . 13 Growth or Decline of Usage versus Breadth or Narrowness of Commitment . 14 Trends for Next Generation Data Warehouse Platform Options . 16 Next Generation Data Warehouse Platform Options . 17 Real-Time Data Warehousing. 17 Data Management Practices . 19 Cloud Computing and Software-as-a-Service (SaaS). .20 In-Memory Processing and 64-Bit Computing . 21 Open Source Software . .22 Advanced Analytics . .23 Services . 24 Processing Architectures. .25 Data Warehouse Appliances and Similar Platforms . .26 New Database Management Systems as Alternative Options. .28 Recommendations . 31 © 2009 by TDWI (The Data Warehousing InstituteTM), a division of 1105 Media, Inc. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. E-mail requests or feedback to [email protected]. Product and company names mentioned herein may be trademarks and/or registered trademarks of their respective companies. www.tdwi.org 1 NENERATIONE x T G DATA WAREHOUSE Pl ATfORMS About the Author PHILIP RUSSOM is the senior manager of TDWI Research at The Data Warehousing Institute (TDWI), where he oversees many of TDWI’s research-oriented publications, services, and events.
[Show full text]
Big Data Landscape for Databases
Big Data Landscape for Databases Bob Baran Senior Sales Enginee [email protected] ! May 12, 2015 Typical Database Workloads OLTP Applications Real-Time Web, Real-Time, Ad-Hoc Analytics Enterprise Data Mobile, and IoT Operational Warehouses Applications Reporting Typical • MySQL • MongoDB • MySQL • Greenplum • Teradata Databases • Oracle • Cassandra • Oracle • Paraccel • Oracle • MySQL • Netezza • Sybase IQ • Oracle Use Cases • ERP, CRM, Supply • Web, mobile, social • Operational • Exploratory • Enterprise Chain • IoT Datastores Analytics Reporting • Crystal Reports • Data Mining Workload • Real-time updates • Real-time updates • Real-time updates • Complex • Parameterized Strengths • ACID transactions • High ingest rates • Canned, queries reports against • High concurrency • High concurrency of parameterized requiring full historical data of small reads/ small reads/ writes reports table scans writes • Range queries • Range queries • Append only • Range queries Operational Analytical 2 Recent History of RDBMSs ▪ RDBMS Definition ▪ Relational with joins ▪ ACID transactions ▪ Secondary indexes ▪ Typically row-oriented ▪ Operational and/or analytical workloads ▪ By early 2000s ▪ Limited innovation ▪ Looked like Oracle and Teradata won… 3 Hadoop Shakes Up Batch Analytics ▪ Data processing framework ▪ Cheap distributed file system ▪ Brute force, batch processing through MapReduce ▪ Great for batch analytics ▪ Great place to dump data to look at later 4 NoSQL Shakes Ups Operational DBs ▪ NoSQL wave ▪ Companies like Google, Amazon and
[Show full text]
A Technical Overview the Paraccel Analytic Database
THE PARACCEL ANALYTIC DATABASE A TECHNICAL OVERVIEW The ParAccel Analytic Database: A Technical Overview The ParAccel Analytic Database: A Technical Overview Version 2.5 February 10, 2010. www.paraccel.com © 2010 ParAccel, Inc. All Rights Reserved. ParAccel product names are trademarks of ParAccel, Inc. Other product names are trademarks of their respective owners. © 2010 ParAccel, Inc. All rights reserved. i The ParAccel Analytic Database: A Technical Overview TABLE OF CONTENTS Introduction ............................................................... 1 System Architecture ................................................... 2 LEADER NODE.......................................................................................2 COMPUTE NODES .................................................................................3 COMMUNICATION FABRIC ....................................................................4 OPTIONAL STORAGE AREA NETWORK (SAN) .....................................4 PADB Features ............................................................ 4 PERFORMANCE.....................................................................................4 Columnar Orientation ................................................................................... 5 Column vs. Row Example ........................................................................... 6 How Does Columnar Orientation Impact Design Considerations?.............. 6 Extensible Analytics ....................................................................................
[Show full text]
The “Tech”Tonic Shift Dale Wickizer Chief Technology Officer U.S
The “Tech”tonic Shift Dale Wickizer Chief Technology Officer U.S. Public Sector NetApp Confidential — Limited Use Today I want to talk to you about the “Tech”tonic shift occurring to traditional enterprise applications as well as the IT organizations that manage them. © 2011 NetApp. All rights reserved. 1 World Data Explosion Growth Over the Next Decade: Servers (Phys/VM): 10x Data/Information: 50x #Files: 75x IT Professionals: <1.5x Source: Revisited: The Rapid Growth in Unstructured Data « Wikibon Blog http://bit.ly/oRSdXm • Growing 9x in 5 yrs! (1.8 ZB in 2011) • > 90% unstructured data Source: Gantz, John and Reinsel, David, “Extracting Value from Chaos”, • End user and machine generated IDC IVIEW, June 2011, page 4. 2 That shift is being driven by an explosion of data being generated and consumed in the world. Data has grown by a factor of 9 over the past 5 years, crossing 1.2 ZB for the first time! (If anyone wonders what 1.2 ZB is, Wikibon has this great graphic, showing it is the equivalent of 75 billion fully loaded iPads, stacked end-to-end and side-by-side, covering Wembley stadium, in a column more than 4 miles high). This year it will grow to 1.8 ZB. More than 90% of this data was unstructured and much of machine generated, in response to data stored by end users. Over the next decade, this data growth is expected to accelerate, increasing by a factor of 50. Over the same time, the number of files is expected to increase by more than a factor of 75, which will break most traditional file systems.
[Show full text]
Systems for Cloud Data Analytics
Peter Boncz SYSTEMS FOR CLOUD DATA ANALYTICS www.cwi.nl/~boncz/badsCloud Data Systems Credits • David DeWitt & Willis Lang (Microsoft) – cloud DW material • Stratis Viglas (Google) – extreme computing course (University Edinburgh) • Marcin Zukowski (Snowflake) • Ippokratis Pandis (Amazon Redshift/Spectrum) • Spark Team – Matei Zaharia, Xiangrui Meng (Stanford), – Ion Stoica, Xifan Pu (UC Berkeley) – Reynold Xin, Alex Behm (Databricks) www.cwi.nl/~boncz/badsCloud Data Systems Is it safe to have enterprise data in the Cloud? 2005: No way! Are you crazy? 2012: Don’t think so... But wait, we store our email where? 2018: Of course! www.cwi.nl/~boncz/badsCloud Data Systems Getting a database in a cloud Hi! I'm a Data Scientist! Hello! I am your account manager at X! I'm looking for a database for our cloud system Sure thing! Let's install our product, DBMS X for you! Awesome! It seems to work! Great. Let me send you that invoice! Just a sec… How much does the storage cost ? Hold on, let me check that Wait, what? And the system is elastic, right? Mommy!!! And I only pay for what I use, right? www.cwi.nl/~boncz/badsCloud Data Systems Traditional DB systems and the cloud • Designed for: –Small, fixed, optimized clusters of machines –Constrained amount of data and resources • Can be delivered via the Cloud –Reduce the complexity of hardware setup, software installation –No elasticity –No cheap storage –Not designed for cloud's poor stability –Not easy to use –Not "always on" –... www.cwi.nl/~boncz/badsCloud Data Systems Data in the Cloud • Data
[Show full text]
TPC Benchmark H Full Disclosure Report Vmware® ESX
TPC Benchmark H Full Disclosure Report VMware® ESX™ Using ParAccel Analytic Database™ Submitted for Review Report Date: April 11, 2010 TPC Benchmark H Full Disclosure Report Pricing revision: August 24, 2010 TPC Benchmark H Full Disclosure Report Page 1 First Edition – April 2010 Copyright © 2010 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents. VMware, ESX, and ESXi are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. TPC-H Benchmark™ is a trademark of the Transaction Processing Performance Council. ParAccel Analytic Database™ is a registered trademark of ParAccel, Inc. VMware, Inc., the Sponsor of this benchmark test, believes that the information in this document is accurate as of the publication date. The information in this document is subject to change without notice. The Sponsor assumes no responsibility for any errors that may appear in this document. The pricing information in this document is believed to accurately reflect the current prices as of the publication date. However, the Sponsor provides no warranty of the pricing information included in this document. Benchmark results are highly dependent upon workload, specific application requirements, and system design and implementation. Relative system performance will vary as a result of these and other factors. Therefore, the TPC Benchmark H should not be used as a substitute for a specific customer application benchmark when critical capacity planning and/or product evaluation decisions are contemplated.
[Show full text]
Big Data: Challenges, Opportunities and Realities
Big Data: Challenges, Opportunities and Realities (This is the pre-print version submitted for publication as a chapter in an edited volume “Effective Big Data Management and Opportunities for Implementation”) Recommended Citation: Bhadani, A., Jothimani, D. (2016), Big data: Challenges, opportunities and realities, In Singh, M.K., & Kumar, D.G. (Eds.), Effective Big Data Management and Opportunities for Implementation (pp. 1-24), Pennsylvania, USA, IGI Global Big Data: Challenges, Opportunities, and Realities Abhay Kumar Bhadani Indian Institute of Technology Delhi, India Dhanya Jothimani Indian Institute of Technology Delhi, India ABSTRACT With the advent of Internet of Things (IoT) and Web 2.0 technologies, there has been a tremendous growth in the amount of data generated. This chapter emphasizes on the need for big data, technological advancements, tools and techniques being used to process big data are discussed. Technological improvements and limitations of existing storage techniques are also presented. Since, the traditional technologies like Relational Database Management System (RDBMS) have their own limitations to handle big data, new technologies have been developed to handle them and to derive useful insights. This chapter presents an overview of big data analytics, its application, advantages, and limitations. Few research issues and future directions are presented in this chapter. Keywords: Big Data, Big Data Analytics, Cloud Computing, Data Value Chain, Grid Computing, Hadoop, High Dimensional Data, MapReduce INTRODUCTION With the digitization of most of the processes, emergence of different social network platforms, blogs, deployment of different kind of sensors, adoption of hand-held digital devices, wearable devices and explosion in the usage of Internet, huge amount of data are being generated on continuous basis.
[Show full text]