DOCSLIB.ORG

Databases on AWS the Right Tool for the Right Job

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Databases on AWS the Right Tool for the Right Job Databases on AWS The Right Tool for the Right Job David Gendel, Sr. Solutions Architect, AWS Wednesday, February 17, 2021 © 2021, Amazon Web Services, Inc. or its Affiliates. Traditional Database Architecture Client Tier one database App/Web Tier for all workloads RDBMS © 2021, Amazon Web Services, Inc. or its Affiliates. Traditional Database Architecture • Key-value access Client Tier • Complex queries • OLAP transactions App/Web Tier • Analytics RDBMS All forced into the relational database © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Data Tier Architecture Client Tier On AWS choose best database service for each App/Web Tier workload Data Tier Cache Data Warehouse Time Series Blob Store NoSQL RDBMS Quantum Ledger Search © 2021, Amazon Web Services, Inc. or its Affiliates. Workload Driven Data Store Selection hot reads analytics logging NoSQL complex queries Periodic rich search simple query & transactions data Graph / Key Value / Document Untampered data Data Tier Cache Data Warehouse Time Series Blob Store NoSQL RDBMS Quantum Ledger Search © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Database Services hot reads analytics logging NoSQL complex queries Periodic rich search simple query & transactions data Graph / Key Value / Document Untampered data Data Tier Amazon Amazon Amazon Amazon S3 ElastiCache Redshift Timestream Amazon DynamoDB Amazon Amazon Amazon Neptune / DocumentDB RDS QLDB ElasticSearch © 2021, Amazon Web Services, Inc. or its Affiliates. Easy to Administer Highly Scalable Amazon RDS Available and Durable Secure © 2021, Amazon Web Services, Inc. or its Affiliates. Amazon RDS Managed relational database service with a choice of popular database engines Easy to administer Performant & scalable Available & durable Secure and compliant Easily deploy and maintain Scale compute Automatic Multi-AZ data Data encryption at rest and in hardware, OS and DB and storage with a few clicks; replication; automated transit; industry compliance software; built-in monitoring minimal downtime for your backup, snapshots, and and assurance programs application failover © 2021, Amazon Web Services, Inc. or its Affiliates. Amazon Aurora MySQL and PostgreSQL compatible relational database built for the cloud Performance and availability of commercial-grade databases at 1/10th the cost Performance Availability Highly Fully & scalability & durability secure managed 5x throughput of standard Fault-tolerant, self-healing Network isolation, Managed by RDS: no MySQL and 3x of standard storage; six copies of encryption at hardware provisioning, PostgreSQL; scale-out up data across three AZs; rest/transit software patching, setup, to15 read replicas continuous backup to S3 configuration, or backups © 2021, Amazon Web Services, Inc. or its Affiliates. Aurora Multi-Master First relational database service with scale-out reads and writes across multiple data centers Scale out both reads and writes Zero application downtime from ANY instance failure Zero application downtime from ANY AZ failure Read/Write Read/Write Read/Write Master 1 Master 2 Master 3 Faster write performance and higher scale Shared distributed storage volume Availability Availability Availability Zone 1 Zone 2 Zone 3 © 2021, Amazon Web Services, Inc. or its Affiliates. Aurora Serverless On-demand, auto-scaling database for applications with variable workloads Application Starts up on demand, shuts down when not in use Database Endpoint Automatically scales with no instances to manage Scalable Database Capacity (Compute + Memory) Warm Capacity Pool Pay per second for the database capacity you use Shared Distributed Storage © 2021, Amazon Web Services, Inc. or its Affiliates. for as low as $934/TB per year Petabyte scale Massively parallel Amazon Columnar Store Redshift Relational data warehouse Fully managed = no admin © 2021, Amazon Web Services, Inc. or its Affiliates. Amazon Redshift – Data Warehousing Fast, powerful, and simple data warehousing at 1/10 the cost Massively parallel, petabyte scale Fast Inexpensive Scalable Secure $ Columnar storage As low as $1000 per Resize your cluster up Data encrypted at rest technology to improve I/O terabyte per year, and down as your and transit. Isolate efficiency and parallelize 1/10th the cost of performance and clusters with VPC. queries. Data load scales traditional data capacity needs change Manage your own keys linearly. warehouse solutions with KMS © 2021, Amazon Web Services, Inc. or its Affiliates. Redshift Spectrum Run SQL queries directly against data in S3 using thousands of nodes High concurrency: Multiple No ETL: Query data in-place Full Amazon Redshift SQL support clusters access same data using open file formats S3 SQL Fast at exabyte scale Elastic and highly available On-demand, pay-per-query © 2021, Amazon Web Services, Inc. or its Affiliates. Redshift Spectrum © 2021, Amazon Web Services, Inc. or its Affiliates. NoSQL database Seamless scalability Zero admin Amazon DynamoDB Single-digit millisecond latency Multi-Master Multi-Region © 2021, Amazon Web Services, Inc. or its Affiliates. Amazon DynamoDB Highly available Fully managed Consistently fast at any scale and durable Integrates with AWS Lambda, Secure Cost-effective Amazon Redshift, and more © 2021, Amazon Web Services, Inc. or its Affiliates. Highly available and durable Designed to support Built for high durability 99.99% of availability WRITES READS 3-way replication Strongly or eventually consistent Persisted to disk No latency trade-off (Custom SSD) Data is always replicated to three Availability Zones © 2021, Amazon Web Services, Inc. or its Affiliates. Fully managed time series database 1,000x faster at 1/10th the cost Amazon Built-in analytics Timestream Serverless © 2021, Amazon Web Services, Inc. or its Affiliates. Amazon Timestream 1,000x faster at 1/10th Analytics optimized Trillions of daily events Serverless the cost of relational for time series data databases Collect fast moving time- Capable of processing trillions Built-in analytics for No servers to manage; time- series data from multiple of events daily; the adaptive interpolation, smoothing, consuming tasks such as hardware sources at the rate of query processing engine and approximation to provisioning, software patching, millions of inserts per maintains steady, predictable identify trends, patterns, setup, & configuration done for you second performance and anomalies © 2021, Amazon Web Services, Inc. or its Affiliates. Select the best tool for the job Amazon Aurora Amazon DocumentDB Amazon DynamoDB Amazon ElastiCache (with MongoDB compatibility) Amazon Redshift AWS Database Amazon Neptune Amazon Quantum Migration Service Ledger Database (QLDB) Amazon RDS Amazon Timestream DB on EC2 instance © 2021, Amazon Web Services, Inc. or its Affiliates. .
Load more

Recommended publications
  • (AWS) Security Workshop - Pre-Read Material
    Amazon Web Services (AWS) Security Workshop - Pre-read material It is highly recommended to go through the pre-read before attending the AWS Security workshop. Shared Responsibility Model Security and Compliance is a shared responsibility between AWS and the customer. This shared model can help relieve the customer’s operational burden as AWS operates, manages and controls the components from the host operating system and virtualization layer down to the physical security of the facilities in which the service operates. The customer assumes responsibility and management of the guest operating system (including updates and security patches), other associated application software as well as the configuration of the AWS provided security group firewall. Customers should carefully consider the services they choose as their responsibilities vary depending on the services used, the integration of those services into their IT environment, and applicable laws and regulations. The nature of this shared responsibility also provides the flexibility and customer control that permits the deployment. As shown in the chart below, this differentiation of responsibility is commonly referred to as Security “of” the Cloud versus Security “in” the Cloud. AWS responsibility “Security of the Cloud” - AWS is responsible for protecting the infrastructure that runs all of the services offered in the AWS Cloud. This infrastructure is composed of the hardware, software, networking, and facilities that run AWS Cloud services. Customer responsibility “Security in the Cloud” – Customer responsibility will be determined by the AWS Cloud services that a customer selects. This determines the amount of configuration work the customer must perform as part of their security responsibilities.
    [Show full text]
  • Amazon Dynamodb
    Dynamo Amazon DynamoDB Nicolas Travers Inspiré de Advait Deo Vertigo N. Travers ESILV : Dynamo Amazon DynamoDB – What is it ? • Fully managed nosql database service on AWS • Data model in the form of tables • Data stored in the form of items (name – value attributes) • Automatic scaling ▫ Provisioned throughput ▫ Storage scaling ▫ Distributed architecture • Easy Administration • Monitoring of tables using CloudWatch • Integration with EMR (Elastic MapReduce) ▫ Analyze data and store in S3 Vertigo N. Travers ESILV : Dynamo Amazon DynamoDB – What is it ? key=value key=value key=value key=value Table Item (64KB max) Attributes • Primary key (mandatory for every table) ▫ Hash or Hash + Range • Data model in the form of tables • Data stored in the form of items (name – value attributes) • Secondary Indexes for improved performance ▫ Local secondary index ▫ Global secondary index • Scalar data type (number, string etc) or multi-valued data type (sets) Vertigo N. Travers ESILV : Dynamo DynamoDB Architecture • True distributed architecture • Data is spread across hundreds of servers called storage nodes • Hundreds of servers form a cluster in the form of a “ring” • Client application can connect using one of the two approaches ▫ Routing using a load balancer ▫ Client-library that reflects Dynamo’s partitioning scheme and can determine the storage host to connect • Advantage of load balancer – no need for dynamo specific code in client application • Advantage of client-library – saves 1 network hop to load balancer • Synchronous replication is not achievable for high availability and scalability requirement at amazon • DynamoDB is designed to be “always writable” storage solution • Allows multiple versions of data on multiple storage nodes • Conflict resolution happens while reads and NOT during writes ▫ Syntactic conflict resolution ▫ Symantec conflict resolution Vertigo N.
    [Show full text]
  • AWS Managed Services (AMS)
    AWS Managed Services (AMS) Application Developer's Guide AMS Advanced Operations Plan Version September 16, 2020 AWS Managed Services (AMS) Application Developer's Guide AMS Advanced Operations Plan AWS Managed Services (AMS) Application Developer's Guide: AMS Advanced Operations Plan Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. AWS Managed Services (AMS) Application Developer's Guide AMS Advanced Operations Plan Table of Contents Application Onboarding to AMS Introduction ........................................................................................ 1 What is Application Onboarding? ................................................................................................. 1 What we do, what we do not do .................................................................................................. 1 AMS Amazon Machine Images (AMIs) ............................................................................................ 2 Security enhanced AMIs ...................................................................................................... 4 Key terms .................................................................................................................................
    [Show full text]
  • [XIAY]⋙ Big Data Science & Analytics: a Hands-On Approach By
    Big Data Science & Analytics: A Hands-On Approach Arshdeep Bahga, Vijay Madisetti Click here if your download doesn"t start automatically Big Data Science & Analytics: A Hands-On Approach Arshdeep Bahga, Vijay Madisetti Big Data Science & Analytics: A Hands-On Approach Arshdeep Bahga, Vijay Madisetti We are living in the dawn of what has been termed as the "Fourth Industrial Revolution", which is marked through the emergence of "cyber-physical systems" where software interfaces seamlessly over networks with physical systems, such as sensors, smartphones, vehicles, power grids or buildings, to create a new world of Internet of Things (IoT). Data and information are fuel of this new age where powerful analytics algorithms burn this fuel to generate decisions that are expected to create a smarter and more efficient world for all of us to live in. This new area of technology has been defined as Big Data Science and Analytics, and the industrial and academic communities are realizing this as a competitive technology that can generate significant new wealth and opportunity. Big data is defined as collections of datasets whose volume, velocity or variety is so large that it is difficult to store, manage, process and analyze the data using traditional databases and data processing tools. Big data science and analytics deals with collection, storage, processing and analysis of massive-scale data. Industry surveys, by Gartner and e-Skills, for instance, predict that there will be over 2 million job openings for engineers and scientists trained in the area of data science and analytics alone, and that the job market is in this area is growing at a 150 percent year-over-year growth rate.
    [Show full text]
  • Data Warehousing on AWS
    Data Warehousing on AWS March 2016 Amazon Web Services – Data Warehousing on AWS March 2016 © 2016, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document is provided for informational purposes only. It represents AWS’s current product offerings and practices as of the date of issue of this document, which are subject to change without notice. Customers are responsible for making their own independent assessment of the information in this document and any use of AWS’s products or services, each of which is provided “as is” without warranty of any kind, whether express or implied. This document does not create any warranties, representations, contractual commitments, conditions or assurances from AWS, its affiliates, suppliers or licensors. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers. Page 2 of 26 Amazon Web Services – Data Warehousing on AWS March 2016 Contents Abstract 4 Introduction 4 Modern Analytics and Data Warehousing Architecture 6 Analytics Architecture 6 Data Warehouse Technology Options 12 Row-Oriented Databases 12 Column-Oriented Databases 13 Massively Parallel Processing Architectures 15 Amazon Redshift Deep Dive 15 Performance 15 Durability and Availability 16 Scalability and Elasticity 16 Interfaces 17 Security 17 Cost Model 18 Ideal Usage Patterns 18 Anti-Patterns 19 Migrating to Amazon Redshift 20 One-Step Migration 20 Two-Step Migration 20 Tools for Database Migration 21 Designing Data Warehousing Workflows 21 Conclusion 24 Further Reading 25 Page 3 of 26 Amazon Web Services – Data Warehousing on AWS March 2016 Abstract Data engineers, data analysts, and developers in enterprises across the globe are looking to migrate data warehousing to the cloud to increase performance and lower costs.
    [Show full text]
  • Performance at Scale with Amazon Elasticache
    Performance at Scale with Amazon ElastiCache July 2019 Notices Customers are responsible for making their own independent assessment of the information in this document. This document: (a) is for informational purposes only, (b) represents current AWS product offerings and practices, which are subject to change without notice, and (c) does not create any commitments or assurances from AWS and its affiliates, suppliers or licensors. AWS products or services are provided “as is” without warranties, representations, or conditions of any kind, whether express or implied. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers. © 2019 Amazon Web Services, Inc. or its affiliates. All rights reserved. Contents Introduction .......................................................................................................................... 1 ElastiCache Overview ......................................................................................................... 2 Alternatives to ElastiCache ................................................................................................. 2 Memcached vs. Redis ......................................................................................................... 3 ElastiCache for Memcached ............................................................................................... 5 Architecture with ElastiCache for Memcached ...............................................................
    [Show full text]
  • Introducing Amazon RDS for Aurora Chris Littlefield, AWS Certified Solutions Architect – Associate Level, AWS Certified Sysops Administrator – Associate Level
    Expert Reference Series of White Papers Introducing Amazon RDS for Aurora 1-800-COURSES www.globalknowledge.com Introducing Amazon RDS for Aurora Chris Littlefield, AWS Certified Solutions Architect – Associate Level, AWS Certified SysOps Administrator – Associate Level Introduction The following document provides an overview of one of the latest offerings from cloud leader Amazon Web Services (AWS). It’s a product that will enable powerful, massively scalable relational databases in Amazon’s cloud environment. The product was designed, built, and tested in secret over the past three years, and now it is nearly ready for production workloads. Some of its detailed design remains confidential, so we’ll present the released features and functionality, as well as pricing for this exciting new product. Background In November 2014, at Amazon Web Services’ annual conference re:Invent, Andy Jassy introduced a series of revolutionary new products. Among those new products is the product described in this paper, which is based on AWS’s hugely successful Relational Database Service (RDS). So what’s this new product, you ask? It’s Amazon Aurora for RDS. Aurora is MySQL compatible database solution that will enable highly scalable databases running across multiple AWS availability zones, at a very low price point. It’s designed to deliver the performance and availability of commercial grade databases at the simplicity and price point of open source databases. What is Amazon Aurora? Aurora is a MySQL compatible database engine that far exceeds the current size limitations of relational databases running on RDS. It spans three AWS availability zones within a region. The great news is that if you already use or are familiar with RDS, you’re already going to be comfortable configuring Aurora.
    [Show full text]
  • Rose Gardner Mysteries
    JABberwocky Literary Agency, Inc. Est. 1994 RIGHTS CATALOG 2019 JABberwocky Literary Agency, Inc. 49 W. 45th St., 12th Floor, New York, NY 10036-4603 Phone: +1-917-388-3010 Fax: +1-917-388-2998 Joshua Bilmes, President [email protected] Adriana Funke Karen Bourne International Rights Director Foreign Rights Assistant [email protected] [email protected] Follow us on Twitter: @awfulagent @jabberworld For the latest news, reviews, and updated rights information, visit us at: www.awfulagent.com The information in this catalog is accurate as of [DATE]. Clients, titles, and availability should be confirmed. Table of Contents Table of Contents Author/Section Genre Page # Author/Section Genre Page # Tim Akers ....................... Fantasy..........................................................................22 Ellery Queen ................... Mystery.........................................................................64 Robert Asprin ................. Fantasy..........................................................................68 Brandon Sanderson ........ New York Times Bestseller.......................................51-60 Marie Brennan ............... Fantasy..........................................................................8-9 Jon Sprunk ..................... Fantasy..........................................................................36 Peter V. Brett .................. Fantasy.....................................................................16-17 Michael J. Sullivan ......... Fantasy.....................................................................26-27
    [Show full text]
  • Amazon Documentdb Deep Dive
    DAT326 Amazon DocumentDB deep dive Joseph Idziorek Antra Grover Principal Product Manager Software Development Engineer Amazon Web Services Fulfillment By Amazon © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda What is the purpose of a document database? What customer problems does Amazon DocumentDB (with MongoDB compatibility) solve and how? Customer use case and learnings: Fulfillment by Amazon What did we deliver for customers this year? What’s next? © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Purpose-built databases Relational Key value Document In-memory Graph Search Time series Ledger Why document databases? Denormalized data Normalized data model model { 'name': 'Bat City Gelato', 'price': '$', 'rating': 5.0, 'review_count': 46, 'categories': ['gelato', 'ice cream'], 'location': { 'address': '6301 W Parmer Ln', 'city': 'Austin', 'country': 'US', 'state': 'TX', 'zip_code': '78729'} } Why document databases? GET https://api.yelp.com/v3/businesses/{id} { 'name': 'Bat City Gelato', 'price': '$', 'rating': 5.0, 'review_count': 46, 'categories': ['gelato', 'ice cream'], 'location': { 'address': '6301 W Parmer Ln', 'city': 'Austin', 'country': 'US', 'state': 'TX', 'zip_code': '78729'} } Why document databases? response = yelp_api.search_query(term='ice cream', location='austin, tx', sort_by='rating', limit=5) Why document databases? for i in response['businesses']: col.insert_one(i) db.businesses.aggregate([ { $group: { _id: "$price", ratingAvg: { $avg: "$rating"}} } ]) db.businesses.find({
    [Show full text]
  • A Motion Is Requested to Authorize the Execution of a Contract for Amazon Business Procurement Services Through the U.S. Communities Government Purchasing Alliance
    MOT 2019-8118 Page 1 of 98 VILLAGE OF DOWNERS GROVE Report for the Village Council Meeting 3/19/2019 SUBJECT: SUBMITTED BY: Authorization of a contract for Amazon Business procurement Judy Buttny services Finance Director SYNOPSIS A motion is requested to authorize the execution of a contract for Amazon Business procurement services through the U.S. Communities Government Purchasing Alliance. STRATEGIC PLAN ALIGNMENT The goals for 2017-2019 includes Steward of Financial Sustainability, and Exceptional, Continual Innovation. FISCAL IMPACT There is no cost to utilize Amazon Business procurement services through the U.S. Communities Government Purchasing Alliance. RECOMMENDATION Approval on the March 19, 2019 Consent Agenda. BACKGROUND U.S. Communities Government Purchasing Alliance is the largest public sector cooperative purchasing organization in the nation. All contracts are awarded by a governmental entity utilizing industry best practices, processes and procedures. The Village of Downers Grove has been a member of the U.S. Communities Government Purchasing Alliance since 2008. Through cooperative purchasing, the Village is able to take advantage of economy of scale and reduce the cost of goods and services. U.S. Communities has partnered with Amazon Services to offer local government agencies the ability to utilize Amazon Business for procurement services at no cost to U.S. Communities members. Amazon Business offers business-only prices on millions of products in a competitive digital market place and a multi-level approval workflow. Staff can efficiently find quotes and purchase products for the best possible price, and the multi-level approval workflow ensures this service is compliant with the Village’s competitive process for purchases under $7,000.
    [Show full text]
  • Database Software Market: Billy Fitzsimmons +1 312 364 5112
    Equity Research Technology, Media, & Communications | Enterprise and Cloud Infrastructure March 22, 2019 Industry Report Jason Ader +1 617 235 7519 [email protected] Database Software Market: Billy Fitzsimmons +1 312 364 5112 The Long-Awaited Shake-up [email protected] Naji +1 212 245 6508 [email protected] Please refer to important disclosures on pages 70 and 71. Analyst certification is on page 70. William Blair or an affiliate does and seeks to do business with companies covered in its research reports. As a result, investors should be aware that the firm may have a conflict of interest that could affect the objectivity of this report. This report is not intended to provide personal investment advice. The opinions and recommendations here- in do not take into account individual client circumstances, objectives, or needs and are not intended as recommen- dations of particular securities, financial instruments, or strategies to particular clients. The recipient of this report must make its own independent decisions regarding any securities or financial instruments mentioned herein. William Blair Contents Key Findings ......................................................................................................................3 Introduction .......................................................................................................................5 Database Market History ...................................................................................................7 Market Definitions
    [Show full text]
  • AWS Certified Developer – Associate (DVA-C01) Sample Exam Questions
    AWS Certified Developer – Associate (DVA-C01) Sample Exam Questions 1) A company is migrating a legacy application to Amazon EC2. The application uses a user name and password stored in the source code to connect to a MySQL database. The database will be migrated to an Amazon RDS for MySQL DB instance. As part of the migration, the company wants to implement a secure way to store and automatically rotate the database credentials. Which approach meets these requirements? A) Store the database credentials in environment variables in an Amazon Machine Image (AMI). Rotate the credentials by replacing the AMI. B) Store the database credentials in AWS Systems Manager Parameter Store. Configure Parameter Store to automatically rotate the credentials. C) Store the database credentials in environment variables on the EC2 instances. Rotate the credentials by relaunching the EC2 instances. D) Store the database credentials in AWS Secrets Manager. Configure Secrets Manager to automatically rotate the credentials. 2) A Developer is designing a web application that allows the users to post comments and receive near- real-time feedback. Which architectures meet these requirements? (Select TWO.) A) Create an AWS AppSync schema and corresponding APIs. Use an Amazon DynamoDB table as the data store. B) Create a WebSocket API in Amazon API Gateway. Use an AWS Lambda function as the backend and an Amazon DynamoDB table as the data store. C) Create an AWS Elastic Beanstalk application backed by an Amazon RDS database. Configure the application to allow long-lived TCP/IP sockets. D) Create a GraphQL endpoint in Amazon API Gateway. Use an Amazon DynamoDB table as the data store.
    [Show full text]