Amazon Documentdb (With Mongodb Compatibility) Deep Dive

Amazon DocumentDB (with MongoDB compatibility) Deep Dive

Jeff Duffy DocumentDB Specialist SA

Relational Key-value Document In-memory Graph Time-series Ledger Wide Column

Amazon Relational Amazon Amazon Amazon Amazon Database Service Amazon Amazon Amazon DynamoDB DocumentDB ElastiCache Neptune (RDS) Timestream Quantum Keyspaces Ledger Aurora Community Commercial Redis Memcached Database

MongoDB Fast Scalable Fully managed compatible

Millions of requests per second Separation of compute and Managed by AWS: Compatible with MongoDB 3.6; with millisecond latency storage scales both no hardware provisioning; use the same SDKs, tools, and independently; scale out to auto patching, quick setup, applications with Amazon 15 read replicas in minutes secure, and automatic DocumentDB backups

Purpose-built document database engineered for the cloud

Learn more: https://aws.amazon.com/documentdb/customers/

JSON RelationalJSON (Client)

(App) (Database)

JSON became Friction when Object-relational Document the de facto converting JSON mappers (ORMs) databases solved data interchange to the relational were created to the problem format model minimize friction

Documents map naturally to how humans model data Denormalized data Normalized data model model Documents (objects/JSON) are common application data models

Document databases store JSON-like documents

Document databases provide flexible schema and indexing

Ad hoc querying and aggregations

Content Mobile Personalization management

Catalog Retail and User profiles marketing

{ userid: 181276, { username: "sue1942", userid: 181276, name: {first: "Susan", username: "sue1942", last: "Benoit"}, name: {first: "Susan", ExplodingSnails: { last: "Benoit"} hi_score: 3185400, } global_rank: 5139, bonus_levels: true },} } promotions: ["new user","5%","snail lover"] }

Application

Single monolithic architectures API

Query processor Not designed for Caching the cloud

Logging

Storage Scale monolithically Fail monolithically

Replication

Disk Disk Disk Disk Node 1 Node 2 Node 3 Node 4

Replication

Disk Disk Disk Disk Node 1 Node 2 Node 3 Node 3’

Node Node Node

Storage Storage Storage Volume Volume Volume

Operational Backup Node Node

Snapshot Snapshot

Replication Replication Node Node Node

Storage Volume Storage Volume Storage Volume

Caching

Logging

Decouple computeStorage and storage

Scale storage Storage layer

Instance Instance Instance (Replica) (Primary) (Replica) Compute Writes Reads Reads Reads

Distributed storage volume

Storage

AZ1 AZ2 AZ3

Instance Instance Instance Instance (Replica) (Primary) (Replica) (Replica) Compute Writes

~8-10 mins Reads Reads Reads Reads

Distributed storage volume

Storage

AZ1 AZ2 AZ3

~30 secs

Instance Instance InstanceInstance Instance (Replica) (Primary) (Replica)(Primary) (Replica) Writes Writes ~8-10 mins Reads Reads Reads Reads Reads Distributed storage volume

Instance Instance Instance Compute (Replica) (Primary) (Replica) Writes Reads Reads Reads Distributed storage volume

Storage

Compute Instance Instance Instance (Replica) (Primary) (Replica) Writes Reads Reads Reads Distributed storage volume

Storage

Continuous Backups Snapshots (PITR) Amazon S3 (Automated and Manual)

Instance Instance Instance Instances: 321 (Replica) (Primary) (Replica) Environment: productiondev/test

Availability goal: 99.99%99.9%99% Writes Durability: highly durable Reads Reads Reads

Distributed storage volume

AZ1 AZ2 AZ3

Automatic failover Automated AWS Integration Pay-as-you-go and recovery maintenance pricing

Failed instances are Automated upgrades and CloudWatch, CloudTrail, Per-second instance automatically replaced patching CloudFormation, VPC, billing, no long-term Secrets Manager commitment

“Our engineering teams now spend less time on operations like backup scripts, scale testing, and managing high availability and instead are able to focus on developing new capabilities for our customers.”

MongoDB 3.6 Same drivers, Replica sets tools

Compatible with MongoDB Use the same MongoDB Read scaling is easy with Community Edition 3.6 drivers and tools with replica set emulation Amazon DocumentDB

“Getting started with DocumentDB was also simple and we migrated our application in a couple of days without needing to make any meaningful code changes. Everything just worked.”

• Over 60 million downloads • Easy for developers to get started • MongoDB API is powerful • Managing MongoDB is hard

Source: http://db-engines.com

Scale out Scale up Autoscale Load in minutes in minutes storage balancing

Scale to 15 read Scale from Storage Scale reads across replicas 16 to 768 GiB automatically replicas of RAM grows from 10 GB to 64 TB

FEB 2019 JAN 2019 MAR 2019 APR 2019 MAY 2019 JUN 2019 JUL 2019 AUG 2019 OCT 2019

Launch DDL Frankfurt Aggregation Per-second Sydney Start/Stop Slow query Change Streams logger Auditing Secrets Operators Billing Cluster Mumbai Manager Tokyo Deletion Aggregation Paris Protection Operators Seoul Singapore London

DEC 2019 FEB 2020 MAR 2020 APR 2020 … with more to come in 2020! Aggregation Canada RBAC Glue ETL Operators

Sharding motivator DocumentDB approach

Scale Reads Add up to 15 read replicas

Scale Writes Scale vertically with very low impact

Scale Storage Storage scales automatically to 64TB

VPC Only Encrypted by Safe defaults Compliance default

Strict network isolation with Encryption at rest with Best practices are the PCI DSS, Amazon Virtual Private Cloud AWS KMS defaults ISO 9001, 27001, 27017, (VPC) Encryption in transit and 27018, with TLS SOC 1, 2 & 3, HIPAA

Automatic backups No performance 35 days of Archive impact PITR snapshots

Automatic, incremental, and Backups do not affect Point-in-time recovery Keep snapshots for as long as continuous backups database performance (PITR) for up to 35 days you need

"Adopting Amazon DocumentDB is a game-changer because we offload management, security, and backup of our MongoDB databases to AWS. With Amazon DocumentDB, we can add or scale instances in minutes, regardless of data size. Further, we get automatic backups and point-in-time restore capabilities, which far exceed other managed DB services at less cost.”

Instances: Size/hr * count (db.r5.large $0.277/hr)

Compute Writes I/O: Count ($0.20/million) Reads Reads Reads Distributed storage volume

Storage Storage: GB/mo ($0.10/GB)

Offline migration using MongoDB utilities: Get started quickly, great for POCs

Online migration using AWS DMS Near-zero downtime migration

Hybrid migration leverages both solutions Best of both worlds

https://docs.aws.amazon.com/documentdb/latest/developerguide/docdb-migration.html All options support MongoDB on-premises and EC2, for both replica sets and sharded clusters