Amazon Elasticache Deep Dive Powering Modern Applications with Low Latency and High Throughput

Home , Amazon DynamoDB, Amazon ElastiCache, Amazon S3, Memcached

Amazon ElastiCache Deep Dive Powering modern applications with low latency and high throughput

Michael Labib Sr. Manager, Non-Relational Databases

• Introduction to Amazon ElastiCache • Redis Topologies & Features • ElastiCache Use Cases • Monitoring, Sizing & Best Practices

Users 1M+ Data volume Terabytes—petabytes Locality Global Performance Microsecond latency Request rate Millions per second Access Mobile, IoT, devices Scale Up-out-in

E-Commerce Media Social Online Shared economy Economics Pay-as-you-go streaming media gaming Developer access Open API

Redis & Extreme Secure Easily scales to Memcached compatible performance and reliable massive workloads

Fully compatible with In-memory data store Network isolation, encryption Scale writes and open source Redis and cache for microsecond at rest/transit, HIPAA, PCI, reads with sharding and Memcached response times FedRAMP, multi AZ, and and replicas automatic failover

© 2020, Amazon Web Services, Inc. or its Affiliates. What is Redis? Initially released in 2009, Redis provides: • Complex data structures: Strings, Lists, Sets, Sorted Sets, Hash Maps, HyperLogLog, Geospatial, and Streams • High-availability through replication • Scalability through online sharding • Persistence via snapshot / restore • Multi-key atomic operations A high-speed, in-memory, non-Relational data store. • LUA scripting Customers love that Redis is easy to use. • Open Source

Initially released in 2003, Memcached provides: • Simple, in-memory, LRU cache • Simple key-value (string-string) store • Supports strings, objects • Multi-threaded Single-Node Instance • Sharding via client-side library Sharded Instance • Easy to Scale • No persistence • Open source

Clients ClientsClients ClientsClients ClientsClients Clients

ElastiCache + RDS ElastiCache + Aurora ElastiCache + Redshift ElastiCache + DynamoDB ElastiCache + DocumentDB ElastiCache + ….

Redis Node Client Client List String

Client Hash Bitmap Set Client Stream Geospatial Sorted Set

Client HyperLogLog Client

Client Client

A=1 A=1 Redis Node WriteClient Client Read Client A:1 A=1 List String A=1

<1ms Read Client Hash Bitmap Set Read Client A=1 Stream Geospatial Sorted Set A=1

Read Client HyperLogLog Read Client A=1 A=1

Read Client Read Client

© 2020, Amazon Web Services, Inc. or its Affiliates. Redis Cluster Mode – Enabled vs. Disabled Feature Redis Cluster (enabled) Redis Cluster (disabled) Recovery Time 10-20 sec (non-DNS) ~30+ sec (DNS) Writes affected on failed shard. Writes affected on entire data set. Failover Impact Reads available Reads available Up to 250* nodes (90 = 15 shards + 5 1 primary Node Scale replicas soft limit) 0-5 replicas (max. 6 nodes) 0–5 replicas per shard Primary Endpoint Storage 170 TB (635 GB x 250) 635 GB Slot 0 Max Connections 16.25 million (65,000 x 250) 390,000 (65,000 x 6) P Slot 1 … R Online Scaling Shards and read replicas Read replicas only Slot 16383

Migration Path Backup/Restore Snapshot Online Migration Tool

• Throughput limited by 1 primary, 5 Configuration • Achieve greater throughput through replicas Endpoint Scalability and horizontal scaling • Horizontal Scale for Reads (Replicas) P Slot 0–5461 R Performance • Horizontal/Vertical scaling supported P Slot 5462–10922 R Supported • Vertical scaling for Replicas/Primary Slot 10923– also supported P R 16383

Cluster Resizing (zero-downtime) • Horizontal Scaling to add/remove Vertical Scaling Scaling Operation shards • Writes/Reads continue during scale up • Read Scalability to add/remove operation replicas

VPC

Availability zone 1 Availability zone 2

Elastic Load Balancing Public subnet Public subnet Connect to Primary for Read/Writes and Replica’s for Reads

Auto Scaling group

Amazon EC2 Amazon EC2

Private subnet Private subnet Keyspace

All keys remain on same single node Redis node Redis node Primary Endpoint Replica Endpoints

© 2020, Amazon Web Services, Inc. or its Affiliates. ElastiCache for Redis Multi-AZ Auto-Failover § Chooses replica with lowest replication lag Automatic Failover to a read replica in case of § DNS endpoint is same primary node failure Multi-AZ ElastiCache Automates writes snapshots for persistence

Use Primary Endpoint Primary ElastiCache for Redis

reads Replica Replica

Use Read Replicas ElastiCache ElastiCache or Reader Endpoint for Redis for Redis Availability Zone A Availability Zone B © 2020, Amazon Web Services, Inc. or its Affiliates. ElastiCache with Redis Multi-AZ

Region

Auto Scaling

ElastiCache Cluster Read Replica Primary

Availability Zone A Availability Zone B

Region

Auto Scaling

ElastiCache Cluster Read Replica Primary

Availability Zone A Availability Zone B

Region

Auto Scaling

ElastiCache Cluster Read Replica Primary

Availability Zone A Availability Zone B

VPC

Availability zone 1 Availability zone 2

Elastic Load Balancing Public subnet Public subnet

Clients use hash value for ( CW Metric: CurrItems ) a key CRC16(key) mod 16384 Auto Scaling group Distribution Equal | Custom Amazon EC2 Amazon EC2

Cluster Map

Private subnet Partitioned by Shard Private subnet

Shard 1 Slot 0 - … Keyspace Configuration Shard 2 Slot … to … Endpoint

Shard 3 Slot … to … Zero Downtime Scaling Redis node Redis node Redis node Redis node Shard 4 Slot … to 16383

Example: S S S 3 shards 2 replicas per shard Multi-AZ Shards, primaries, and read replicas • Add shards to scale reads/writes, increase in-memory capacity

Redis Cluster • Add replicas to scale reads, increase availability • Able to specify availability zones. Multi-AZ default

P R R P R R • Able to customize slot distributions, equal distribution Slots 0–5454 Slots 5455–10909 Slots 0–5454 Slots 5455–10909 Slots 0–5454 Slots 5455–10909 default R R P

Slots10910–16363 Slots10910–16363 Slots10910–16363

Availability Zone A Availability Zone B Availability Zone C

CW Metric: ReplicationLag

x15 Shard

async replication

Cache node Cache node ElastiCache for Redis Primary Replica

Failover Detection Shard

async replication

Cache node Cache node ElastiCache for Redis Primary Replica

SNS Event: ElastiCache:CacheNodeReplaceComplete SNS Event: ElastiCache:FailoverComplete

Shard

Test with Failover API async replication Cache node Cache node ElastiCache for Redis Primary Replica

Automatic Failover (with no DNS propagation)

Shard 1 Shard 2 Shard 3

0-5461 5462--10922 10923-16383

Simple API aws elasticache modify-replication-group-shard-configuration --replication-group-id rep-group-id --apply-immediately --node-group-count 5

Scale In || Out

Shard 1 Shard 2 Shard 3 0-2909, 5462-5783, 0-5461 5462--10922 10923-1638314199 5095-5461 6876-9830

Uniform slot distribution across shards reads/ writes Amazon EC2 Shard 4 No Application Interruption Shard 5

2910-5094, 5784-6875, 9831--10922 14200-16383

Shard 1 Shard 2 Shard 3

0-5461 5462--10922 10923-16383

Uniform slot distribution across shards reads/ writes Amazon EC2 Shard 4 No Application Interruption Shard 5

Example for a worldwide application • One-click setup for existing clusters • Write locally, read globally • Enable cross-region disaster Secondary recovery (Passive) Region Read Primary • Leverage extreme performance with (active) region Read/Write

Redis’ sub-millisecond latency Secondary (Passive) Region • Secure encryption in transit for Read cross-region traffic • Use with AWS Management Console, or latest AWS SDK or CLI

• Scale up to of

• Dynamic network processing to enhance

Difficult Hard to make Difficult Expensive to manage highly available to scale

Manage server provisioning, Need to implement Online scaling can be error Invest in people, processes, software patching, setup, fast error detection prone, replication hardware, and software configuration, and backups and remediation performance needs to be monitored

Cluster Mode (enabled)

{ "Version": "2012-10-17", Recommendation: Leverage planned maintenance window "Id": "Policy15397346", "Statement": [ { "Sid": "Stmt15399483", "Effect": "Allow", "Principal": { "Service": "ap-east-1.elasticache-snapshot.amazonaws.com" 1. Create a Redis Backup }, "Action": [ "s3:GetObject", 2. Create an Amazon S3 Bucket and Folder "s3:ListBucket", "s3:GetBucketAcl" ], 3. Upload Your Backup to Amazon S3 "Resource": [ "arn:aws:s3:::example-bucket", "arn:aws:s3:::example-bucket/backup1.rdb", 4. Grant ElastiCache Read Access to the .RDB File "arn:aws:s3:::example-bucket/backup2.rdb" ] } 5. Seed the ElastiCache Cluster with the .RDB File Data ] © 2020, Amazon Web Services, Inc. or its Affiliates. } Migrate using the Online Migration tool

Cluster Mode (disabled) Cluster Mode (disabled)

Source Target Overview: • Replicates data in real-time • Supported Instances include T3, M4, M5, R4 and R5 • Health monitoring during and after the migration • Customer decides when to cutover to the migrated cluster

In-Transit Encryption At-Rest Encryption

Encrypts application-to-node and node-to-node Used for snapshots and during replication network communications May impact performance TLS 1.0 – 1.2 supported

Server verification / authentication

May impact performance Authentication

Ability to set AUTH token Compliance

• HIPAA Eligibility for ElastiCache for Redis • Included in AWS Business Associate Addendum • Redis 3.2.6

Caching Real-time Gaming Geospatial Media analytics leaderboards streaming

Session Chat apps Message Machine store queues learning

Amazon ElastiCache Lazy Loading 1 1 Cache hit: read from cache

2 Cache miss: read from database

3 Write data to cache (with TTL) 3 2

Your Data Store Application

Amazon EC2 Web/App Server

Amazon EC2 Load Balancer Web/App Server

Amazon EC2 Amazon Web/App Server ElastiCache

Amazon EC2 Web/App Server

Amazon ElastiCache

Amazon In-memory data store ElastiCache and cache to decrease access latency, increase throughput, and ease the Amazon Redshift load off databases and applications Amazon Elasticsearch Service

Clients Amazon S3

Amazon EC2 AWS Lambda Amazon API Gateway

Amazon RDS

Amazon Neptune

Amazon DynamoDB

Authenticate

Amazon Cognito

Geospatial and cached data Authorize

APIs geo-based Amazon API Gateway AWS Lambda queries Amazon ElastiCache Write-back, for Redis geo add

Lambda function

Amazon DynamoDB

Other Resources Reduce backend pressure

APIs AWS Lambda Rate Limiter (Redis Counter) Amazon ElastiCache for Redis

Amazon EC2 application

Devices

House Lightbulb Generic Thermostat Utility

AWS IoT Core Rule AWS Lambda Amazon ElastiCache for Redis

Data Lake

Amazon Kinesis Amazon S3 Data Firehose

Stream (Analytics) Stream (Analytics)

Amazon Kinesis

Parse Cleansed Stream Stream (Cleansed)

AWS Lambda

RAW

Decorate and Filter Amazon Raw Data Amazon Kinesis AWS Lambda ElastiCache for Parse Redis Raw Stream Filter Raw Stream (RAW) Stream

Redis steams support a time sequenced series of records (like a log file).

Operations: • Add records to the end of the stream • Trim/Discard old entries from the stream • Ranges of records can be retrieved and/or counted • Multiple clients can independently process the same stream • Consumer groups allow clients to split a stream across clients

• Messages are categorized into channels • Subscribers can subscribe to multiple patterns or channels • Publishers publish to a given channel • Messages are not persisted • Clients must be connected to receive > psubscribe sports:* • Two main commands: PUBLISH and Reading messages... SUBSCRIBE 1) "psubscribe" 2) "sports:*" > publish sports:patriots "Goooo team!" 3) (integer) 1 (integer) 1 1) "pmessage" 2) "sports:*" 3) "sports:patriots" 4) "Goooo team!"

• CPUUtilization • Memcached – up to 90% ok • EngineCPUUtilization • Redis CPU [Up to 90% OK] • SwapUsage low • CacheMisses / CacheHits Ratio low • Evictions near zero • Exception: Russian doll caching • CurrConnections stable

• Setup alarms with CloudWatch Metrics

Eviction Policy Type Subtype Name Description

LRU ** All Keys allkeys-lru Evicts the least recently used (LRU) regardless of TTL set

LRU * Volatile volatile-lru Evicts the least recently used (LRU) from those that have a TTL set

LFU ** All Keys allkeys-lfu Evict any key using approximated least frequently used (LFU)

LFU * Volatile volatile-lfu Evict using approximated LFU among the keys with a TTL set

TTL * Volatile volatile-ttl Evicts the keys with shortest TTL set

Random * Volatile volatile-random Randomly evicts keys with a TTL set

Random ** All Keys allkeys-random Randomly evicts keys regardless of TTL set

No Doesn’t evict keys at all. This blocks future writes until memory frees No Eviction no-eviction Eviction up.

* Volatile policies only evicts keys with TTLs ** Highlighted policies are typically considered safest until key patterns are well understood

• Cluster mode enabled Scale Out/in [add/remove shards]: • No downtime, cluster remains available for requests while slots are evenly distributed across Shards • If applicable, it is recommended to resize a cluster during off-peak hours to avoid a performance penalty • Scaling Vertically: • A new cluster is initialized beside the existing, new node type is applied to all nodes. • Upon cluster synchronization, Redis 5.0.5 cutover is <1sec, older versions can take up to a minute. • Cluster mode disabled Read-only Scaling [add replicas] : • Add/Remove replicas incurs no downtime to application. • Reader endpoint stays up-to-date in real-time during replica addition/removal & distributes traffic evenly. • Compute Node Types: • R5 & M5 instance types leveraging AWS Nitro System optimizations and Enhanced IO improvements. • This provides significantly better price/throughput allowing your cluster to handle more traffic while keeping the cost low

• Cluster mode enabled Scale Out/in [add/remove shards]: • No downtime, cluster remains available for requests while slots are evenly distributed across Shards • If applicable, it is recommended to resize a cluster during off-peak hours to avoid a performance penalty • Scaling Vertically: • A new cluster is initialized beside the existing, new node type is applied to all nodes. • Upon cluster synchronization, Redis 5.0.5 cutover is <1sec, older versions can take up to a minute. • Cluster mode disabled Read-only Scaling [add replicas] : • Add/Remove replicas incurs no downtime to application. • Reader endpoint stays up-to-date in real-time during replica addition/removal & distributes traffic evenly. • Compute Node Types: • R5 & M5 instance types leveraging AWS Nitro System optimizations and Enhanced IO recommended. • In addition to significantly better price/throughput, improves seamless scaling and failover operations. • Redis Engine Version In-Place Upgrade: • Upgrade of version with minimal downtime. • Cluster available for reads during engine upgrades, writes are interrupted only for <1sec with version 5.0.5 • Upgrading versions earlier than 5.0.5 can incur <1minute interruption due to DNS propagation.