S V S 4 0 5 - R A Serverless Journey: Under the Hood of AWS Lambda Holly Mesrobian Marc Brooker Director of Engineering Senior Principal Engineer Amazon AWS Lambda Amazon AWS Serverless Amazon Web Services Amazon Web Services © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SERVERLESS AT SCALE IS THE NEW NORM processes 4,000 requests executes 16 million per second requests a month processes half a trillion validations of stock trades daily ingests, analyzes and processes tens of stores 17+ petabytes of billions of data data per season points monthly API traffic to register and license more than 47 million driver records in Great Britain, © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Load Balancing Scaling Up and Down Handling Failures Predictable Low Latency AWS Lambda Handles Event Processing Stream Processing Predictable Performance Innovations in Isolation © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability Zone 1 Worker Mgr Worker Claim Worker Invoke Placement Front End Invoke Reserve Sandbox Worker Mgr Init Lambda customer Worker (New Function or Scaling Up) Invoke Front End Worker Availability Zone 2 AWS Cloud Region Availability Zone 1 Worker Mgr Worker Invoke Front End Invoke Reserve Sandbox Lambda customer Worker Mgr Worker (Existing Worker, Existing Sandbox) Invoke Front End Worker Availability Zone 2 Poller Consumes events and ensures they are processed State Manager or Stream Tracker Handles scaling by managing Pollers and event or stream source resources Leasing Service Assigns Pollers to work on a specific event or streaming source © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. “We believe 'once you go serverless you never go back,' hence all of our new developments are serverless by default.” Denis Bauer Head of Cloud Computing & Bioinformatics © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability Zone 1 Amazon SQS Poller Front End Amazon SQS Poller Front End Lambda customer Front End Amazon SQS Poller Front End Availability Zone 2 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability Zone 1 Amazon SQS Amazon SQS Poller Front End Amazon SQS Poller Front End Lambda customer Front End Amazon SQS Poller Front End Availability Zone 2 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability Zone 1 Poller Leasing State Manager Service Amazon SQS Poller State Manager Leasing Service Poller Availability Zone 2 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability Zone 1 Poller Leasing Service Poller Leasing Service Poller Availability Zone 2 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability Zone 1 Amazon SQS Amazon SQS Poller Front End Amazon SQS Poller Front End Lambda customer Front End Front End Amazon SQS Poller Availability Zone 2 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability Zone 1 Poller Amazon Kinesis Kinesis Poller Front End Lambda customer Front End Kinesis Poller Availability Zone 2 AWS Cloud Region Availability Zone 1 Poller Kinesis Stream Tracker Leasing Service Poller Kinesis Stream Tracker Leasing Service Poller Kinesis Availability Zone 2 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability Zone 1 Poller Kinesis Stream Tracker Leasing Kinesis Service Kinesis Poller Stream Tracker Leasing Service Poller Kinesis Availability Zone 2 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability Zone 1 Poller Amazon SQS Kinesis Poller Kinesis Front End Lambda customer Poller Kinesis Front End Availability Zone 2 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda Provisioned Concurrency Provisioned Concurrency Why Not Provisioned Capacity? Capacity: Depends on hardware performance. Concurrency: Does not depend on hardware. Capacity: Need to model host and AZ failure. Concurrency: Host and AZ failure tolerance built in. Capacity: Customer and business changes can effect how efficiently you use capacity. Concurrency: Unit of work done. Why Not Provisioned Rate (Requests per Second)? RPS: Does not consider work-per- unit (batching, etc). Concurrency: Considers work-per- unit. RPS: Does not consider cost-per- unit (e.g. list vs get APIs) Concurrency: Considers cost-per- unit. RPS: Does not consider effects of contention. Concurrency: Contention and concurrency are directly linked. Naïve, Stateless & Non- Coordinating Stateless Amdahl’s Law Most Real Systems Stateless Amdahl’s Law Most Real Contention Systems Stateless Amdahl’s Law Sweet Spot Most Real Systems Concurrency = Rate x Latency Concurrency = Rate x Latency (which depends on concurrency) Provisioned Concurrency Sweet Spot Sweet Spot Sweet Spot Scale Up Reject Load Traditional Architectures Lambda with Provision Provisioned Concurrency Traditional Architectures Lambda with Provision Provisioned Concurrency © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Dedicated to your function What Does KVM Do? What Does Firecracker Do? Guest OS Virtual Devices Hypervisor Hardware Host OS Physical Devices Firecracker Device Emulation © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Learn serverless with AWS Training and Certification Resources created by the experts at AWS to help you learn modern application development Free, on-demand courses on serverless, including • Introduction to Serverless • Amazon API Gateway for Development Serverless Applications • Getting into the Serverless • Amazon DynamoDB for Mindset Serverless Architectures • AWS Lambda Foundations Additional digital and classroom trainings cover modern application development and computing Visit the Learning Library at https://aws.training © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved..