From AWS Backup) © 2019, Amazon Web Services, Inc
Total Page:16
File Type:pdf, Size:1020Kb
STG 304- R Network file system (NFS) evolved: Deep dive on Amazon EFS Kris Alexander Sarwar Raza Principal Product Manager Senior Manager, Product Management Amazon EFS Amazon EFS AWS AWS © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda How to transform faster with file storage? What is Amazon EFS? Deep dive on Amazon EFS Seamless integration with AWS computing models Customer use cases Summary and Q&A Related breakouts STG202 What’s new in AWS file storage STG211 How to use AWS storage for on-premises file-based applications STG238 Deep dive: File storage for business-critical applications STG341 Deep dive on AWS Backup, ft. Rackspace STG342 Rapid online data transfer with AWS DataSync STG364 Securing Amazon EFS for container and data science applications STG312 Securing Amazon EFS for Modern Applications and Data Science STG304-R, R1 Network file system (NFS) evolved: Deep dive on Amazon EFS STG306-R, R1 Deep dive on Amazon FSx for Windows File Server STG307-R, R1 Amazon S3 & FSx for Lustre: Deep dive on high-performance file storage Let’s talk STG312 Modernize your applications with containers using Amazon EFS STG335-R, R1 Reduce TCO for your file-based applications STG336 Using hybrid cloud storage to close a data center and migrate STG347 Choosing the right storage for your high-performance workloads STG361 Simplify and automate backup of AWS data STG403 Security best practices with Amazon EFS STG405 Maximizing Amazon EFS performance for Linux workloads © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How can you transform faster with file storage? Infrastructure Architecture Business Improve fundamentals: Increase agility and Maximize your security, availability, ability to innovate business results performance, and cost Accelerating your infrastructure transformation • Provide highly secure and reliable storage, always available and accessible • Deliver price and performance to match your application needs at scale Infrastructure • Enable native data protection, compliance, and backup Accelerating your architectural transformation • Capitalize on what the cloud offers by integrating natively with AWS services • Secure data using native security methods, encryption, and IAM authorization Architecture • Support latest compute models such as containers, microservices, and serverless Accelerating your business transformation • Reduce the cost and complexity of managing, maintaining, and operating infrastructure • Generate opportunities to increase business value through analytics and machine learning Business • Enable you to focus on innovation Using the right tool for yourthe job workload Amazon Elastic Amazon FSx Amazon FSx File Service (EFS) for Windows File Server for Lustre Linux-Based Windows-Based Compute-Intensive Workloads Workloads Workloads © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Elastic File System (Amazon EFS) Amazon EFS is a fully managed file system that is… Cloud native Highly reliable Cost optimized Use cases for Amazon EFS Home directories Lift and shift enterprise apps Analytics Container storage Web serving Media workflows Application test/dev Content management Database backups Metadata-intensive jobs Scale-out jobs Low latency and serial I/O High throughput and parallel I/O Cloud native | Amazon EFS Elastic Scalable Integrated • Grow & shrink on demand • Grow up to petabytes • Shared access from on- premises, inter region, and • No need to provision and • Performance modes for low cloud-native applications manage infrastructure & latencies and maximum I/O capacity • Integrated with various • Throughput that scales AWS computing models • Pay as you go, pay with storage only for what you use • Access concurrently from • Provisioned throughput thousands of Amazon • Simple to use, create a file available EC2 instances system in seconds • Attach to containers launched by both Amazon ECS and EKS • Use with Amazon SageMaker Cloud native notebooks Shared access VPC VPC Inter-region VPC Peering NFS Clients NFS Clients NFS Clients Other AWS Regions Intra-region VPC Peering Amazon EFS AWS VPN AWS Direct Connect On-Premises Servers NFS Clients Cloud native Highly integrated, shared access VPC VPC Intra-region VPC Peering NFS Clients NFS Clients Amazon EC2 Amazon EKS Other AWS Regions Inter-region VPC Peering Amazon EFS AWS VPN AWS Direct Connect On-Premises Servers Amazon ECS Amazon SageMaker Cloud native Amazon EFS meets you where you are today and tomorrow Your transformational journey On premises Lift and shift Cloud native Highly reliable | Amazon EFS Highly Secure Global available, durable footprint New! • Stores data across three • Control network traffic • New regions recently added: availability zones for high Bahrain, Sao Paulo, availability and durability • Control file and Stockholm, Hong Kong directory access • Access your file system from • Amazon EFS is now multiple AZs • Control administrative available in 19 regions (API) access with AWS IAM • Strong consistency for concurrent access • Encrypt data at rest and in transit • Control NFS client access with AWS IAM Highly reliable Coming soon! Benefits of 3 independent AZs in each AWS Region AZ Amazon EFS AZ AZ Highly reliable Available in 19 Regions | Added 9 Regions since re:Invent 2018 • US East (N. Virginia) Launched in November 2019 • US East (Ohio) • Middle East (Bahrain) • AWS GovCloud (US-West) 2018 • EU (Stockholm) • US West (Oregon) • South America (Sao Paulo) • US West (N. California) • Asia Pacific (Hong Kong) • Canada (Central) 2019 • EU (Ireland) • EU (London) 2019 • EU (Frankfurt) • EU (Paris) 2019 • Asia Pacific (Sydney) • Asia Pacific (Seoul) • Asia Pacific (Tokyo) • Asia Pacific (Singapore) 19 • Asia-Pacific (Mumbai) 2019 Regions Cost optimized | Amazon EFS No minimum No need to provision Use with Spot Automatic lifecycle commitments storage Instances management to lower or upfront fees cost storage Cost optimized Enable Lifecycle management Standard storage class Infrequent access storage class General-purpose file storage Cost-optimized for files not accessed every day $0.30/GB-month* $0.025/GB-month* for storage $0.01/GB* for access Pay-as-you-go, effective storage cost of $0.08/GB-month* Cost optimized * Pricing in the US East (N. Virginia) region © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. What’s new with Amazon EFS | Year 2019 February 2019 June 2019 August 2019 October 2019 • EFS Infrequent Access • Asia Pacific (Mumbai) • Amazon SageMaker • EC2 Launch Instance Wizard • Tag-on-create • EU (Paris) integration • AWS Private Link Today January 2019 May 2019 July 2019 September 2019 November 2019 • Native integration with AWS Backup • Canada (Central) • EFS Additional Lifecycle • 44% IA Price Reduction • EFS IA 7-day LM policy • EU (London) • EKS CSI Driver Alpha Management (LM) policies • EKS CSI Driver Beta • Middle East (Bahrain) • Enable LM for existing file • Service Quotas • EU (Stockholm) systems • FedRAMP High certification • South America (Sao • Mount helper config update Paulo) • Asia Pacific (Hong Kong) What’s new with Amazon EFS | Coming soon! Single File Restore IAM Authorization Performance Enhancements for for Amazon EFS NFS Clients (from AWS Backup) © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EFS Infrequent Access Amazon EFS IA storage class for infrequently accessed files for $0.025/GB per month* No changes to existing Cost Automated applications using savings up lifecycle Amazon EFS to 92% management * Pricing in the US East (N. Virginia) region Enabling EFS Lifecycle Management Lifecycle policies can be configured to 7 New! ,14, 30, 60, or 90 days since last access All EFS features are supported with Amazon EFS IA Enable Lifecycle Infrequently accessed Management, choose files automatically lifecycle policy moved to Amazon EFS IA Amazon EFS performance modes General Purpose (default) Recommended for the majority of workloads EFS performance modes General Purpose Max (default) I/O Recommended for the Recommended for majority of workloads scale-out workloads EFS performance modes: General purpose What it’s for Latency-sensitive Large-scale and applications and general- data-heavy applications purpose workloads Advantages Lowest latencies Virtually unlimited for file operations ability to scale out throughput/IOPS General Purpose Trade-offs Max (default) Limit of 7k ops/sec Slightly higher I/O Enhancements coming metadata latencies Recommended for the soon! Recommended for majority of workloads scale-out workloads When to use Best choice for most Consider for large workloads scale-out workloads EFS throughput modes Bursting Throughput (default) Recommended for the majority of workloads EFS throughput modes Bursting Throughput Provisioned (default) Throughput Recommended for the Recommended for majority of workloads higher throughput-to- storage ratio workloads EFS throughput modes What it’s for Varying throughput Higher-consistent workloads throughput workloads Advantages Auto-scaling User-defined throughput throughput Trade-offs Bursting Throughput Fixed throughput-to- Separate Provisioned (default) storage ratio throughput charge Throughput Recommended for the When to use Recommended for majority of workloads Best choice for most Higher throughput-to- higher throughput-to- storage ratio workloads workloads storage ratio Loading more than 2.1 TB Provisioned Throughput Independent Switch or throughput Increase decrease Provision throughput As often