From AWS Backup) © 2019, Amazon Web Services, Inc

STG 304- R Network file system (NFS) evolved: Deep dive on Amazon EFS

Kris Alexander Sarwar Raza Principal Product Manager Senior Manager, Product Management Amazon EFS Amazon EFS AWS AWS

How to transform faster with file storage? What is Amazon EFS? Deep dive on Amazon EFS Seamless integration with AWS computing models Customer use cases Summary and Q&A Related breakouts

STG202 What’s new in AWS file storage STG211 How to use AWS storage for on-premises file-based applications STG238 Deep dive: File storage for business-critical applications STG341 Deep dive on AWS Backup, ft. Rackspace STG342 Rapid online data transfer with AWS DataSync STG364 Securing Amazon EFS for container and data science applications STG312 Securing Amazon EFS for Modern Applications and Data Science STG304-R, R1 Network file system (NFS) evolved: Deep dive on Amazon EFS STG306-R, R1 Deep dive on Amazon FSx for Windows File Server STG307-R, R1 Amazon S3 & FSx for Lustre: Deep dive on high-performance file storage Let’s talk

STG312 Modernize your applications with containers using Amazon EFS STG335-R, R1 Reduce TCO for your file-based applications STG336 Using hybrid cloud storage to close a data center and migrate STG347 Choosing the right storage for your high-performance workloads STG361 Simplify and automate backup of AWS data STG403 Security best practices with Amazon EFS STG405 Maximizing Amazon EFS performance for Linux workloads © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How can you transform faster with file storage?

Infrastructure Architecture Business

Improve fundamentals: Increase agility and Maximize your security, availability, ability to innovate business results performance, and cost Accelerating your infrastructure transformation

• Provide highly secure and reliable storage, always available and accessible

• Deliver price and performance to match your application needs at scale Infrastructure • Enable native data protection, compliance, and backup Accelerating your architectural transformation

• Capitalize on what the cloud offers by integrating natively with AWS services

• Secure data using native security methods, encryption, and IAM authorization Architecture • Support latest compute models such as containers, microservices, and serverless Accelerating your business transformation

• Reduce the cost and complexity of managing, maintaining, and operating infrastructure

• Generate opportunities to increase business value through analytics and machine learning Business • Enable you to focus on innovation Using the right tool for yourthe job workload

Amazon Elastic Amazon FSx Amazon FSx File Service (EFS) for Windows File Server for Lustre

Amazon EFS is a fully managed file system that is…

Cloud native Highly reliable Cost optimized Use cases for Amazon EFS

Home directories Lift and shift enterprise apps Analytics Container storage Web serving Media workflows Application test/dev Content management Database backups

Metadata-intensive jobs Scale-out jobs

Low latency and serial I/O High throughput and parallel I/O Cloud native | Amazon EFS

Elastic Scalable Integrated

• Grow & shrink on demand • Grow up to petabytes • Shared access from on- premises, inter region, and • No need to provision and • Performance modes for low cloud-native applications manage infrastructure & latencies and maximum I/O capacity • Integrated with various • Throughput that scales AWS computing models • Pay as you go, pay with storage only for what you use • Access concurrently from • Provisioned throughput thousands of Amazon • Simple to use, create a file available EC2 instances system in seconds • Attach to containers launched by both Amazon ECS and EKS • Use with Amazon SageMaker Cloud native notebooks Shared access

VPC VPC

Inter-region VPC Peering

NFS Clients NFS Clients NFS Clients

Other AWS Regions

Intra-region VPC Peering

Amazon EFS

AWS VPN

AWS Direct Connect On-Premises Servers

NFS Clients

Cloud native Highly integrated, shared access

VPC VPC

Intra-region VPC Peering

NFS Clients NFS Clients Amazon EC2 Amazon EKS

Other AWS Regions

Inter-region VPC Peering

Amazon EFS

AWS VPN

AWS Direct Connect On-Premises Servers Amazon ECS Amazon SageMaker

Cloud native Amazon EFS meets you where you are today and tomorrow

Your transformational journey

On premises Lift and shift Cloud native Highly reliable | Amazon EFS

Highly Secure Global available, durable footprint New!

• Stores data across three • Control network traffic • New regions recently added: availability zones for high Bahrain, Sao Paulo, availability and durability • Control file and Stockholm, Hong Kong directory access • Access your file system from • Amazon EFS is now multiple AZs • Control administrative available in 19 regions (API) access with AWS IAM • Strong consistency for concurrent access • Encrypt data at rest and in transit • Control NFS client access with AWS IAM Highly reliable Coming soon! Benefits of 3 independent AZs in each AWS Region

Amazon EFS AZ

Highly reliable Available in 19 Regions | Added 9 Regions since re:Invent 2018

• US East (N. Virginia) Launched in November 2019 • US East (Ohio) • Middle East (Bahrain) • AWS GovCloud (US-West) 2018 • EU (Stockholm) • US West (Oregon) • South America (Sao Paulo) • US West (N. California) • Asia Pacific (Hong Kong) • Canada (Central) 2019 • EU (Ireland) • EU (London) 2019 • EU (Frankfurt) • EU (Paris) 2019 • Asia Pacific (Sydney) • Asia Pacific (Seoul) • Asia Pacific (Tokyo) • Asia Pacific (Singapore) 19 • Asia-Pacific (Mumbai) 2019 Regions Cost optimized | Amazon EFS

No minimum No need to provision Use with Spot Automatic lifecycle commitments storage Instances management to lower or upfront fees cost storage

Cost optimized Enable Lifecycle management

Standard storage class Infrequent access storage class

General-purpose file storage Cost-optimized for files not accessed every day $0.30/GB-month* $0.025/GB-month* for storage $0.01/GB* for access

Pay-as-you-go, effective storage cost of $0.08/GB-month*

February 2019 June 2019 August 2019 October 2019 • EFS Infrequent Access • Asia Pacific (Mumbai) • Amazon SageMaker • EC2 Launch Instance Wizard • Tag-on-create • EU (Paris) integration • AWS Private Link

Today

January 2019 May 2019 July 2019 September 2019 November 2019 • Native integration with AWS Backup • Canada (Central) • EFS Additional Lifecycle • 44% IA Price Reduction • EFS IA 7-day LM policy • EU (London) • EKS CSI Driver Alpha Management (LM) policies • EKS CSI Driver Beta • Middle East (Bahrain) • Enable LM for existing file • Service Quotas • EU (Stockholm) systems • FedRAMP High certification • South America (Sao • Mount helper config update Paulo) • Asia Pacific (Hong Kong) What’s new with Amazon EFS | Coming soon!

Single File Restore IAM Authorization Performance Enhancements for for Amazon EFS NFS Clients (from AWS Backup) © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EFS Infrequent Access Amazon EFS IA storage class for infrequently accessed files for $0.025/GB per month*

No changes to existing Cost Automated applications using savings up lifecycle Amazon EFS to 92% management

* Pricing in the US East (N. Virginia) region Enabling EFS Lifecycle Management Lifecycle policies can be configured to 7 New! ,14, 30, 60, or 90 days since last access

All EFS features are supported with Amazon EFS IA Enable Lifecycle Infrequently accessed Management, choose files automatically lifecycle policy moved to Amazon EFS IA Amazon EFS performance modes

General Purpose (default)

Recommended for the majority of workloads EFS performance modes

General Purpose Max (default) I/O

Recommended for the Recommended for majority of workloads scale-out workloads EFS performance modes: General purpose

What it’s for Latency-sensitive Large-scale and applications and general- data-heavy applications purpose workloads

Advantages Lowest latencies Virtually unlimited for file operations ability to scale out throughput/IOPS

General Purpose Trade-offs Max (default) Limit of 7k ops/sec Slightly higher I/O Enhancements coming metadata latencies Recommended for the soon! Recommended for majority of workloads scale-out workloads When to use Best choice for most Consider for large workloads scale-out workloads EFS throughput modes

Bursting Throughput (default)

Recommended for the majority of workloads EFS throughput modes

Bursting Throughput Provisioned (default) Throughput

Recommended for the Recommended for majority of workloads higher throughput-to- storage ratio workloads EFS throughput modes

What it’s for Varying throughput Higher-consistent workloads throughput workloads

Advantages Auto-scaling User-defined throughput throughput

Trade-offs Bursting Throughput Fixed throughput-to- Separate Provisioned (default) storage ratio throughput charge Throughput

Recommended for the When to use Recommended for majority of workloads Best choice for most Higher throughput-to- higher throughput-to- storage ratio workloads workloads storage ratio Loading more than 2.1 TB Provisioned Throughput

Independent Switch or throughput Increase decrease

Provision throughput As often as you need Once every 24+ hours independently of data stored Bursting Throughput pricing Single pricing dimension

Storage price $0.30/GB-month* Includes 50 KiB/s throughput per GiB of storage

* Pricing in the US East (N. Virginia) region Provisioned Throughput pricing Two pricing dimensions

Storage price $0.30/GB-month* Includes 50 KiB/s throughput per GiB of storage

* Pricing in the US East (N. Virginia) region Provisioned Throughput pricing Two pricing dimensions

Storage price Throughput price $0.30/GB-month* $6.00/MiBps-month* Includes 50 KiB/s throughput per GiB of storage Above 50 KiB/s throughput per GiB of storage

* Pricing in the US East (N. Virginia) region Security and compliance

Control Control file and Control administrative Encrypt Achieve network traffic directory access access (API access) data compliance

Using Amazon VPC Using POSIX Using AWS IAM, action- At rest and HIPAA security groups and permissions level and resource-level in transit network ACLs permissions, identity- GDPR based policies PCI-DSS SOC ISO FedRAMP Amazon EFS | IAM authorization for NFS clients Restricting EFS access using IAM Identity Policy Coming soon!

IAM Creds Orchestrates

{ "Statement" : { "Effect" : "allow", "Action" : "elasticfilesystem:Client*", "Resource": "fs-deadbeef" } } Amazon EFS | IAM authorization for NFS clients Restricting EFS access using IAM Resource Policy Coming soon!

IAM Creds Orchestrates

{ "Statement" : { "Effect" : "allow", "Action" : "elasticfilesystem:Client*", "Principal" : { "AWS": "myrole" } } } © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS integrations | Amazon EFS

Amazon AWS AWS Amazon AWS AWS AWS Direct AWS VPN VPC IAM KMS CloudWatch CloudTrail CloudFormation Connect

Amazon EFS

Amazon Amazon Amazon AWS Backup Amazon AWS AWS EC2 SageMaker EKS ECS DataSync Fargate Coming soon! Amazon EC2 launch wizard | Amazon EFS

• Mount Amazon EFS file systems directly from the Amazon EC2 launch wizard

• Select an existing file system to mount or create a new file system Container high availability

Availability Zone Availability Zone

Examples Active Standby Jupyter JIRA

Jenkins EFS Mount EFS Mount Target Target

Amazon EFS file system Shared storage for scale-out apps

Availability Zone Availability Zone

Examples WordPress Drupal

ML training EFS Mount EFS Mount (MXNet, TensorFlow) Target Target

Amazon EFS file system Backup for Amazon EFS

• EFS file systems can be backed up and restored using AWS Backup • AWS Backup provides Amazon EFS automated backup scheduling and retention per user defined policy Backup encryption • AWS Backup offers two classes Warm storage of service backup storage with

AWS Backup the ability to lifecycle to cold Cold storage storage • Restore individual files and directories Coming soon! Migration to Amazon EFS

On premises Region AWS storage resources

Amazon S3 bucket NFS TLS

Shared AWS DataSync file systems AWS DataSync agent service Amazon EFS file system

AWS DataSync: Online transfer service that simplifies, automates, and accelerates moving data between on-premises storage and AWS © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Use cases for Amazon EFS

Home directories Lift and shift enterprise apps Analytics Container storage Web serving Media workflows Application test/dev Content management Database backups

Metadata-intensive jobs Scale-out jobs

Low latency and serial I/O High throughput and parallel I/O Web serving

• Nearly 225 million page views and “We now have capacity on demand. We only pay 50 million unique users monthly for what we use. We are confident that we will NEVER run out of storage space. • No up-front provisioning CAPEX We are always on the latest hardware. • EFS saved $10K/month on storage Our bandwidth is no longer capped, and we don’t have to worry about hitting limits. • 67% cheaper overall—and more secure We have the added security of WAF with • Encrypted data transfer with AWS Advanced Shield. Best of all, we were able to get a better service, Snowball for 67% less!” • Eliminated capacity concerns, colocation contracts, bandwidth caps, hardware upgrades - Edward Zhitnitsky Media technology manager MicroStrategy uses AWS to deploy platform in minutes, not weeks

Challenge Solution Benefits

MicroStrategy needed to MicroStrategy offers its full • Enables deployment in 30 minutes instead meet customer demands analytics and mobility of weeks for faster analytics by platform on the AWS Cloud, Company: MicroStrategy • Helps beverage company give sales people deploying the full taking advantage of faster access to data Industry: Big Data, Analytics, MicroStrategy platform in services such as Amazon Batch Processing, & BI less time. The company EFS, Amazon RDS, AWS • Enables pharmaceutical firm to deliver Country: United States also needed an easier way CodeDeploy, and Amazon intelligence to 90% of field representatives to scale its platform EC2. • Helps online marketing company empower Employees: 100 quickly. business leaders with real-time business Website: www.microstrategy.com KPI updates

About MicroStrategy MicroStrategy (Nasdaq: MSTR) is a leading worldwide provider of enterprise analytics and mobility software and services. The By optimizing our full analytics and mobility platform company’s mission is to make every enterprise a more intelligent ““ on AWS, we are enabling deployment, installation, enterprise. MicroStrategy delivers and configuration with only 3 clicks instead of 400. modern analytics on an open, comprehensive enterprise platform. Vijay Anand, Vice President for Product Marketing, MicroStrategy ” Container storage

““We are a large organization with lots of • Large scale container clusters running applications with varying requirements for Kubernetes and Mesos availability and performance. EFS provides us with a common storage platform that meets • 16K containers under management these requirements across the board.” • Chose EFS for business agility; reduced time to application/service delivery, security - Amreth Chandrasehar • They use EFS both in General Purpose Principal Architect, T-Mobile and Max IO modes, depending on application type • Combination of homegrown applications (customer facing web apps, analytics) and commercial applications (Tibco) Scale out applications

• Cut genomics and chemistry analysis jobs from weeks to hours “Amazon EFS is easy, scales, works with what we have and was key to accelerating innovation. • Accelerated HPC system creation by over a Ultimately it speeds our time to market and year while delivering practically infinite enables us to help our patients faster.” compute and storage capacity

• Avoided $1.2M in SaaS vendor licenses

- Lance Smith • EFS integrates with existing scientific IT director workflows seamlessly

Test in Start with Consider Provisioned General Purpose Bursting Throughput mode for Linux kernel 4.3+ EFS mount helper performance mode Throughput mode loading >2.1 TB (NFSv4.1)

Large IO size Multiple Multiple Multiple (aggregate IO) threads instances directories Amazon EFS | Best practices

Enable Lifecycle Management Enable encryption Create a backup to automatically at rest and in transit plan to further save up to 92% for sensitive workloads protect your data

Monitor throughput utilization, burst credits, and PercentIOLimit Getting started with Amazon EFS

Sign up for an 10-minute tutorial Start building AWS account Learn how to use your cloud-native Instantly get access Amazon EFS file system to the AWS Free Tier Step-by-step walkthroughs

Get started: https://aws.amazon.com/efs/getting-started/

Tutorials: https://github.com/aws-samples/amazon-efs-tutorial Migration: https://d1.awsstatic.com/whitepapers/aws-load-store-protect-linux.pdf © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you!

Kris Alexander Sarwar Raza [email protected] [email protected]