AWS Storage Gateway Fred Hutch - Partly Cloudy Conference

Home , Cloud storage gateway

Jeff Bartley | Storage Solutions Architect | [email protected]

October 25th, 2018

Amazon Amazon Amazon Amazon Amazon Amazon AWS Amazon EMR Athena S3 Glacier EBS EC2 Lambda CloudFront

AWS Storage Gateway

Enterprise Data Centers Remote Offices Research Sites Small-Medium Businesses

Multimedia content File servers Databases App. servers Devices Users Backup servers

File Gateway Store and access objects in Amazon S3 from file-based applications with local caching

Volume Gateway Block storage on-premises backed by cloud storage with local caching, EBS snapshots, and clones

Tape Gateway Drop-in replacement for physical tape infrastructure backed by cloud storage with local caching

Files Amazon S3 (NFS or SMB) AWS Storage Gateway Amazon EBS Volumes snapshots (iSCSI) HTTPS Gateway Service Amazon Glacier Tapes AWS Identity and Access (iSCSI VTL) AWS CloudTrail Management (IAM)

Amazon AWS Key Management Service Customer Premises CloudWatch (AWS KMS)

Gateway provides applications Native storage in AWS • Protocol conversion and device emulation • Objects in S3 (file) • Caching (read-through / write-back) • Snapshots in EBS (volume) • Optimized data transfer • Archives in Glacier (tape)

Customer Premises

NFS or HTTPS SMB Application File Gateway Objects in your Server S3 bucket

Reduce on-premises Fully managed local Durability, scalability, storage infrastructure cache for low latency and reliability of access Amazon S3 storage

SMB v2 or v3 File Gateway Objects in your client Amazon S3 bucket Store and access objects in Amazon S3 buckets from file-based Windows applications

On-premises Active Share-level access can POSIX ACLs Objects created Directory (AD) or AWS be restricted to (compatible subset of directly in S3 inherit Directory services configured users and NTFS) for object-level ACLs from parent groups permissions folder

Customer Premises AWS Region

1 2 3

Database File Gateway Objects in S3 Database server instance

Amazon 1 Database server creates backup file CloudWatch Events 2 Gateway takes care of upload to S3 3 Automate restore or validation in-cloud or lifecycle to Glacier for archival

1 2

Application Objects in your S3 Application File Gateway bucket File Gateway (read-only) San Diego Data Center AWS Region Sydney Data Center

1 Application in San Diego writes files which are uploaded to S3 by gateway

2 After refresh cache, files are visible to applications in Sydney. Local cache improves access performance.

2 NAS

Application File Gateway AWS Direct S3 Standard S3-Infrequent Amazon LTO Servers Connect Access Glacier Customer Data Center

1 Use Snowball to ship data from on-premises offline archives 2 Online access to all data through gateway, reduces on-premises storage cost, S3 lifecycle policies to lower costs further

“We have applications that don’t support S3 natively, and we can’t make changes to the applications, so being able to use native protocols is very important to us. Storage Gateway allows Celgene to get data to the cloud without making any changes to the application or environment.”

– Lance Smith, IT Director, Celgene

© 2018,© 2018, Amazon Amazon Web Web Services, Services, Inc. Inc. or itsor Affiliates.its Affiliates. All rightsAll rights reserved. reserved. Amazon Confidential and Trademark File Gateway – Control Over Data Storage and Access

NFS or SMB File Gateway Objects in your S3 client bucket

File share options Amazon S3 options per bucket • Restrict access by IP (NFS) or AD (SMB) • IAM role • Read-only/read-write • Storage class • Default ownership and permissions • Encryption with AWS KMS • User squashing (NFS) • Guess MIME type, requester pays, bucket owner ACL, …

\\{IP}\{Bucket1}\{Folder}\{File1} s3://{Bucket1}/{Prefix}/{Object1} \\{IP}\{Bucket1}\{Folder}\{File2} s3://{Bucket1}/{Prefix}/{Object2} \\{IP}\{Bucket1}\{Folder}\{File3} s3://{Bucket1}/{Prefix}/{Object3}

SMB v2 or v3 client Objects in Bucket1 your Amazon S3 bucket NFS v3 or v4 File Gateway client Bucket2

{IP}:/{Bucket2}/{Folder}/{File1} s3://{Bucket2}/{Prefix}/{Object1} {IP}:/{Bucket2}/{Folder}/{File2} s3://{Bucket2}/{Prefix}/{Object2} {IP}:/{Bucket2}/{Folder}/{File3} s3://{Bucket2}/{Prefix}/{Object3}

File Gateway

File Cache share Disk NFS or SMB S3 Bucket client

Customer Premises AWS Region

File Gateway

Data cache File Cache share Disk NFS or SMB Metadata S3 Bucket client cache

Customer Premises AWS Region

File Gateway