Moving to Managed File Systems

September 2020

Notices Customers are responsible for making their own independent assessment of the information in this document. This document: (a) is for informational purposes only, (b) represents current AWS product offerings and practices, which are subject to change without notice, and () does not create any commitments or assurances from AWS and its affiliates, suppliers or licensors. AWS products or services are provided “as is” without warranties, representations, or conditions of any kind, whether express or implied. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers.

© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved.

Contents

Introduction ...... 1 Amazon EFS ...... 2 Amazon FSx for Windows File Server ...... 3 Use Cases ...... 3 Use Case 1: Moving to fully managed SMB file shares with AWS ...... 3 Use Case 2: Moving to EFS Hosted Home Directories for Users and Applications ...... 5 Moving your data to the Cloud using Online Data Transfer ...... 6 Moving your data to the Cloud using Offline Data Transfer to AWS ...... 7 Conclusion ...... 8 Contributors ...... 9 Further Reading ...... 9 Document Revisions...... 9

Abstract As enterprises and medium sized businesses take advantage of globalization, they need to store and distribute data and information globally. For example, employees in India may need to access data stored in London or New York, and the other way around. The need to access data globally in a timely manner increases the total cost of ownership (TCO) for storage. Organizations need to consider replication solutions, additional networking WAN/LAN costs, and additional data center and infrastructure costs. This whitepaper discusses two AWS services that can help organizations meet these challenges by moving to a managed . Amazon Web Services Moving to Managed File Systems

Introduction According to IDG's research, 80% of data stored by organizations around the world is unstructured, as reported by TechRepublic1. Unstructured data includes office documents, department shared files, content management systems, web sites, test and development workflows, database , social media feeds, digital pictures and videos, audio transmissions, sensor data, and more. Unstructured data is most commonly stored as files on a file system.

To meet the high demand for file storage, organizations have made significant financial investments into traditional on-premises file storage systems and network attached storage (NAS) systems from on-premises storage vendors. This has led to a proliferation of file shares in corporate data centers over the last two decades.

However, the cost of owning and operating on-premises file storage systems goes beyond the initial capital investment. There are operational costs, such as data center real estate, power, cooling, hardware and software maintenance, regular patching and updating, and repair or replacement of faulty hardware, as well as the specialized staff required to manage and maintain this infrastructure. All this takes time, effort, and money.

Also, for all storage systems, since they are hardware infrastructure, there will come a time when the system can no longer provide enough storage, is not performant enough, or becomes unsupported by the vendor. When this happens, a hardware refresh is required.

While hardware refreshes typically involve the procurement and deployment of new infrastructure, what is often overlooked during storage systems refresh is that customers are repurchasing the capacity and performance of the retiring storage system all over again. They are not simply investing in the capacity, capability, and performance delta between the systems.

Apart from procuring and deploying the new infrastructure, a hardware refresh also involves migrating the data. Whenever data is migrated from one system to another, there is a risk of data loss or data corruption, as well as the risks associated with cutting over to a new storage system due to the potential downtime and application reconfiguration involved. Additionally, the man-hours required for these types of migrations can take weeks, if not months. All of which are hidden, but nonetheless real costs.

1 Amazon Web Services Moving to Managed File Systems

What organizations are ultimately looking for are storage solutions that meet the following criteria:

• Enable them to procure capacity on-demand, avoid large upfront capital investments to test business ideas, and avoid stranded capacity costs.

• Provide global access to data in a secure and compliant manner. • Reduce overhead costs associated with managing and supporting storage overall.

• Reduce the total cost of ownership (TCO) associated with ever-expanding storage needs.

• Transform their organizations faster, innovate quicker, while delivering the highest level of availability, durability, and recoverability. With Amazon Web Services (AWS), customers don’t need to worry about the underlying infrastructure and only need to focus on the functionality of the service they want to use, at any capacity or performance scale.

Amazon Web Services managed file services give customers the ability to move these workloads to the cloud. We will discuss the two most common services in the following section.

Amazon EFS

Amazon Elastic File System (Amazon EFS) provides a simple, scalable, fully managed, elastic cloud-native NFS file system for use with AWS Cloud services and on-premises resources. It supports NFS v4 and is frequently used by customers in simplifying their migrating NAS applications from on-premises to the AWS Cloud. It is built to scale on demand to petabytes without disrupting applications, growing and shrinking automatically as you add and remove files, eliminating the need to provision and manage capacity.

By using Amazon EFS, customers get managed file storage infrastructure, which removes the complexity of deploying, patching, and maintaining the underlying complex infrastructure. Customers can also use Amazon EFS to migrate locally running enterprise applications to the AWS Cloud.

2 Amazon Web Services Moving to Managed File Systems

Amazon FSx for Windows File Server

Amazon FSx for Windows File Server provides fully managed, highly reliable file storage that is accessible over the industry-standard (SMB) protocol. It is built on Windows Server, delivering a wide range of administrative features such as user quotas, end-user file restores, and Microsoft (AD) integration. It offers single-AZ and multi-AZ deployment options, fully managed backups, and encryption of data at rest and in transit. Amazon FSx file storage is accessible from Windows, Linux, and macOS compute instances and devices running on AWS or on premises. You can optimize cost and performance with SSD and HDD storage options. Amazon FSx helps you reduce TCO with data deduplication.

Use Cases In this white , we explore use cases where fully managed cloud file systems excel:

• Migrating and running managed Windows File Shares (SMB)

• Running (NFS) file shares on AWS Use Case 1: Moving to fully managed SMB file shares with AWS

Figure 1 shows a very typical enterprise with on-premises NAS file storage with access to/from multiple geographical locations.

3 Amazon Web Services Moving to Managed File Systems

Figure 1 - How users access data on NAS or traditional file servers

File storage is widely used by organizations within their on-premises infrastructure. Windows file storage is used for general purpose file/print servers, home directories, media repositories and a wide range of other uses since it is easy for end-users to share files, regardless of where they are located. is often integrated with users’ native environments. AWS provides a fully managed file system service that reduces operational burdens for IT teams, helps customers to lower costs, and improve agility, which leads to better and faster business insights. Using FSx for Windows File Server, customers can improve file storage availability with the use of multiple Availability Zones (AZs), which eliminates the need to deal with the complexity or cost of setting up multi-site clustering and replication. With the data in AWS, customers can innovate faster by taking advantage of integrations with other AWS services. Amazon FSx integrates with AWS hosted directory services, such as AWS Managed Microsoft AD.

Using AWS Best Practices for Windows File Services can help organizations with the following activities:

• Consolidating file shares with DFS and FSx

• Extending Enterprise AD to FSx for Windows File Server

4 Amazon Web Services Moving to Managed File Systems

• Using Shadow copies to effectively protect data on FSx Figure 2 shows multiple ways to access a centralized data store in the cloud that is fully managed and secured by AWS. For SMB file shares, customers can deploy Amazon FSx for Windows File Server.

Figure 2 - How global users access data stored in AWS cloud file services Use Case 2: Moving to EFS Hosted Home Directories for Users and Applications

Many business-critical enterprise applications depend on NFS file systems, other scientific and engineering workloads like CAD and Bio-Informatics also rely on this type of shared storage. Amazon EFS provides a fully managed file system service, which means that customers no longer need to worry about procuring, managing, or supporting storage system hardware or software. There is no need to plan for infrastructure obsolescence and data migrations resulting from hardware refreshes also go away.

AWS offers multiple fully managed cloud file services to meet the varying needs of different workloads. The following options enable customers to unlock the true and full potential of their data:

5 Amazon Web Services Moving to Managed File Systems

• Home directories: A common use case for shared file services. Customers with learning management systems turn to Amazon EFS to store data that supports their education platform. They are able to leverage the performance of Amazon EFS with little modification to existing processes or permissions. Customers no longer run into capacity, performance, and reliability issues as opposed to on- premises storage.

• Application Storage: Many applications depend on file-based storage for unstructured data, including pictures, pdfs, video, and even binary frameworks. Developers are now able to define infrastructure as code, and engineers are now able to reproduce environments in minutes and iteratively improve software components in a cost-effective way.

Amazon EFS will respect the UIDs/GIDs on Linux hosts that are integrated with enterprise identity and directory services, such as LDAP. Additionally, Amazon EFS integrates with AWS hosted identity services, such as AWS Identity and Access Management (IAM). Apart from reducing management overhead, both services provide different storage options for cost optimization, enabling enterprises to scale capacity and performance independently as needed. Amazon EFS provides a fully managed file system matched to the specific application needs. No proprietary knowledge of the inner workings of the underlying storage infrastructure, nor specifics around the access protocol are needed. Customers are able to quickly deploy file systems based on capacity and performance requirements, eliminating operational burden.

Moving your data to the Cloud using Online Data Transfer AWS DataSync makes it simple and fast to move large amounts of data online between on-premises storage and Amazon EFS, or Amazon FSx for Windows File Server.

AWS DataSync eliminates or automatically handles many manual administrative tasks including scripting copy jobs, scheduling and monitoring transfers, validating data, and optimizing network utilization. DataSync uses a purpose-built protocol to transfer hundreds of terabytes and millions of files at speeds up to 10 times faster than open- source tools, over the internet or AWS Direct Connect links.

6 Amazon Web Services Moving to Managed File Systems

DataSync can be used to migrate active data sets or archives to AWS, transfer data to the cloud for timely analysis and processing, or replicate data to AWS for business continuity.

To learn how to transfer files from an existing on-premises file system to file system, see Getting Started with AWS DataSync in the AWS DataSync User Guide.

Data can also be transferred in a more traditional manner by leveraging an existing connection to AWS and combining that with specific or open-source tools. For example, customers could use rsync or fpsync for NFS shares, and RoboCopy for SMB shares.

Moving your data to the Cloud using Offline Data Transfer to AWS The AWS Snow Family helps customers that need to run operations in austere, non- data center environments, and in locations where there's lack of consistent network connectivity. The Snow Family, comprised of AWS Snowcone, AWS Snowball, and AWS Snowmobile, offers a number of physical devices and capacity points, most with built-in computing capabilities. These services help physically transport up to exabytes of data into and out of AWS.

• AWS Snowcone is the smallest member of the AWS Snow Family of edge computing, edge storage, and data transfer devices, weighing in at 4.5 pounds (2.1 kg) with 8 terabytes of usable storage. You can execute compute applications at the edge, and you can ship the device with data to AWS for offline data transfer, or you can transfer data online with AWS DataSync from edge locations.

• AWS Snowball Edge is a petabyte-scale edge computing, data migration, and edge storage device designed for local storage, large scale-data transfer, advanced machine learning, and full motion video analysis in disconnected environments.

• AWS Snowmobile moves up to 100 PB of data in a 45-foot long ruggedized shipping container and is ideal for multi-petabyte or Exabyte-scale digital media migrations and data center closures.

7 Amazon Web Services Moving to Managed File Systems

Conclusion Regardless of the workload that a customer wishes to run; either traditional on-premises workloads or modern application deployments, many applications require access to shared file storage.

AWS offers several simple, scalable, and fully managed cloud-based file services to meet the needs of varying workloads. By moving workloads to AWS, customers can accelerate digital transformation projects, improve application performance, and reduce cost. Additionally, by leveraging the multi-AZ capabilities of AWS, customers can often improve the reliability of their storage architecture, without the costs associated with traditional multi-site on-premises storage system deployments.

Customers no longer have to set up and maintain multiple datacenters, license software to integrate the two sites, maintain the network connections between the sites, and operate and manage the infrastructure at each site. In doing so, AWS can save customers significant capital and operational expenses.

Depending on whether a customer is moving from an on-premises storage system to a fully managed cloud file system, or whether a customer is implementing a modern application deployment model for the first time in the cloud.

Customers can create file systems on Amazon EFS and Amazon FSx in a single AZ, or across multiple AZs, using the AWS Management Console, AWS CLI, or AWS SDK. File systems can scale storage capacity and scale throughput capacity at any time, making it easy to respond to changing business needs.

To help customers accelerate moving on-premises data into the AWS cloud, AWS offers several different ways of getting data into the AWS cloud. Alternatively, customers can also leverage the AWS Migration Acceleration Program for Storage. MAP for Storage provides prescriptive guidance, repeatable methodology, tools, expertise, and incentives to accelerate storage migration to the cloud. As part of the AWS Migration Services portfolio, MAP for Storage can offer AWS Professional Services engagements, tools to understand your data and workflows, and even credits to offset some of the costs of migration.

8 Amazon Web Services Moving to Managed File Systems

Contributors Contributors to this document include:

• Scott Franks, Storage Specialist Solutions Architect, AWS • Ju-Lien Lim, Sr Specialist Solutions Architect, Storage

Further Reading For additional information, see:

• Shared File Storage on AWS

Document Revisions

Date Description September 2020 First publication

Notes

1 TechRepublic

9