Migrating data between Azure file shares.
Find the closest match scenario below for guidance on the recommended way to migrate data between Azure file shares.
A. I want to migrate from a standard file share to a premium file share to get increased performance for my application workload.
We recommend mounting your existing share and new share on an IaaS Windows VM in the same region and then use Robocopy to move the data. See Migrate shares when not using Azure File Sync for more details.
B. All other migration scenarios where you want to move from one Azure file share to another.
If you are not using Azure File Sync, see Migrate between Azure file shares using Robocopy without Azure File Sync
If you are using Azure File Sync, chose one of the following methods: Azure File Sync Share Migration when cloud tiering is off. Azure File Sync Share migration when cloud tiering is on.
Migrate shares when not using Azure File Sync Deploy a Windows VM in Azure IaaS to keep the network traffic in Azure. Keeping the data in Azure will be fast and avoid outbound data transfer charges. Run this VM in the same region as your source share. To ensure your migration performs well, we recommend a multi-core machine with at least 56GiB of memory. Something like Standard_DS5_v2
Mount both file shares to the VM. Mount using storage account key to make sure the machine has access to all the files.
Run this command: (Robocopy is a tool built into Windows.) Robocopy
If your source share was mounted as s:\ and target was t:\ the command looks like this: Robocopy s:\ t:\ /mir /copyall /mt:16 /DCOPY:DAT
You can run the command while your source is still online but be aware that any IO would work against your throttle limits on your existing share.
After the initial run completes, you should disconnect your application from the existing share and run the same robocopy command again. This will copy over all the changes that happened since the initial run, skipping anything that is already copied over.
After the command completes for the second time, you can redirect your application to the new share.
Azure File Sync Share Migration when cloud tiering is off
These instructions can only be followed when cloud tiering is not enabled and all the files are fully synced to the on-prem server. If you have tiered a small amount of data (<1TB), you might want to turn cloud tiering off and trigger the files to sync back down using Invoke-StorageSyncFileRecall cmdlet so you can follow these instructions. They are simpler than the cloud tiering instructions, but you would have to turn off tiering and run a command to recall all the data to follow them.
If you are not using cloud tiering, then all your data is on your Azure File Sync server, and you can use Azure File Sync to upload the data to another share.
The following instructions assume you have one Azure File Sync server in your sync group. If you have more than one Azure File Sync server connected the existing share, you should remove all the other server endpoints first. Perform the full migration on one endpoint, then you can reconnect the other server endpoints to the new sync group. Be sure to follow the instructions for removing server endpoints.
Initial Step:
1. Ensure tiering is OFF on the server endpoint. You can check and change the status from the Azure portal under the server endpoint properties. 2. Run Invoke-StorageSyncFileRecall cmdlet with retries to ensure data is now local on server. Because there could have been an active cloud tiering session when you first ran this cmdlet, it is a good idea to run it twice to ensure everything is fully recalled.
3. Create new Azure file share
4. Create a new sync group and associate the cloud endpoint to the Azure file share created in step 3. The sync group must be in a storage sync service in the same region as the new target Azure file share.
Option 1 - Connect your data to the new Azure File Share using the same local file server (recommended):
Remove the existing sever endpoint. This will keep all the data and just removes the association with the existing sync group and existing file share.
If your new sync group is not in the same storage sync service, you first need to unregister the server from that storage sync service and register it with the new service. However, keep in mind that a server can only be registered with on storage sync service.
Add a new server endpoint in the sync group you created in step #4 and connect it to the same local data.
Option 2 – If you want to move to a new server. Move to new Azure File Sync server using Storage Migration Service:
Setup a new on-prem file server and connect the new server to AFS and the new cloud endpoint. Using a new server allows you to use Storage Migration Service (SMS) which will copy over all your share level permissions, make several passes to catch up with changes that happened during migration and orchestrate the failover. Optionally you can manually copy the source share to another share on the existing file server.
Next use Storage Migration Service to migrate from the source server to target server. SMS will also cutover to the new server for you as well as copy over all file and share permissions.
Azure File Sync Share migration when cloud tiering is on
If you are using Azure File Sync’s cloud tiering feature, we recommend copying the data from within Azure to prevent unnecessary cloud recalls through the source.
Setup
Deploy a Windows VM in Azure IaaS in the same region as the source share.
To ensure good performance we recommend a multi-core VM with at least 56GiB of memory and premium storage. Something like Standard_DS5_v2.
You will need to create a new sync group attached to your target share. Depending on if you are migrating cross region or not, you may need to create a new storage sync service. The sync group needs to be in the same region as the file share and sync service.
An Azure File Sync registered server can only join one storage sync service and the storage sync service needs to be in the same region as the share. So, if you are moving between regions, you will need to migrate to a new Azure File Sync server connected to the target share. If you are moving within the same region you can use the existing AFS server.
Cross Region:
• Create a new storage sync service in the target region and a new sync group there. • Create a new on-prem AFS file server. Do not connect your new target server to the sync group yet. • For Source AFS Azure VM, use a small disk as your source data connected to the existing sync group. Connect AFS to the original sync group. You can enable cloud tiering on this sever endpoint. You will pull data out of this as your source. • For your Target AFS Azure VM use one larger disk that can hold your entire data set for your target. Also mount a drive to the source share on the source AFS VM. • Setup the target file share as the cloud endpoint of your a new sync group. Connect the IaaS VM as a server endpoint for the target share and set the on prem path to T:\target.
Within the same region:
• Create your new target sync group in the same storage sync service • Re-use your existing VM – If you are concerned about impacting the existing share, you can optionally create a new server instead of re-using your existing VM. • Do not connect your local server to the new sync group yet.
In your IaaS VM • Use different disks for local storage of source and target. One small disk as your source data connected to the existing sync group and one larger disk that can hold your entire data set for your target. • Connect AFS to the original sync group. You can enable cloud tiering on this sever endpoint. You will pull data out of this as your source. • Setup the target file share as the cloud endpoint of your a new sync group. Connect the IaaS VM as a server endpoint for the target share and set the on prem path to T:\target.
Initial Copy
Run this command: Robocopy
If your source share was mounted as s:\ and target was t:\ the command looks like this: Robocopy s:\ t:\ /mir /copyall /mt:16 /DCOPY:DAT
You can run the command while your source is still online but be aware that any IO would work against your throttle limits on your shares. So, you may want to do this during off-peak hours.
Connect the target on-prem AFS server to the new sync group so the namespace metadata sync can progress there. Be sure to use the same root folder name as the existing share. For example, if your current cache location is D:\cache, use T:\cache as the cache for the new server endpoint. If you are using the existing AFS server (for migrations within the same region), place the local cache on a separate volume from the existing endpoint. Using the same volume is okay as long as the directory is not the same or sub-directory of the server endpoint connected with the source share. The endpoint should be enabled with cloud tiering so that none of the data will automatically download to the on-prem server.
Wait for the initial Robocopy run to complete successfully and for the sync from the IaaS VM to the Azure File Sync target to complete. You will need to wait for 1 hour to allow file sync to sync the remaining changes to Azure. To check that sync has synced all changes to the cloud follow the instructions here.
1. Connect the on-prem server to the new sync group.
Configure your new target with a high free space policy at first because we will be copying in the latest changes and need to ensure we have enough room.
Once the server is connected to the sync group, allow some time for it to sync the namespace data.
See documentation on how to verify sync status.
2. Sync final changes 2.1 Turn off SMB sharing for the existing share or at least make it read-only. 2.2 If you have connectivity between the source file share and the target, the fastest thing to do is Robocopy recent changes over to the target.
Robocopy S:\ T:\ /mir /copyall /mt:16 /DCOPY:DAT /XD S:\$RECYCLE.BIN /XD "S:\System Volume Information"
If you cannot copy the latest changes directly to the new file share, you can follow these steps: 2.2.1 After turning off SMB sharing, you will need to wait for 1 hour to allow file sync to sync the remaining changes to Azure. To check that sync has synced all changes to the cloud follow the instructions here. 2.2.2 Run the Robocopy mirror command again on the IaaS VM.
This will sync over all the changes that happened since the initial run, skipping anything that is already copied over. Robocopy s:\ t:\target /mir /copyall /mt:16 /DCOPY:DAT 2.2.3 Once your IaaS VM sync completes, the local target agent will also be up-to- date.
3. Enable sharing on the new server endpoint
If you were migrating to a new AFS server, you should rename the old server to a random name, then rename the new server to the same name as the old server. This way, the file share URL will be the same for your end users.
Enable the new share “T:\cache”. All the same file ACLs will be there.
You will need to recreate any file share level permissions that existed on the old share.
4. Remove the old server endpoint and sync group.
Once you have confirmed everything is working correctly with the new sync group, you can deprovision the old sync group. Starting with removing the server endpoints. You do not need to recall all the data to the old server before removing the server endpoint. https://docs.microsoft.com/en-us/azure/storage/files/storage-sync-files-server- endpoint#remove-a-server-endpoint