Azure.Storage.DataMovement.Files.Shares 12.0.0

Azure Storage Data Movement File Shares client library for .NET

Azure Storage is a Microsoft-managed service providing cloud storage that is highly available, secure, durable, scalable, and redundant. Azure Storage includes Azure Blobs (objects), Azure Data Lake Storage Gen2, Azure Files, and Azure Queues.

The Azure Storage Data Movement library is optimized for uploading, downloading and copying customer data.

The Azure.Storage.DataMovement.Files.Shares library provides infrastructure shared by the other Azure Storage client libraries.

Source code | Package (NuGet) | API reference documentation | REST API documentation | Product documentation

Getting started

Install the package

Install the Azure Storage client library for .NET you'd like to use with NuGet and the Azure.Storage.DataMovement.Files.Shares client library will be included:

dotnet add package Azure.Storage.DataMovement
dotnet add package Azure.Storage.DataMovement.Files.Shares

Prerequisites

You need an Azure subscription and a Storage Account to use this package.

To create a new Storage Account, you can use the Azure Portal, Azure PowerShell, or the Azure CLI. Here's an example using the Azure CLI:

az storage account create --name MyStorageAccount --resource-group MyResourceGroup --location westus --sku Standard_LRS

Authenticate the client

The Azure.Storage.DataMovement.Files.Shares library uses clients from the Azure.Storage.Files.Shares package to communicate with the Azure File Storage service. For more information see the Azure.Storage.Files.Shares authentication documentation.

Permissions

The authenticated share storage resource needs the following permissions to perform a transfer:

1. Read
2. List (for directory transfers)
3. Write
4. Delete (for cleanup of a failed transfer item)
5. Create

Key concepts

The Azure Storage Common client library contains shared infrastructure like [authentication credentials][auth_credentials] and RequestFailedException.

Thread safety

We guarantee that all client instance methods are thread-safe and independent of each other (guideline). This ensures that the recommendation of reusing client instances is always safe, even across threads.

Additional concepts

Client options | Accessing the response | Long-running operations | Handling failures | Diagnostics | Mocking | Client lifetime

Examples

This section demonstrates usage of Data Movement for interacting with file shares.

Initializing File Share StorageResource

Azure.Storage.DataMovement.Files.Shares exposes ShareFilesStorageResourceProvider to create StorageResource instances for files and directories. The resource provider should be initialized with a credential to properly authenticate the storage resources. The following demonstrates this using an Azure.Core token credential.

ShareFilesStorageResourceProvider shares = new(tokenCredential);

To create a share StorageResource, use the methods FromFile or FromDirectory.

StorageResource directory = await shares.FromDirectoryAsync(
    new Uri("http://myaccount.files.core.windows.net/share/path/to/directory"));
StorageResource rootDirectory = await shares.FromDirectoryAsync(
    new Uri("http://myaccount.files.core.windows.net/share"));
StorageResource file = await shares.FromFileAsync(
    new Uri("http://myaccount.files.core.windows.net/share/path/to/file.txt"));

Storage resources can also be initialized with the appropriate client object from Azure.Storage.Files.Shares. Since these resources will use the credential already present in the client object, no credential is required in the provider when using FromClient(). However, a ShareFilesStorageResourceProvider must still have a credential if it is to be used in TransferManagerOptions for resuming a transfer.

StorageResource shareDirectoryResource = ShareFilesStorageResourceProvider.FromClient(directoryClient);
StorageResource shareFileResource = ShareFilesStorageResourceProvider.FromClient(fileClient);

Upload

An upload takes place between a local file StorageResource as source and file share StorageResource as destination.

Upload a file.

TokenCredential tokenCredential = new DefaultAzureCredential();
ShareFilesStorageResourceProvider shares = new(tokenCredential);
TransferManager transferManager = new TransferManager(new TransferManagerOptions());
TransferOperation fileTransfer = await transferManager.StartTransferAsync(
    sourceResource: LocalFilesStorageResourceProvider.FromFile(sourceLocalFile),
    destinationResource: await shares.FromFileAsync(destinationFileUri));
await fileTransfer.WaitForCompletionAsync();

Upload a directory.

TransferOperation folderTransfer = await transferManager.StartTransferAsync(
    sourceResource: LocalFilesStorageResourceProvider.FromDirectory(sourceLocalDirectory),
    destinationResource: await shares.FromDirectoryAsync(destinationFolderUri));
await folderTransfer.WaitForCompletionAsync();

Download

A download takes place between a file share StorageResource as source and local file StorageResource as destination.

Download a file.

TransferOperation fileTransfer = await transferManager.StartTransferAsync(
    sourceResource: await shares.FromFileAsync(sourceFileUri),
    destinationResource: LocalFilesStorageResourceProvider.FromFile(destinationLocalFile));
await fileTransfer.WaitForCompletionAsync();

Download a Directory.

TransferOperation directoryTransfer = await transferManager.StartTransferAsync(
    sourceResource: await shares.FromDirectoryAsync(sourceDirectoryUri),
    destinationResource: LocalFilesStorageResourceProvider.FromDirectory(destinationLocalDirectory));
await directoryTransfer.WaitForCompletionAsync();

File Copy

A copy takes place between two share StorageResource instances. Copying between two files or directories uses PUT from URL REST APIs, which do not transfer data through the machine running DataMovement.

Copy a single file.

TransferOperation fileTransfer = await transferManager.StartTransferAsync(
    sourceResource: await shares.FromFileAsync(sourceFileUri),
    destinationResource: await shares.FromFileAsync(destinationFileUri));
await fileTransfer.WaitForCompletionAsync();

Copy a directory.

TransferOperation directoryTransfer = await transferManager.StartTransferAsync(
    sourceResource: await shares.FromDirectoryAsync(sourceDirectoryUri),
    destinationResource: await shares.FromDirectoryAsync(destinationDirectoryUri));
await directoryTransfer.WaitForCompletionAsync();

Resume using ShareFilesStorageResourceProvider

To resume a transfer with Share File(s), valid credentials must be provided. See the sample below.

TokenCredential tokenCredential = new DefaultAzureCredential();
ShareFilesStorageResourceProvider shares = new(tokenCredential);
TransferManager transferManager = new TransferManager(new TransferManagerOptions()
{
    ProvidersForResuming = new List<StorageResourceProvider>() { shares }
});
// Get resumable transfers from transfer manager
await foreach (TransferProperties properties in transferManager.GetResumableTransfersAsync())
{
    // Resume the transfer
    if (properties.SourceUri.AbsoluteUri == "https://storageaccount.blob.core.windows.net/containername/blobpath")
    {
        await transferManager.ResumeTransferAsync(properties.TransferId);
    }
}

For more information regarding pause, resume, and/or checkpointing, see Pause and Resume Checkpointing.

Troubleshooting

See Handling Failed Transfers and Enabling Logging to assist with any troubleshooting.

See Known Issues for detailed information.

Next steps

Get started with our Blob DataMovement samples.

For more base Transfer Manager scenarios see DataMovement samples.

Contributing

See the Storage CONTRIBUTING.md for details on building, testing, and contributing to these libraries.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.