Azure.Storage.Blobs.ChangeFeed 12.0.0-preview.48

Prefix Reserved
This is a prerelease version of Azure.Storage.Blobs.ChangeFeed.
There is a newer prerelease version of this package available.
See the version list below for details.
dotnet add package Azure.Storage.Blobs.ChangeFeed --version 12.0.0-preview.48                
NuGet\Install-Package Azure.Storage.Blobs.ChangeFeed -Version 12.0.0-preview.48                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Azure.Storage.Blobs.ChangeFeed" Version="12.0.0-preview.48" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Azure.Storage.Blobs.ChangeFeed --version 12.0.0-preview.48                
#r "nuget: Azure.Storage.Blobs.ChangeFeed, 12.0.0-preview.48"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install Azure.Storage.Blobs.ChangeFeed as a Cake Addin
#addin nuget:?package=Azure.Storage.Blobs.ChangeFeed&version=12.0.0-preview.48&prerelease

// Install Azure.Storage.Blobs.ChangeFeed as a Cake Tool
#tool nuget:?package=Azure.Storage.Blobs.ChangeFeed&version=12.0.0-preview.48&prerelease                

Azure Storage Blobs Change Feed client library for .NET

Server Version: 2021-02-12, 2020-12-06, 2020-10-02, 2020-08-04, 2020-06-12 2020-04-08, 2020-02-10, and 2019-12-12

The purpose of the change feed is to provide transaction logs of all the changes that occur to the blobs and the blob metadata in your storage account. The change feed provides ordered, guaranteed, durable, immutable, read-only log of these changes. Client applications can read these logs at any time. The change feed enables you to build efficient and scalable solutions that process change events that occur in your Blob Storage account at a low cost.

Source code | Package (NuGet) | Product documentation

Getting started

Install the package

Install the Azure Storage Blobs client library for .NET with NuGet:

dotnet add package Azure.Storage.Blobs.ChangeFeed --prerelease

Prerequisites

You need an Azure subscription and a Storage Account to use this package.

To create a new Storage Account, you can use the Azure Portal, Azure PowerShell, or the Azure CLI. Here's an example using the Azure CLI:

az storage account create --name MyStorageAccount --resource-group MyResourceGroup --location westus --sku Standard_LRS

Authenticate the Client

Authentication works the same as in Azure.Storage.Blobs.

Key concepts

The change feed is stored as blobs in a special container in your storage account at standard blob pricing cost. You can control the retention period of these files based on your requirements (See the conditions of the current release). Change events are appended to the change feed as records in the Apache Avro format specification: a compact, fast, binary format that provides rich data structures with inline schema. This format is widely used in the Hadoop ecosystem, Stream Analytics, and Azure Data Factory.

You can process these logs incrementally or in-full. Any number of client applications can independently read the change feed, in parallel, and at their own pace. Analytics applications such as Apache Drill or Apache Spark can consume logs directly as Avro files, which let you process them at a low-cost, with high-bandwidth, and without having to write a custom application.

Thread safety

We guarantee that all client instance methods are thread-safe and independent of each other (guideline). This ensures that the recommendation of reusing client instances is always safe, even across threads.

Additional concepts

Client options | Accessing the response | Long-running operations | Handling failures | Diagnostics | Mocking | Client lifetime

Examples

Get all events in the Change Feed

// Get all the events in the change feed.
List<BlobChangeFeedEvent> changeFeedEvents = new List<BlobChangeFeedEvent>();
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync())
{
    changeFeedEvents.Add(changeFeedEvent);
}

Get events between a start and end time

// Create the start and end time.  The change feed client will round start time down to
// the nearest hour, and round endTime up to the next hour if you provide DateTimeOffsets
// with minutes and seconds.
DateTimeOffset startTime = new DateTimeOffset(2017, 3, 2, 15, 0, 0, TimeSpan.Zero);
DateTimeOffset endTime = new DateTimeOffset(2020, 10, 7, 2, 0, 0, TimeSpan.Zero);

// You can also provide just a start or end time.
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync(
    start: startTime,
    end: endTime))
{
    changeFeedEvents.Add(changeFeedEvent);
}

Resume with continuationToken

string continuationToken = null;
await foreach (Page<BlobChangeFeedEvent> page in changeFeedClient.GetChangesAsync().AsPages(pageSizeHint: 10))
{
    foreach (BlobChangeFeedEvent changeFeedEvent in page.Values)
    {
        changeFeedEvents.Add(changeFeedEvent);
    }

    // Get the change feed continuation token.  The continuation token is not required to get each page of events,
    // it is intended to be saved and used to resume iterating at a later date.
    continuationToken = page.ContinuationToken;
    break;
}

// Resume iterating from the pervious position with the continuation token.
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync(
    continuationToken: continuationToken))
{
    changeFeedEvents.Add(changeFeedEvent);
}

Troubleshooting

All Blob service operations will throw a RequestFailedException on failure with helpful ErrorCodes. Many of these errors are recoverable.

Next steps

Get started with our Change Feed samples:

  1. Hello World: Get changes that have occurred in your storage account (or asynchronously)
  2. Auth: Authenticate with connection strings, public access, shared keys, shared access signatures, and Azure Active Directory.

Contributing

See the Storage CONTRIBUTING.md for details on building, testing, and contributing to this library.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Product Compatible and additional computed target framework versions.
.NET net5.0 was computed.  net5.0-windows was computed.  net6.0 is compatible.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
.NET Core netcoreapp2.0 was computed.  netcoreapp2.1 was computed.  netcoreapp2.2 was computed.  netcoreapp3.0 was computed.  netcoreapp3.1 was computed. 
.NET Standard netstandard2.0 is compatible.  netstandard2.1 was computed. 
.NET Framework net461 was computed.  net462 was computed.  net463 was computed.  net47 was computed.  net471 was computed.  net472 was computed.  net48 was computed.  net481 was computed. 
MonoAndroid monoandroid was computed. 
MonoMac monomac was computed. 
MonoTouch monotouch was computed. 
Tizen tizen40 was computed.  tizen60 was computed. 
Xamarin.iOS xamarinios was computed. 
Xamarin.Mac xamarinmac was computed. 
Xamarin.TVOS xamarintvos was computed. 
Xamarin.WatchOS xamarinwatchos was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
12.0.0-preview.49 462 9/19/2024
12.0.0-preview.48 1,322 8/7/2024
12.0.0-preview.47 352 7/25/2024
12.0.0-preview.46 497 7/16/2024
12.0.0-preview.45 1,574 6/11/2024
12.0.0-preview.44 1,681 5/14/2024
12.0.0-preview.43 975 4/16/2024
12.0.0-preview.42 12,799 12/5/2023
12.0.0-preview.41 1,379 11/14/2023
12.0.0-preview.40 1,346 11/6/2023
12.0.0-preview.39 1,212 10/16/2023
12.0.0-preview.38 415 9/12/2023
12.0.0-preview.37 1,862 8/8/2023
12.0.0-preview.36 689 7/11/2023
12.0.0-preview.35 2,166 5/30/2023
12.0.0-preview.34 23,585 4/11/2023
12.0.0-preview.33 189 3/28/2023
12.0.0-preview.32 309 3/24/2023
12.0.0-preview.31 1,166 2/22/2023
12.0.0-preview.30 191 2/8/2023
12.0.0-preview.29 6,120 10/12/2022
12.0.0-preview.28 946 8/23/2022
12.0.0-preview.27 475 7/8/2022
12.0.0-preview.26 3,685 6/15/2022
12.0.0-preview.25 7,285 5/2/2022
12.0.0-preview.24 2,137 4/20/2022
12.0.0-preview.23 1,518 4/13/2022
12.0.0-preview.22 150 4/12/2022
12.0.0-preview.21 475 3/30/2022
12.0.0-preview.20 12,091 3/10/2022
12.0.0-preview.19 243 2/7/2022
12.0.0-preview.18 496 11/30/2021
12.0.0-preview.17 334 11/4/2021
12.0.0-preview.16 470 9/9/2021
12.0.0-preview.15 255 7/23/2021
12.0.0-preview.14 185 7/23/2021
12.0.0-preview.13 394 6/9/2021
12.0.0-preview.12 272 5/12/2021
12.0.0-preview.11 3,468 4/9/2021
12.0.0-preview.10 880 3/10/2021
12.0.0-preview.9 279 2/10/2021
12.0.0-preview.8 91,830 1/12/2021
12.0.0-preview.7 346 12/7/2020
12.0.0-preview.6 1,530 11/10/2020
12.0.0-preview.5 2,111 10/1/2020
12.0.0-preview.4 3,253 8/18/2020
12.0.0-preview.2 292 7/28/2020
12.0.0-preview.1 417 7/3/2020