Streamon.Azure.TableStorage 0.1.1

Streamon

Event streaming store platform for real-time data processing and analytics.

Providers

• In Memory
• Azure Table Storage
• Azure Cosmos DB (in progress, you can connect to CosmosDb using the Azure Table Storage provider above)

Features

• POCO events, no base classes or inheritance required
• Event ids and Metadata detection by using both Attribute and Interface markers
• Customizable serialization and type resolution
• Flexible stream sorage naming and partitioning, e.g. allowing for multitenancy by using one stream per tenant
• Optimistic concurrency control
• Soft and hard deletion modes
• Global event positioning & tracking
• Subscriptions
• Snapshots and Checkpoints

To Do's

• Telemetry
• Relational Stores (Entity Framework?)
• Claim Checks for large events
• Stream Projections
• Stream Sweeper (Archiving and Purging)

Azure Table Storage Provider Details

The Azure Table Storage provider is a simple implementation of the IStreamStore interface. It uses Azure Table Storage to store events in a single table. The table is partitioned by the stream id and the row key is the event id. The provider uses the Ulid library to generate unique identifiers for events.

Table Storage only support batches of up to 100 entities, trying to write more than that will result in an exception. The responsibility of handling this is left to the caller, as the provider does not implement any batching logic due to the fact that it can't guarantee the consistency of persistence across different batches.

Thanks

For inspiration and ideas, thanks to:

Streamstone Eveneum Eventflow and Eventuous

Icon:

Pipe icons created by srip - Flaticon

Dependencies

• CosmosDb Azure SDK CosmosDb provider
• Azure Storage SDK Azure Table Storage provider
• xUnit for unit testing
• Act for local testing of Github Actions
• Ulid for unique identifiers generation
• Test Containers for .NET for integration testing

Continuous Integration & Deployment

The project is built and tested using GitHub Actions. The build artifacts are published to GitHub Packages.

Github actions are configured to publish the packages to GitHub Packages on every push to the main branch.

Local testing and development of Github actions can be done using the act tool.

License

MIT License