AvroConvert 3.2.6
Install-Package AvroConvert -Version 3.2.6
dotnet add package AvroConvert --version 3.2.6
<PackageReference Include="AvroConvert" Version="3.2.6" />
paket add AvroConvert --version 3.2.6
#r "nuget: AvroConvert, 3.2.6"
// Install AvroConvert as a Cake Addin
#addin nuget:?package=AvroConvert&version=3.2.6
// Install AvroConvert as a Cake Tool
#tool nuget:?package=AvroConvert&version=3.2.6
Avro format combines readability of JSON and compression of binary data serialization.
Benefits
The main purpose of the project was to enhance HTTP communication between microservices. Replacing JSON with Avro brought three main benefits:
- Decreased the communication time between microservices
- Reduced the network traffic by about 30%
- Increased communication security - the data was not visible in plain JSON text
Features
AvroConvert | Apache.Avro | Newtonsoft.Json | |
---|---|---|---|
Rapid serialization | ✔️ | ✔️ | ✔️ |
Easy to use | ✔️ | ❌ | ✔️ |
Built-in compression | ✔️ | ✔️ | ❌ |
Low memory allocation | ✔️ | ✔️ | ✔️ |
Support for C# data structures: Dictionary, List, DateTime... | ✔️ | ❌ | ✔️ |
Support for compression codecs | Deflate<br/> Snappy<br/> GZip<br/> Brotli | Deflate | ❌ |
Readable schema of data structure | ✔️ | ✔️ | ✔️ |
Data is encrypted | ✔️ | ✔️ | ❌ |
Benchmark
Results of BenchmarkDotNet:
Converter | Request Time [ms] | Allocated Memory [MB] | Compressed Size [kB] |
---|---|---|---|
Json | 672.3 | 52.23 | 6044 |
Avro | 384.7 | 76.58 | 2623 |
Json_Gzip | 264.1 | 88.32 | 514 |
Avro_Gzip | 181.2 | 75.05 | 104 |
Json_Brotli | 222.5 | 86.15 | 31 |
Avro_Brotli | 193.5 | 74.75 | 31 |
Article describing Avro format specification and benchmark methodology: https://www.c-sharpcorner.com/blogs/avro-rest-api-as-the-evolution-of-json-based-communication-between-mic
Conclusion: <br> Using Avro for communication between your services significantly reduces communication time and network traffic. Additionally choosing encoding (compression algorithm) can improve the results even further.
Code samples
- Serialization
byte[] avroObject = AvroConvert.Serialize(object yourObject);
<br/>
- Deserialization
CustomClass deserializedObject = AvroConvert.Deserialize<CustomClass>(byte[] avroObject);
<br/>
- Read schema from Avro object
string schemaInJsonFormat = AvroConvert.GetSchema(byte[] avroObject)
<br/>
- Deserialization of large collection of Avro objects one by one
using (var reader = AvroConvert.OpenDeserializer<CustomClass>(new MemoryStream(avroObject)))
{
while (reader.HasNext())
{
var item = reader.ReadNext();
// process item
}
}
- Generation of C# models from Avro file or schema
string resultModel = AvroConvert.GenerateModel(_avroBytes);
Support
If you find my work useful, you can support it 😃
<p align="center"> <a href="https://www.buymeacoffee.com/adrianstrugala" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 31px !important;width: 130px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a> </p>
Related packages
- Library containing functionalities, which enable communication between microservices via Http using Avro data format
- Library containing extensions for Confluent Kafka platform - used together with Kafka Consumer and Producer
License
The project is CC BY-NC-SA 3.0 licensed.
For commercial purposes purchase AvroConvert on website - Xabe.net
Contribution
We want to improve AvroConvert as much as possible. If you have any idea, found next possible feature, optimization opportunity or better way for integration, leave a comment or pull request.
Product | Versions |
---|---|
.NET | net5.0 net5.0-windows net6.0 net6.0-android net6.0-ios net6.0-maccatalyst net6.0-macos net6.0-tvos net6.0-windows |
.NET Core | netcoreapp2.0 netcoreapp2.1 netcoreapp2.2 netcoreapp3.0 netcoreapp3.1 |
.NET Standard | netstandard2.0 netstandard2.1 |
.NET Framework | net461 net462 net463 net47 net471 net472 net48 |
MonoAndroid | monoandroid |
MonoMac | monomac |
MonoTouch | monotouch |
Tizen | tizen40 tizen60 |
Xamarin.iOS | xamarinios |
Xamarin.Mac | xamarinmac |
Xamarin.TVOS | xamarintvos |
Xamarin.WatchOS | xamarinwatchos |
-
.NETStandard 2.0
- BrotliSharpLib (>= 0.3.3)
- FastMember (>= 1.5.0)
- IronSnappy (>= 1.3.0)
- Microsoft.CSharp (>= 4.7.0)
- Newtonsoft.Json (>= 12.0.3)
NuGet packages (5)
Showing the top 5 NuGet packages that depend on AvroConvert:
Package | Downloads |
---|---|
SolTechnology.Avro.Kafka
Library containing extensions for Confluent Kafka platform |
|
SolTechnology.Avro.Http
Library containing functionalities, which enable communication between microservices using Avro data format |
|
OptimoveSdk.Engager.Integration
SDK for Engagers intergration with Optimove |
|
SolTechnology.Core.ApiClient
SolTechnology Core API client library |
|
SolTechnology.Core.BlobStorage
SolTechnology Core Azure blob storage library |
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
3.2.6 | 136 | 5/16/2022 |
3.2.5 | 2,552 | 4/13/2022 |
3.2.4 | 112 | 4/10/2022 |
3.2.3 | 13,840 | 10/29/2021 |
3.2.2 | 16,251 | 10/2/2021 |
3.2.1 | 407 | 9/29/2021 |
3.2.0 | 1,293 | 9/14/2021 |
3.1.5 | 15,269 | 8/30/2021 |
3.1.4 | 2,064 | 8/12/2021 |
3.1.3 | 2,547 | 8/2/2021 |
3.1.2 | 3,715 | 6/24/2021 |
3.1.1 | 900 | 6/14/2021 |
3.1.0 | 5,085 | 5/15/2021 |
3.0.1 | 1,144 | 4/2/2021 |
3.0.0 | 21,509 | 3/26/2021 |
2.7.1 | 20,803 | 2/5/2021 |
2.7.0 | 33,245 | 12/18/2020 |
2.6.3 | 2,693 | 10/25/2020 |
2.6.2 | 260 | 10/23/2020 |
2.6.1 | 2,867 | 7/25/2020 |
2.6.0 | 330 | 7/22/2020 |
2.5.1 | 1,157 | 6/11/2020 |
2.5.0 | 1,619 | 5/15/2020 |
2.4.1 | 1,140 | 4/14/2020 |
2.4.0 | 632 | 3/25/2020 |
2.3.0 | 511 | 3/19/2020 |
2.2.2 | 756 | 2/7/2020 |
2.2.1 | 326 | 2/7/2020 |
2.2.0 | 408 | 1/27/2020 |
2.1.0 | 322 | 1/16/2020 |
2.0.0 | 316 | 1/4/2020 |
1.8.1 | 606 | 12/9/2019 |
1.8.0 | 1,677 | 8/30/2019 |
1.7.0 | 381 | 8/22/2019 |
1.6.0 | 397 | 7/22/2019 |
1.5.1 | 424 | 7/14/2019 |
1.5.0 | 478 | 7/11/2019 |
1.4.0 | 386 | 6/17/2019 |
1.3.0 | 415 | 6/4/2019 |
1.2.0 | 432 | 6/3/2019 |
1.1.0 | 410 | 5/28/2019 |
1.0.1 | 459 | 5/21/2019 |
1.0.0 | 703 | 4/24/2019 |
0.8.0 | 644 | 5/1/2019 |
0.7.0 | 729 | 4/22/2019 |
0.6.3 | 657 | 4/19/2019 |
0.6.2 | 675 | 4/19/2019 |
0.6.1 | 660 | 4/19/2019 |
0.6.0 | 664 | 4/19/2019 |
0.5.0 | 742 | 4/18/2019 |
0.3.0 | 866 | 4/11/2019 |
0.2.1 | 698 | 4/11/2019 |
0.2.0 | 705 | 4/11/2019 |
0.1.0 | 706 | 4/10/2019 |