SharpPulsar 2.9.0

.NET 5.0 .NET Standard 2.1
There is a newer prerelease version of this package available.
See the version list below for details. 
NuGet\Install-Package SharpPulsar -Version 2.9.0
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package. 
dotnet add package SharpPulsar --version 2.9.0
<PackageReference Include="SharpPulsar" Version="2.9.0" />
For projects that support PackageReference, copy this XML node into the project file to reference the package. 
paket add SharpPulsar --version 2.9.0
#r "nuget: SharpPulsar, 2.9.0"
#r directive can be used in F# Interactive, C# scripting and .NET Interactive. Copy this into the interactive tool or source code of the script to reference the package. 
// Install SharpPulsar as a Cake Addin
#addin nuget:?package=SharpPulsar&version=2.9.0

// Install SharpPulsar as a Cake Tool
#tool nuget:?package=SharpPulsar&version=2.9.0

Build Tests

SharpPulsar

SharpPulsar is an Apache Pulsar Client built on top Akka.net, which can handle millions of Apache Pulsar Producers/Consumers (in theory).

What Is Akka.NET?

Akka.NET is a toolkit and runtime for building highly concurrent, distributed, and fault tolerant event-driven applications on .NET & Mono that is able to support up to 50 million msg/sec on a single machine, with small memory footprint and ~2.5 million actors(or Apache Pulsar Producers/Consumers) per GB of heap.

What Is Apache Pulsar?

Apache Pulsar is a cloud-native, distributed messaging and streaming platform that is able to support millions of topics while delivering high-throughput and low-latency performance.

Supported features

Client

  • TLS
  • Authentication (token, tls, OAuth2)
  • Multi-Hosts Service URL
  • Proxy
  • SNI Routing
  • Transactions
  • Subscription(Durable, Non-durable)

Producer

  • Exclusive Producer
  • Partitioned Topics
  • Batching
  • Compression (LZ4, ZLIB, ZSTD, SNAPPY)
  • Schema (Primitive, Avro, Json, KeyValue, AutoSchema)
  • User-defined properties
  • Key-based batcher
  • Delayed/Scheduled messages
  • Interceptors
  • Message Router (RoundRobin, ConsistentHashing, Broadcast, Random)
  • End-to-end Encryption
  • Chunking
  • Transactions

Consumer

  • User-defined properties
  • HasMessageAvailable
  • Subscription Type (Exclusive, Failover, Shared, Key_Shared)
  • Subscription Mode (Durable, Non-durable)
  • Interceptors
  • Ack (Ack Individual, Ack Commulative, Batch-Index Ack)
  • Ack Timeout
  • Negative Ack
  • Dead Letter Policy
  • End-to-end Encryption
  • SubscriptionInitialPosition
  • Partitioned Topics
  • Batching
  • Compression (LZ4, ZLIB, ZSTD, SNAPPY)
  • Schema (Primitive, Avro, Json, KeyValue, AutoSchema)
  • Compacted Topics
  • Multiple Topics
  • Regex Consumer
  • Broker Entry Metadata

Reader

  • User-defined properties
  • HasMessageAvailable
  • Schema (Primitive, Avro, Json, KeyValue, AutoSchema)
  • Seek (MessageID, Timestamp)
  • Multiple Topics
  • End-to-end Encryption
  • Interceptors

Extras

  • Pulsar SQL
  • Pulsar Admin REST API
  • Function REST API
  • EventSource(Reader/SQL)

Getting Started

Install the NuGet package SharpPulsar and follow the Tutorials.

//pulsar client settings builder
            var clientConfig = new PulsarClientConfigBuilder()
                .ServiceUrl("pulsar://localhost:6650");

            //pulsar actor system
            var pulsarSystem = PulsarSystem.GetInstance(clientConfig);

            var pulsarClient = pulsarSystem.NewClient();

            var consumer = pulsarClient.NewConsumer(new ConsumerConfigBuilder<sbyte[]>()
                .Topic(myTopic)
                .ForceTopicCreation(true)
                .SubscriptionName("myTopic-sub"));

            var producer = pulsarClient.NewProducer(new ProducerConfigBuilder<sbyte[]>()
                .Topic(myTopic));

            for (var i = 0; i < 10; i++)
            {
                var data = Encoding.UTF8.GetBytes($"tuts-{i}").ToSBytes();
                producer.NewMessage().Value(data).Send();
            }
			Thread.Sleep(TimeSpan.FromSeconds(5));
            for (var i = 0; i < 10; i++)
            {
                var message = (Message<sbyte[]>)consumer.Receive();
                consumer.Acknowledge(message);
                var res = Encoding.UTF8.GetString(message.Data.ToBytes());
                Console.WriteLine($"message '{res}' from topic: {message.TopicName}");
            }

Logical Types

Avro Logical Types are supported. Message object MUST implement ISpecificRecord

    AvroSchema<LogicalMessage> avroSchema = AvroSchema<LogicalMessage>.Of(ISchemaDefinition<LogicalMessage>.Builder().WithPojo(typeof(LogicalMessage)).WithJSR310ConversionEnabled(true).Build());

    public class LogicalMessage : ISpecificRecord
    {
        [LogicalType(LogicalTypeKind.Date)]
        public DateTime CreatedTime { get; set; }
		
        [LogicalType(LogicalTypeKind.TimestampMicrosecond)]
        public DateTime StampMicros { get; set; }

        [LogicalType(LogicalTypeKind.TimestampMillisecond)]
        public DateTime StampMillis { get; set; }
		
	[LogicalType(LogicalTypeKind.TimeMicrosecond)]
        public TimeSpan TimeMicros { get; set; }

        [LogicalType(LogicalTypeKind.TimeMillisecond)]
        public TimeSpan TimeMillis { get; set; }
        
        public AvroDecimal Size { get; set; }
		
        public string DayOfWeek { get; set; }

        [Ignore]
        public Avro.Schema Schema { get; set; }

        public object Get(int fieldPos)
        {
            switch (fieldPos)
            {
                case 0: return CreatedTime; 
	        case 1: return StampMicros;
                case 2: return StampMillis;
	        case 3: return TimeMicros;
                case 4: return TimeMillis;
                case 5: return Size;
                case 6: return DayOfWeek;
                default: throw new AvroRuntimeException("Bad index " + fieldPos + " in Get()");
            };
        }

        public void Put(int fieldPos, object fieldValue)
        {
            switch (fieldPos)
            {
                case 0: CreatedTime = (DateTime)fieldValue; break;
		case 1: StampMicros = (DateTime)fieldValue; break;
                case 2: StampMillis = (DateTime)fieldValue; break;
	        case 3: TimeMicros = (TimeSpan)fieldValue; break;
                case 4: TimeMillis = (TimeSpan)fieldValue; break;
                case 5: Size = (AvroDecimal)fieldValue; break;
                case 6: DayOfWeek = (String)fieldValue; break;
                default: throw new AvroRuntimeException("Bad index " + fieldPos + " in Put()");
            };
        }
    }

KeyValue Schema ALERT!!!!

Because I have become lazy and a lover of "peace of mind":

  • For schema type of KEYVALUESCHEMA:
    • csharp producer.NewMessage().Value<TK, TV>(data).Send();` or `csharp producer.Send<TK, TV>(data)

TK, TV represents the key and value types of the KEYVALUESCHEMA respectively.

Running SharpPulsar Tests in docker container

You can run SharpPulsar tests in docker container. A Dockerfile and docker-compose file is provided at the root folder to help you run these tests in a docker container. docker-compose.yml:

version: "2.4"

services:
  akka-test:
    image: sharp-pulsar-test
    build: 
      context: .
    cpu_count: 1
    mem_limit: 1g
    environment:
      run_count: 2
      # to filter tests, uncomment
      # test_filter: "--filter FullyQualifiedName=SharpPulsar.Test.MessageChunkingTest"
      test_file: Tests/SharpPulsar.Test/SharpPulsar.Test.csproj

Dockerfile:

FROM mcr.microsoft.com/dotnet/sdk:6.0 
ENV test_file="Tests/SharpPulsar.Test/SharpPulsar.Test.csproj"
ENV test_filter=""
ENV run_count=2
RUN mkdir sharppulsar
COPY . ./sharppulsar
RUN ls
WORKDIR /sharppulsar
CMD ["/bin/bash", "-c", "x=1; c=0; while [ $x -le 1 ] && [ $c -le ${run_count} ]; do dotnet test ${test_file} ${test_filter} --framework net6.0 --logger trx; c=$(( $c + 1 )); if [ $? -eq 0 ]; then x=1; else x=0; fi;  done"]

How to:

cd into the root directory and execute docker-compose up run-count is the number of times you want the test repeated. test_filter is used when you need to test a specific test instead of running all the tests in the test suite.

License

This project is licensed under the Apache License Version 2.0 - see the LICENSE file for details.

Product Versions
.NET net5.0 net5.0-windows net6.0 net6.0-android net6.0-ios net6.0-maccatalyst net6.0-macos net6.0-tvos net6.0-windows
.NET Core netcoreapp3.0 netcoreapp3.1
.NET Standard netstandard2.1
MonoAndroid monoandroid
MonoMac monomac
MonoTouch monotouch
Tizen tizen60
Xamarin.iOS xamarinios
Xamarin.Mac xamarinmac
Xamarin.TVOS xamarintvos
Xamarin.WatchOS xamarinwatchos
Compatible target framework(s)
Additional computed target framework(s)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
2.10.0-rc1162 47 9/14/2022
2.10.0-rc1158 57 8/9/2022
2.10.0-rc1157 44 8/9/2022
2.10.0-rc1150 41 8/8/2022
2.10.0-rc1136 87 7/23/2022
2.10.0-rc1135 57 7/23/2022
2.10.0-rc1134 58 7/23/2022
2.10.0-rc1133 62 7/23/2022
2.10.0-rc1132 50 7/21/2022
2.10.0-rc1130 54 7/21/2022
2.10.0-rc1129 76 7/17/2022
2.10.0-rc1125 84 7/10/2022
2.10.0-rc1123 67 7/9/2022
2.10.0-rc.1117 63 7/3/2022
2.10.0-rc.1108 97 6/25/2022
2.10.0-rc.1090 73 6/9/2022
2.10.0-rc.1088 58 6/9/2022
2.10.0-rc.1083 59 6/8/2022
2.10.0-rc.1070 57 6/3/2022
2.10.0-rc.1051 56 5/26/2022
2.10.0-rc.1037 67 5/16/2022
2.10.0-rc.1035 63 5/13/2022
2.10.0-rc.1034 65 5/13/2022
2.10.0-rc.1033 60 5/13/2022
2.10.0-rc.1032 60 5/13/2022
2.10.0-rc.1031 63 5/13/2022
2.10.0-rc.1022 167 4/2/2022
2.10.0-rc.1013 60 3/26/2022
2.9.0 446 2/21/2022
2.9.0-rc.975 82 2/13/2022
2.9.0-beta.971 62 2/13/2022
2.9.0-beta.47 74 1/9/2022
2.9.0-beta.45 78 12/29/2021
2.9.0-beta.44 83 12/21/2021
2.9.0-beta.43 87 12/19/2021
2.9.0-beta.1 59 2/13/2022
2.2.4 321 11/22/2021
2.2.4-beta.42 97 11/10/2021
2.2.4-beta.41 117 11/3/2021
2.2.4-beta.40 94 9/29/2021
2.2.4-beta 199 9/28/2021
2.2.3 280 9/22/2021
2.2.3-beta 204 9/21/2021
2.2.2 296 9/13/2021
2.2.2-beta 180 9/13/2021
2.2.1 331 9/11/2021
2.2.1-beta 234 9/11/2021
2.2.0 309 9/10/2021
2.2.0-beta 221 9/10/2021
2.1.0 307 9/5/2021
2.1.0-beta.33 111 9/5/2021
2.0.18 260 8/14/2021
2.0.0-beta.31 113 8/14/2021
2.0.0-beta.30 95 8/13/2021
2.0.0-beta.29 108 8/12/2021
2.0.0-beta.28 101 8/11/2021
2.0.0-beta.27 111 8/10/2021
2.0.0-beta.26 97 8/9/2021
2.0.0-beta.25 113 8/6/2021
2.0.0-beta.24 108 8/5/2021
2.0.0-beta.23 124 8/4/2021
2.0.0-beta.22 111 8/4/2021
2.0.0-beta.20 97 7/31/2021
2.0.0-beta.19 153 7/30/2021
2.0.0-beta.15 126 5/13/2021
2.0.0-beta.14 109 5/12/2021
2.0.0-beta.13 112 5/11/2021
2.0.0-beta.12 134 5/10/2021
2.0.0-beta.11 122 5/9/2021
2.0.0-beta.10 157 5/7/2021
2.0.0-beta.9 123 4/23/2021
2.0.0-beta.8 126 4/22/2021
2.0.0-beta.7 119 4/22/2021
2.0.0-beta.6 103 4/22/2021
2.0.0-beta.5 107 4/15/2021
2.0.0-beta.4 117 4/14/2021
2.0.0-beta 196 4/10/2021
1.4.2.1 433 9/3/2020
1.4.2 359 9/2/2020
1.4.1 385 8/29/2020
1.4.0 407 8/29/2020
1.4.0-release.1 63 2/13/2022
1.3.5 387 6/9/2020
1.3.4 380 6/9/2020
1.3.3 394 6/8/2020
1.3.2 356 6/8/2020
1.3.1 506 6/5/2020
1.3.0 414 6/3/2020
1.2.0 583 5/26/2020
1.1.0 432 5/26/2020
1.0.0 456 5/23/2020
0.9.0 399 5/21/2020
0.8.5 398 5/20/2020
0.8.4 418 5/9/2020
0.8.3 412 5/8/2020
0.8.2 401 5/2/2020
0.8.1 403 4/30/2020
0.8.0 425 4/28/2020
0.7.0 409 4/20/2020
0.6.5 372 4/16/2020
0.6.4 412 4/15/2020
0.6.3 406 4/14/2020
0.6.2 409 4/14/2020
0.6.1 392 4/13/2020
0.6.0 423 4/12/2020
0.5.3 538 4/5/2020
0.5.2 420 3/30/2020
0.5.1 503 3/28/2020
0.5.0 389 3/27/2020
0.4.0 421 3/17/2020
0.3.0 404 3/13/2020
0.2.0 388 3/11/2020
0.0.1.1 450 3/7/2020
0.0.1 403 3/7/2020
0.0.1-alpha 329 3/8/2020

• This release contains new feature and fixes found in Apache Pulsar 2.9.0 official java client/drive
• Apache Pulsar BrokerMetadata
• Apache Pulsar Exlusive Producer
• Replaced Ask<T> with TaskCompletionSource<T> to await creation of Pulsar client, producer, consumer and reader
• Fixed TimeZoneId KeyNotFound exception in SQL
• Deploy Apache Pulsar with TestContainer for esay testing

Full changelog at https://github.com/eaba/SharpPulsar/blob/main/CHANGELOG.md