CerbiStream 1.0.16

CerbiStream: Dev-Friendly, Governance-Enforced Logging for .NET

🚀 View CerbiStream Benchmarks

Compare against Serilog, NLog, and others. CerbiStream is engineered for high performance, strict governance, and enterprise-grade log routing.

Table of Contents

• Overview
• Highlights
• Features
• Architecture & Implementation
• Preset Modes and Configuration
• Usage Examples
• Integration & Supported Platforms
• Governance and Compliance
• Telemetry Provider Support
• Unit Testing
• Cerbi Suite: The Bigger Picture
• Contributing
• License

Overview

CerbiStream is a high-performance, dev-friendly logging framework for .NET that not only emphasizes low latency and high throughput but also enforces structured logging governance. Designed for modern applications using ILogger<T>, CerbiStream integrates seamlessly into ASP.NET Core, Blazor, and Entity Framework projects. Its flexible configuration options and robust governance support make it ideal for both development and enterprise-scale deployments.

CerbiStream is a core component of the Cerbi Suite—a set of tools designed for observability, log governance, routing, and telemetry in regulated and performance-critical environments.

Highlights

• Dev-Friendly & Flexible:
  Works out of the box with .NET Core’s logging abstractions.

• High Performance:
  Engineered for low latency and high throughput with efficient resource usage even when logs are output in realistic development modes.

• Governance-Enforced Logging:
  Ensures logs conform to structured formats and compliance requirements via internal or pluggable governance validators.

• Multiple Integration Options:
  Supports RabbitMQ, Kafka, Azure Service Bus, AWS SQS/Kinesis, and Google Pub/Sub.

• Encryption & Security:
  Offers flexible encryption modes—including Base64 and AES—to secure sensitive logging data.

• Queue-First Architecture:
  Decouples log generation from delivery, enhancing resilience and scalability via the CerbIQ routing component.

• Telemetry & Analytics:
  Built-in support for telemetry providers (OpenTelemetry, Azure App Insights, AWS CloudWatch, etc.) for end-to-end observability.

Features

• Developer Modes:

  • EnableDevModeMinimal(): Minimal console logging without metadata injection—perfect for lightweight development debugging.
  • EnableDeveloperModeWithoutTelemetry(): Console logging with metadata injection but no telemetry data.
  • EnableDeveloperModeWithTelemetry(): Full-fledged logging with metadata, telemetry, and governance checks.
  • EnableBenchmarkMode(): A silent mode disabling all outputs, enrichers, and telemetry, ideal for performance measurement.

• Governance Enforcement:
  Use built-in or externally provided validators (via the CerbiStream.GovernanceAnalyzer or custom hooks) to enforce required log fields and format consistency.

• Encryption Options:
  Configure encryption modes to secure log data with options for None, Base64, or AES encryption.

• Queue-First, Sink-Agnostic Logging:
  Route logs through configured messaging queues first for enhanced fault tolerance and delayed sink processing via CerbIQ.

• Telemetry Integration:
  Out-of-the-box support for major telemetry platforms ensures your log data is immediately useful for observability and diagnostics.

Architecture & Implementation

CerbiStream’s architecture is designed to maximize logging performance while ensuring compliance and structured data integrity:

• Asynchronous Processing:
  The logging pipeline is built using modern asynchronous patterns to avoid blocking I/O. This allows high-volume log generation without impacting application performance.

• Queue-First Model:
  Log events are first enqueued (supporting various messaging systems) before being dispatched to the final sink. This decoupling reduces processing spikes and improves reliability.

• Modular & Configurable:
  The framework uses a fluent API for configuration, letting you easily switch modes, enable encryption, set up telemetry, and plug in custom governance validators.

• Governance & Validation:
  A key aspect of CerbiStream is its governance engine, which enforces structured logging policies. Whether integrated via CerbiStream.GovernanceAnalyzer or custom logic, it ensures that all log messages meet your organizational standards.

• Performance Modes:
  Different preset modes (Benchmark, Developer Modes) allow you to choose the level of output and enrichment based on your environment—from raw performance testing to full-featured dev logging.

Preset Modes and Configuration

Developer Modes

• EnableDevModeMinimal()
  Minimal logging output to the console without metadata or governance checks.

  builder.Logging.AddCerbiStream(options => options.EnableDevModeMinimal());
  

• EnableDeveloperModeWithoutTelemetry()
  Console logging with metadata injection but without telemetry.

  builder.Logging.AddCerbiStream(options => options.EnableDeveloperModeWithoutTelemetry());
  

• EnableDeveloperModeWithTelemetry()
  Full developer mode with metadata enrichment and telemetry.

  builder.Logging.AddCerbiStream(options => options.EnableDeveloperModeWithTelemetry());
  

Benchmark Mode

• EnableBenchmarkMode()
  Disables all outputs and enrichments for pure performance testing.

  builder.Logging.AddCerbiStream(options => options.EnableBenchmarkMode());
  

Advanced Customization

Queue Configuration:

options.WithQueue("RabbitMQ", "localhost", "logs-queue");

Encryption:

options.WithEncryptionMode(EncryptionType.AES)
       .WithEncryptionKey(myKey, myIV);

Governance Validator:

options.WithGovernanceValidator((profile, log) =>
{
    return log.ContainsKey("UserId") && log.ContainsKey("IPAddress");
});

Feature Toggles:

options.WithTelemetryLogging(true)
       .WithConsoleOutput(true)
       .WithMetadataInjection(true)
       .WithGovernanceChecks(true);

Usage Examples

Basic Example

var builder = WebApplication.CreateBuilder(args);
builder.Logging.AddCerbiStream(options =>
{
    options.EnableDevModeMinimal();
});
var app = builder.Build();
app.Run();

Developer Mode with Custom Queue and Encryption

builder.Logging.AddCerbiStream(options =>
{
    options.WithQueue("Kafka", "broker-address", "app-logs")
           .WithEncryptionMode(EncryptionType.Base64)
           .EnableDeveloperModeWithoutTelemetry();
});

Full Mode with Telemetry and Governance

builder.Logging.AddCerbiStream(options =>
{
    options
        .WithQueue("AzureServiceBus", "sb://myservicebus.servicebus.windows.net", "logs-queue")
        .WithEncryptionMode(EncryptionType.AES)
        .WithGovernanceValidator((profile, log) =>
        {
            return log.ContainsKey("UserId") && log.ContainsKey("IPAddress");
        })
        .EnableDeveloperModeWithTelemetry();
});

Integration & Supported Platforms

CerbiStream is designed to work in a variety of environments:

Messaging Queues:
Supports RabbitMQ, Kafka, Azure Service Bus, AWS SQS/Kinesis, and Google Pub/Sub.

Telemetry Providers:

Pluggable Sinks:
CerbiStream is “sink-agnostic” – logs are initially routed to queues, and you can integrate downstream tools (e.g., CerbIQ) to manage final delivery to systems such as Splunk, Elasticsearch, or Blob Storage.

Governance and Compliance

A key differentiator of CerbiStream is its built-in governance:

Structured Logging Enforcement:
Ensures that every log entry adheres to predefined schemas for consistency, aiding in compliance with regulatory standards (e.g., HIPAA, GDPR).

External Governance Hook:
If needed, you can provide your own governance validator:

options.WithGovernanceValidator((profile, log) =>
{
    return log.ContainsKey("UserId") && log.ContainsKey("IPAddress");
});

Optional CerbiStream.GovernanceAnalyzer:
An add-on package that performs static code analysis to ensure that logging policies are consistently followed.

Telemetry Provider Support

CerbiStream integrates seamlessly with popular telemetry systems to provide extended observability:

Supported providers include: OpenTelemetry, Azure App Insights, AWS CloudWatch, GCP Trace, and Datadog.

Configuration is straightforward through the fluent API, ensuring that enriched log data is automatically forwarded to your chosen telemetry platform.

Unit Testing

Example unit test for CerbiStream logging:

var mockQueue = Substitute.For<IQueue>();
var logger = new CerbiLoggerBuilder()
    .WithQueue("TestQueue", "localhost", "unit-test-logs")
    .UseNoOpEncryption()
    .Build(mockQueue);

var result = await logger.LogEventAsync("Test log event", LogLevel.Information);
Assert.True(result);

Cerbi Suite: The Bigger Picture

CerbiStream is a core component of the Cerbi suite—a broader ecosystem aimed at providing enterprise-grade observability, governance, and log routing solutions. The suite includes:

• CerbiStream: For high-performance, governance-enforced logging.
• CerbIQ: For advanced log routing, aggregation, and delivery to various sinks.
• CerbiStream.GovernanceAnalyzer: For static and runtime validation ensuring consistent log compliance.

Contributing

Contributions are welcome!

• Report Bugs or Request Features: Open an issue on GitHub.
• Submit Pull Requests: Follow our code style guidelines and ensure tests pass.
• Join the Community: Star the repo, share feedback, and help improve CerbiStream.

License

This project is licensed under the MIT License.

Star the repo ⭐ | Contribute 🔧 | File issues 🐛
Created by @Zeroshi