SerilogEcsMapper 0.20.0

SerilogEcsMapper

An Elastic Common Schema (ECS) custom mapper for Serilog.

Features

Allows the use of structured log properties in Serilog to directly map to fields on an ECS-formatted log event, which is expected to be sent to Logstash and Elasticsearch.

It lets you do this:

var agent = new Elastic.CommonSchema.Agent
{
    Id = "8a4f500d",
    EphemeralId = "8a4f500f",
    Name = "foo",
    Type = "filebeat",
    Version = "6.0.0-rc2"
};

logger.Information("{@Agent}", agent);

This will result in the agent field being located at:

_source.agent

Caveats

Note that the default behavior is still in effect for this also, so this field will also exist under metadata, which may or may not be desirable. I personally didn't care for it because all of the empty values were deserialized into null, and I'd prefer they just simply not be there.

The Serilog LogEvents are readonly when we intercept them, so it doesn't seem possible for me to resolve this in the library. The simplest way I found to remove these duplicate fields was to use the mutate filter in logstash.conf:

filter {
    mutate {
        remove_field => [ "[metadata][agent]" ]
        // Any other fields you mapped and want removed
    }
}

Usage

The EcsTextFormatterConfigurationBuilder is intended to provide an easy way to allow you to create custom maps for your log properties. You can use this to create a configuration with any maps you want before passing it to your EcsTextFormatter.

The builder makes use of AutoMapper to handle the mappings internally, so you'll need to create an IMapper and register the EcsMapperProfile before creating your builder:

var mapper = new MapperConfiguration(cfg =>
{
    cfg.AddProfile(new SerilogEcsMapperProfile());
}).CreateMapper();

var builder = new EcsTextFormatterConfigurationBuilder(mapper);

If you're using ASP.NET Core you should be able to use this same configuration to add the mapper to your DI container.

Examples

Lets say you wanted to create a custom map for just the Agent field. Here's how you would accomplish that:

var mapper = new MapperConfiguration(cfg =>
{
    cfg.AddProfile(new SerilogEcsMapperProfile());
}).CreateMapper();

var builder = new EcsTextFormatterConfigurationBuilder(mapper);
var config = builder.WithAgent().Build();
var textFormatter = new EcsTextFormatter(config);

var logger = new LoggerConfiguration()
    .WriteTo.Console(formatProvider: CultureInfo.InvariantCulture)
    .WriteTo.Http(
        requestUri: "http://logstash:5000",
        queueLimitBytes: null,
        textFormatter: textFormatter)
    .CreateLogger();

The builder is intended to behave much like other builder-type classes found in the .NET world. Use it to create your initial text formatter configuration, and then do anything else you need to do to this configuration before passing it to your new EcsTextFormatter. Once your formatter is created, provide it to the Serilog.Sinks.HTTP sink, and you should be all set.

For convenience, there's also a WithAll() you can use to add everything in one go:

var config = builder.WithAll().Build();

This will add every available map to your configuration, and you can use the full range of ECS types available.

Credits

Thanks to Github user sgtobin for providing the original solution that this library is based on.