BenchmarkCmp 2.1.0

dotnet tool install --global BenchmarkCmp --version 2.1.0                
This package contains a .NET tool you can call from the shell/command line.
dotnet new tool-manifest # if you are setting up this repo
dotnet tool install --local BenchmarkCmp --version 2.1.0                
This package contains a .NET tool you can call from the shell/command line.
#tool dotnet:?package=BenchmarkCmp&version=2.1.0                
nuke :add-package BenchmarkCmp --version 2.1.0                

dotnet-benchmarkcmp NuGet Made in Ukraine

Inspired by go-benchcmp, dotnet-benchmarkcmp displays performance changes between benchmarks.

benchmarkcmp parses the output of two BenchmarkDotNet benchmark runs, correlates the results per benchmark (based on name), and displays the deltas.

Getting started

Installation

To install the tool, use dotnet tool install command:

dotnet tool install -g BenchmarkCmp

Use

To measure the performance impact of a change, use BenchmarkDotNet to run benchmarks before / after the change and save artifacts. Then feed the benchmark artifacts (check Supported exporters section to see supported artifacts) to benchmarkcmp:

dotnet benchmarkcmp old.json new.json

benchmarkcmp will summarize and display the performance changes, in a format like this:

 benchmark                     | old time/op | new time/op | delta
-------------------------------+-------------+-------------+--------
 SimpleBenchmark.A: DefaultJob | 1,210 s     | 1,108 s     | -8,44%

 benchmark                     | old allocs | new allocs | delta
-------------------------------+------------+------------+-------
 SimpleBenchmark.A: DefaultJob | 0          | 0          | ~

 benchmark                     | old G0 collects | new G0 collects | delta
                               | per 1k ops      | per 1k ops      |
-------------------------------+-----------------+-----------------+-------
 SimpleBenchmark.A: DefaultJob | 0               | 0               | ~

 benchmark                     | old G1 collects | new G1 collects | delta
                               | per 1k ops      | per 1k ops      |
-------------------------------+-----------------+-----------------+-------
 SimpleBenchmark.A: DefaultJob | 0               | 0               | ~

 benchmark                     | old G2 collects | new G2 collects | delta
                               | per 1k ops      | per 1k ops      |
-------------------------------+-----------------+-----------------+-------
 SimpleBenchmark.A: DefaultJob | 0               | 0               | ~

Supported data

  • Base Statistics
    • Min / median / mean / max run time
  • Memory Diagnoser
    • Allocated memory
    • Gen 0, 1, 2 allocation
  • Threading Diagnoser
    • Completed work items
    • Lock contentions
  • Native Memory Profiler
    • Allocated native memory
    • Native memory leak
  • Hardware Counter Diagnoser
    • Branch instructions
    • Branch mispredictions
    • Cache misses
  • Exception Diagnoser
    • Exception frequency
  • JIT Stats Diagnoser
    • Amount of JITted methods
    • Amount of tiered methods
    • JIT allocated memory

Supported exporters

  • JsonExporter.Brief
  • JsonExporter.Full
  • XmlExporter.Brief
  • XmlExporter.Full
Product Compatible and additional computed target framework versions.
.NET net6.0 is compatible.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

This package has no dependencies.

Version Downloads Last updated
2.1.0 211 7/1/2023
2.0.0 189 6/17/2023
1.1.1 514 2/18/2022
1.1.0 354 3/8/2021
1.0.0 352 2/14/2021

- Added min / median / max statistics support.
- Added XML exporter support (Brief and Full).