CsharpRAPL 0.1.1
See the version list below for details.
dotnet add package CsharpRAPL --version 0.1.1
NuGet\Install-Package CsharpRAPL -Version 0.1.1
<PackageReference Include="CsharpRAPL" Version="0.1.1" />
paket add CsharpRAPL --version 0.1.1
#r "nuget: CsharpRAPL, 0.1.1"
// Install CsharpRAPL as a Cake Addin
#addin nuget:?package=CsharpRAPL&version=0.1.1
// Install CsharpRAPL as a Cake Tool
#tool nuget:?package=CsharpRAPL&version=0.1.1
CsharpRAPL
CSharpRAPL is a framework for benchmarking C# in regards to energy.
In the Benchmarks folder, all the benchmarks created for this project can be found.
CSharpRAPL contains the library code we have implemented to use for benchmarking.
This code is a continuation and extension of lrecht/ParadigmComparison, which is an earlier university project.
By default, the CSharpRAPL tries to make each loop iteration take 2 seconds called Dynamic Loop Iteration Scaling.
CSharpRAPL also has Dynamic Iteration Calculation which scales the number of loop iterations according to the deviation of the results.
The results generated is a CSV file that contains the DRAM Energy, Temperature, Elapsed Time, Package Energy, and the return value of the Benchmark.
Note that the separator is ; and not ,
Example
ElapsedTime;PackageEnergy;DramEnergy;Temperature;BenchmarkReturnValue
0.31062468886375416;3341.3842320442195;172.80355095863342;43;19
0.310964733362198;3335.9728753566737;172.84899950027466;43;19
0.31095251441001887;3353.798389434814;173.25878143310547;43;19
0.31064271926879894;3366.4405345916743;172.3937690258026;43.5;19
The results can be visualized using Box Plots an example of this can be seen on the following image (Note that the following image is only for illustrative purposes and the results might not be correct):
Requirements
- Linux with access to RAPL
- libgdiplus - Is required for plotting
- cpuset - Required for creating a cpuset to execute benchmarks. Can be used without.
Usage
To make use of Dynamic Iteration Calculation and Dynamic Loop Iteration Scaling your class must contain the following fields:
public static int Iterations;
public static int LoopIterations;
Without these fields, Dynamic Iteration Calculation and Dynamic Loop Iteration Scaling won't be able to be used.
Your benchmark class doesn't have to be static same goes for benchmarks themselves.
Registring Benchmarks
There are two ways of registering benchmarks, either using Attributes or doing it manually. The most tested method is using Attributes.
Attributes
We can register a benchmark by adding the Benchmark attribute where the first argument is the benchmark group and the second is the description for the benchmark. An example can be seen below.
[Benchmark("Addition", "Tests simple addition")]
public static int Add() {
int a = 10;
int res = 0;
for (int i = 0; i < LoopIterations; i++) {
res = a + i;
}
return res;
}
Here you have to use BenchmarkCollector which collects all methods with the Benchmark attribute except for those where skip is true.
Manual
You can add benchmarks manually by either using BenchmarkCollector or BenchmarkSuite to do that use the RegisterBenchmark method.
Example of manual registration:
public static void Main(string[] args){
var benchmarkSuit = new BenchmarkSuite();
benchmarkSuit.RegisterBenchmark("TestGroup", DummyBenchmark);
}
public static int DummyBenchmark() {
return 1;
}
Running the Benchmarks
To run the benchmarks simply call RunAll
on either your BenchmarkSuite or BenchmarkCollector instance.
For more examples look at Suite.cs or at the tests.
Scripts
We have provided the scripts:
- Benchmarks/execute.sh
- Sets the CPU scaling governor to performance
- Executes the benchmarks on shielded cores (see
cpuSet.sh
) - Sets the CPU scaling governor to powersave after running the benchmarks
- Benchmarks/removeResults.sh
- Removes old results both in the form of CSV and plots. (Alternatively use -r)
- Benchmarks/zipResults.sh
- Zips CSV results into a zip file. (Alternatively use -z)
- Scripts/cpuSet.sh
- Shields cores number 2 and 4 (0 indexed) from other processes, including the kernel. Uses the ubuntu package "cpuset", which is a Python wrapper over the Linux
cpuset
interface. This shield is to be used when executing benchmarks.
- Shields cores number 2 and 4 (0 indexed) from other processes, including the kernel. Uses the ubuntu package "cpuset", which is a Python wrapper over the Linux
- Scripts/runAtStartup.sh
- First makes sure hyperthread cores are disabled, then sets swappiness to 10, disables ASLR, lowers
perf
sampling interval, and disables Intel turbo-boost for consistency.
- First makes sure hyperthread cores are disabled, then sets swappiness to 10, disables ASLR, lowers
- Scripts/setPerformance.sh
- Sets the cpu scaling governor to performance.
- Scripts/setPowerSave.sh
- Sets the cpu scaling governor to powersave.
CLI Options
-g, --SkipPlotGroups If plotting each benchmark group should be skipped.
-i, --Iterations (Default: -1) Sets the target iterations. (Disables Dynamic Iteration Calculation)
-l, --LoopIterations (Default: -1) Sets the target loop iterations. (Disables Dynamic Loop Iteration Scaling)
-r, --RemoveOldResults If set removes all files from the output folder and the plot folder.
-o, --OutputPath (Default: results/) Set the output path for results.
-p, --PlotOutputPath (Default: _plots/) Sets the output path for plots.
-a, --BenchmarksToAnalyse The names of the benchmarks to analyse.
-z, --ZipResults Zips the CSV results and plots into a single zip file.
--OnlyPlot Plots the results in the output path.
--OnlyAnalysis Analysis the results in the output path.
--help Display this help screen.
--version Display version information.
--Verbose Enables debug information.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net6.0
- Accord.Statistics (>= 3.8.0)
- CommandLineParser (>= 2.9.0-preview1)
- CsvHelper (>= 27.2.1)
- Humanizer.Core (>= 2.14.1)
- ScottPlot (>= 4.1.27)
- System.Drawing.Common (>= 5.0.2)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
0.2.15 | 446 | 5/20/2022 |
0.2.14 | 423 | 5/11/2022 |
0.2.13 | 382 | 5/11/2022 |
0.2.12 | 506 | 3/16/2022 |
0.2.11 | 384 | 3/16/2022 |
0.2.10 | 386 | 3/16/2022 |
0.2.9 | 400 | 3/15/2022 |
0.2.8 | 402 | 3/8/2022 |
0.2.7 | 477 | 3/2/2022 |
0.2.6 | 382 | 3/1/2022 |
0.2.5 | 391 | 3/1/2022 |
0.2.4 | 384 | 3/1/2022 |
0.2.3 | 385 | 3/1/2022 |
0.2.2 | 390 | 3/1/2022 |
0.2.1 | 475 | 2/21/2022 |
0.2.0 | 398 | 2/21/2022 |
0.1.9 | 387 | 2/21/2022 |
0.1.8 | 414 | 2/17/2022 |
0.1.7 | 385 | 2/17/2022 |
0.1.6 | 388 | 2/16/2022 |
0.1.5 | 407 | 2/9/2022 |
0.1.4 | 389 | 2/9/2022 |
0.1.3 | 392 | 2/8/2022 |
0.1.2 | 374 | 2/8/2022 |
0.1.1 | 372 | 2/8/2022 |
0.1.0 | 416 | 2/7/2022 |
0.0.6 | 397 | 2/3/2022 |
0.0.5 | 386 | 2/1/2022 |