Xunit.Extensions.Ordering 1.2.2

.NET Core 2.2
There is a newer version of this package available.
See the version list below for details.
dotnet add package Xunit.Extensions.Ordering --version 1.2.2
NuGet\Install-Package Xunit.Extensions.Ordering -Version 1.2.2
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Xunit.Extensions.Ordering" Version="1.2.2" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Xunit.Extensions.Ordering --version 1.2.2
#r "nuget: Xunit.Extensions.Ordering, 1.2.2"
#r directive can be used in F# Interactive, C# scripting and .NET Interactive. Copy this into the interactive tool or source code of the script to reference the package.
// Install Xunit.Extensions.Ordering as a Cake Addin
#addin nuget:?package=Xunit.Extensions.Ordering&version=1.2.2

// Install Xunit.Extensions.Ordering as a Cake Tool
#tool nuget:?package=Xunit.Extensions.Ordering&version=1.2.2

Xunit extension for ordering test collections, test classes and test cases. It's useful for integration testing using Xunit if you cannot or you don't want to make each test method atomic.

Nuget: https://www.nuget.org/packages/Xunit.Extensions.Ordering/

There is very limiting space for adding support of ordered (integration) testing into xunit without rewriting runner.

Usage:

Setup

Add AssemblyInfo.cs with only following lines of code

using Xunit;

//Optional
[assembly: CollectionBehavior(DisableTestParallelization = true)]
//Optional
[assembly: TestCaseOrderer("Xunit.Extensions.Ordering.TestCaseOrderer", "Xunit.Extensions.Ordering")]
//Optional
[assembly: TestCollectionOrderer("Xunit.Extensions.Ordering.CollectionOrderer", "Xunit.Extensions.Ordering")]

Ordering test classes and cases

Add Order Attribute to test classes and methods. Tests are executed in ascending order. If no Order attribute is specified default 0 is assigned. Multiple Order attributes can have same value. Their execution order in this case is deterministic but unpredictible.

[Order(1)]
public class TC2
{
	[Fact, Order(2)]
	public void M1() { /* ... */ }

	[Fact, Order(3)]
	public void M2() { /* ... */ }

	[Fact, Order(1)]
	public void M3() { /* ... */ }
}

Ordering test classes in collection

You can order test classes in collections by adding Order attribute too but you have to use patched test framework by add following line to AssemblyInfo.cs

using Xunit;

[assembly: TestFramework("Xunit.Extensions.Ordering.TestFramework", "Xunit.Extensions.Ordering")]

Now you can order collections like this

[CollectionDefinition("C1")]
public class Collection1 { }
[Collection("C1"), Order(2)]
public class TC3
{
	[Fact, Order(1)]
	public void M1() { /* 3 */ }

	[Fact, Order(2)]
	public void M2() { /* 4 */ }
}
[Collection("C1"), Order(1)]
public partial class TC5
{
	[Fact, Order(2)]
	public void M1() { /* 2 */ }

	[Fact, Order(1)]
	public void M2() { /* 1 */ }

}

Ordering test collection

You can order test collections by adding Order attribute too definition collection class

[CollectionDefinition("C1"), Order(3)]
public class Collection3 { }
[CollectionDefinition("C2"), Order(1)]
public class Collection3 { }

Mixing test classes in collections and test classes without explicit collection assignement

Test classes without explicitely assigned collection are collections implicitely in Xunit (collection per class). So if you mix test classes with assigned collection and test classes without assigned collection they are on the same level and Order is applied following this logic.

[CollectionDefinition("C1"), Order(3)]
public class Collection3 { }
[CollectionDefinition("C2"), Order(1)]
public class Collection3 { }
[Order(2)]
public class TC2
{
	[Fact]
	public void M1() { /* 4 */ }
}
[Collection("C1")]
public class TC3
{
	[Fact]
	public void M1() { /* 5 */ }
}
[Collection("C2"), Order(2)]
public partial class TC5
{
	[Fact]
	public void M1() { /* 3 */ }
}
[Collection("C2"), Order(1)]
public partial class TC5
{
	[Fact, Order(2)]
	public void M1() { /* 2 */ }

	[Fact, Order(1)]
	public void M2() { /* 1 */ }
}

Checking continuity of order indexes and detection of duplicates

You can enable warning messages about continuity and duplicates of Order indexes by enabling diagnosticMessages.

  1. Create xnuit.runner.json in root of your test project
{
	"$schema": "https://xunit.github.io/schema/current/xunit.runner.schema.json",
	"diagnosticMessages": true
}
  1. Set "Copy to output directory" for this file in visual studio to "Copy if newer"
  2. In the Output Visual Studio window choose "Tests" option in the "Show output from" dropdown
  3. You will see warnings like
Missing test collection order sequence from '4' to '39'.
Missing test case order '1' in test class 'Xunit.Extensions.Ordering.Tests.TC6'.
Missing test classes order sequence from '3' to '29' for collection 'C1'.
Missing test case order sequence from '2' to '19' in test class 'Xunit.Extensions.Ordering.Tests.TC5'.
Product Versions
.NET net5.0 net5.0-windows net6.0 net6.0-android net6.0-ios net6.0-maccatalyst net6.0-macos net6.0-tvos net6.0-windows net7.0 net7.0-android net7.0-ios net7.0-maccatalyst net7.0-macos net7.0-tvos net7.0-windows
.NET Core netcoreapp2.2 netcoreapp3.0 netcoreapp3.1
Compatible target framework(s)
Additional computed target framework(s)
Learn more about Target Frameworks and .NET Standard.
  • .NETCoreApp 2.2

NuGet packages (1)

Showing the top 1 NuGet packages that depend on Xunit.Extensions.Ordering:

Package Downloads
Refactorius.Data.Xunit

The MS SQL database enabing xUnit extension library.

GitHub repositories (3)

Showing the top 3 popular GitHub repositories that depend on Xunit.Extensions.Ordering:

Repository Stars
AvaloniaUI/Avalonia
A cross-platform UI framework for .NET
VirtoCommerce/vc-platform
VirtoCommerce Platform repository
linvi/tweetinvi
Tweetinvi, an intuitive Twitter C# library for the REST and Stream API. It supports .NET, .NETCore, UAP (Xamarin)...
Version Downloads Last updated
1.4.5 1,777,874 2/11/2019
1.4.4 929 2/10/2019
1.4.3 613 2/10/2019
1.4.2 596 2/10/2019
1.4.1 589 2/10/2019
1.4.0 687 2/9/2019
1.3.1 558 2/9/2019
1.3.0 503 2/8/2019
1.2.3 517 2/7/2019
1.2.2 663 2/6/2019
1.2.1 528 2/6/2019
1.1.1 539 2/4/2019
1.1.0 507 2/4/2019
1.0.1 511 2/4/2019

Order index checking fixes