Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData 1.1.0

Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData

This package allows you to read Form Data values and files. Files streams are not read in binding model process and are provided to be used in user code.

How it works

Default ASP .NET Core behaviour reads all form contents including files during model binding process. File streams are fully drained to the end and their contents are saved in memory and/or disk before any user custom code.

This behaviour is not convenient when we want to treat files as streams. Possible cases:

• You process file streams consequentially. For example, you want to calculate file hash only.
• You want to read form string values first and validate its data before reading any files' content.

This package allows you to use default binding model except for files that will be available to user code later after model binding process.

There are some drawbacks:

• If you read IFormStreamedFile object from FormStreamedFileCollection then previous files' streams will become empty either you have read them earlier or not. It is not obvious behaviour.
• Swagger is not supported.
• FormOptions settings are not supported yet.
• RequestFormLimitsAttribute is not supported yet.

It is recommended to use standard behaviour if both conditions are met:

1. You don't need streaming.
2. You have enough resources to store incoming files' content of all simultaneous requests in memory and on disk.

In Benchmarking section you can see that new behaviour is faster in high performance stream processing use cases. It is strongly recommended to measure performance for your use case before using it in production environment.

Implementation

The implementation is based on Microsoft suggestion with addition:

• DisableFormValueModelBindingAttribute is presented as SetFormStreamedDataValueProviderAttribute which actually replaces value provider with new provider.
• Model binding is enabled. It means that all standard and custom binding to your properties of specific types from string values is supported.
• Model binding of property or parameter of type FormStreamedFileCollection is introduced. This type allows reading file contents consequentially by user code after model binding process.
• You can read Form Data with ReadFormStreamedDataAsync extensions method of HttpRequest.

New behaviour usage requirements

• Always provide form string values first.
• Always provide form file values last.
• Read file streams consequentially, that is at the beginning get first IFormStreamedFile object then read its stream to the end. Then get next IFormStreamedFile object and then read its stream to the end. When you get next IFormStreamedFile object the first file stream will be read automatically to the end and you will not be able to read it.
• Use only one property or one parameter of type FormStreamedFileCollection because it will always be one stream.
• Do not combine standard behaviour and this behaviour in one controller action. For example, do not use IFormFile and FormStreamedFileCollection in one action.

Installing

dotnet add package Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData

Usage

Register model binder for FormStreamedFileCollection type.

  services.AddControllers(o => o.AddFormStreamedFileCollectionBinder());

Set SetFormStreamedDataValueProviderAttribute for controller actions where file streams should be read. This will replace default value providers for Form Data with FormStreamedDataValueProvider.

Set FromFormStreamedDataAttribute for action parameters or request object properties that should be bound to form values. To read files' streams use only one property or one parameter of type FormStreamedFileCollection.

Examples:

public class NewRequestDto
{
	public string Name { get; set; }
	public int Age { get; set; }
	public FormStreamedFileCollection StreamedFiles { get; set; }
}

[ApiController]
[Route("[controller]")]
public class FilesController : ControllerBase
{
	private readonly FileService _fileService;

	public FilesController(FileService fileService)
	{		
		_fileService = fileService;
	}

	[HttpPost("SaveNew")]
	[RequestSizeLimit(long.MaxValue)]
	[SetFormStreamedDataValueProvider]
	public async Task<ActionResult> SaveNewWay(
		[FromFormStreamedData] NewRequestDto requestDto,
		CancellationToken cancellationToken)
	{		
		await foreach (var file in requestDto.StreamedFiles.WithCancellation(cancellationToken))
		{
			await using var stream = file.OpenReadStream();
			await _fileService.SaveFileAsync(stream, file.FileName, cancellationToken);
		}

		return Ok();
	}

	[HttpPost("SaveNewByParameter")]
	[RequestSizeLimit(long.MaxValue)]
	[SetFormStreamedDataValueProvider]
	public async Task<ActionResult> SaveNewWayByParameter(
		FormStreamedFileCollection files,
		CancellationToken cancellationToken)
	{
		await foreach (var file in files.WithCancellation(cancellationToken))
		{
			await using var stream = file.OpenReadStream();
			var filePath = await _fileService.SaveFileAsync(stream, file.FileName, cancellationToken);
		}

		return Ok();
	}
}

Benchmarking

Use cases and implementation

BenchmarkNet was used to measure new behaviour performance. 3 use cases were tested:

1. File content hashing.
2. Saving to disk.
3. Uploading to localhost Minio S3 storage.

All three cases were implemented in TestApi project. Performance tests were implemented in (PerformanceTests)[tests/PerformanceTests] project.

There were used three values for TestFileSize parameter:

1. Small - API receives 5 1Mb-sized generated files.
2. Big - API receives 2 100Mb-sized generated files.
3. Large - API receives 1 1Gb-sized generated files.

All these values can be changed in FileGeneratorSetting class.

Results

File content hashing use case

Saving to disk use case

Uploading to localhost Minio S3 storage use case

Summary

1. New behaviour is faster if file contents are being read rapidly. It can be observed in first two use cases.
2. Standard (old) behaviour is faster if files streams are sent to S3. It is much faster to read all content before sending it to next service as another stream. BufferedStream can boost new behaviour speed but it will not be faster anyway.
3. Use new behaviour only if you need to process files` content in high performance method or if you do not have enough resources to store incoming files' content of all simultaneous requests in memory and on disk.

Api site, performance test console application and Mino S3 server storage were launched on one machine. So these results may not reflect production environment performance. You are welcome to update them if performance is measured in production (or at least preproduction) environment.

Contributing

To contribute, you will need to setup your local environment, see prerequisites. For the contribution and workflow guide, see package development lifecycle.

Prerequisites

Make sure you have installed all of the following prerequisites on your development machine:

• Git - Download & Install Git. OSX and Linux machines typically have this already installed.
• .NET Core 3.1 or higher - Download & Install .NET Core 3.1.

Package development lifecycle

• Implement package logic in src
• Add or adapt integration tests (prefer before and simultaneously with coding) in tests
• Add or change the documentation as needed
• Open pull request in the correct branch. Target the project's master branch

Maintainers

github.maintain@byndyusoft.com