Microsoft.ML.Tokenizers
0.21.1
Prefix Reserved
There is a newer prerelease version of this package available.
See the version list below for details.
See the version list below for details.
dotnet add package Microsoft.ML.Tokenizers --version 0.21.1
NuGet\Install-Package Microsoft.ML.Tokenizers -Version 0.21.1
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Microsoft.ML.Tokenizers" Version="0.21.1" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Microsoft.ML.Tokenizers --version 0.21.1
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
#r "nuget: Microsoft.ML.Tokenizers, 0.21.1"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install Microsoft.ML.Tokenizers as a Cake Addin
#addin nuget:?package=Microsoft.ML.Tokenizers&version=0.21.1
// Install Microsoft.ML.Tokenizers as a Cake Tool
#tool nuget:?package=Microsoft.ML.Tokenizers&version=0.21.1
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
Microsoft.ML.Tokenizers contains the implmentation of the tokenization used in the NLP transforms.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net5.0 was computed. net5.0-windows was computed. net6.0 was computed. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
.NET Core | netcoreapp2.0 was computed. netcoreapp2.1 was computed. netcoreapp2.2 was computed. netcoreapp3.0 was computed. netcoreapp3.1 was computed. |
.NET Standard | netstandard2.0 is compatible. netstandard2.1 was computed. |
.NET Framework | net461 was computed. net462 was computed. net463 was computed. net47 was computed. net471 was computed. net472 was computed. net48 was computed. net481 was computed. |
MonoAndroid | monoandroid was computed. |
MonoMac | monomac was computed. |
MonoTouch | monotouch was computed. |
Tizen | tizen40 was computed. tizen60 was computed. |
Xamarin.iOS | xamarinios was computed. |
Xamarin.Mac | xamarinmac was computed. |
Xamarin.TVOS | xamarintvos was computed. |
Xamarin.WatchOS | xamarinwatchos was computed. |
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.
-
.NETStandard 2.0
- System.Text.Json (>= 6.0.1)
NuGet packages (3)
Showing the top 3 NuGet packages that depend on Microsoft.ML.Tokenizers:
Package | Downloads |
---|---|
Microsoft.ML.TorchSharp
Microsoft.ML.TorchSharp contains ML.NET integration of TorchSharp. |
|
Microsoft.Teams.AI
SDK focused on building AI based applications for Microsoft Teams. |
|
Microsoft.KernelMemory.AI.TikToken
Provide TikToken tokenizers in Kernel Memory |
GitHub repositories (2)
Showing the top 2 popular GitHub repositories that depend on Microsoft.ML.Tokenizers:
Repository | Stars |
---|---|
microsoft/semantic-kernel
Integrate cutting-edge LLM technology quickly and easily into your apps
|
|
lindexi/lindexi_gd
博客用到的代码
|
Version | Downloads | Last updated |
---|---|---|
0.22.0-preview.24179.1 | 6,932 | 4/2/2024 |
0.22.0-preview.24162.2 | 13,883 | 3/13/2024 |
0.21.1 | 40,306 | 1/18/2024 |
0.21.0 | 43,592 | 11/27/2023 |
0.21.0-preview.23511.1 | 49,952 | 10/13/2023 |
0.21.0-preview.23266.6 | 48,335 | 5/17/2023 |
0.21.0-preview.22621.2 | 1,929 | 12/22/2022 |
0.20.1 | 66,988 | 2/1/2023 |
0.20.1-preview.22573.9 | 1,988 | 11/24/2022 |
0.20.0 | 25,625 | 11/8/2022 |
0.20.0-preview.22551.1 | 200 | 11/1/2022 |