Mapache.PromptEvalDotNet 0.0.2

Mapache.PromptEvalDotNet NuGet Package

This NuGet package, Mapache.PromptEvalDotNet, empowers your .NET projects with a robust and versatile library for evaluating, testing, and validating applications and systems using Large Language Models (LLMs). Seamlessly integrate AI/ML evaluations into your projects to assess performance, accuracy, and robustness with ease.

Installation

Install the Mapache.PromptEvalDotNet NuGet package via the NuGet Package Manager or the .NET CLI:

dotnet add package Mapache.PromptEvalDotNet

Usage

Sample Code

// Sample code for using Mapache.PromptEvalDotNet
var kernelBuilder = Kernel.CreateBuilder();

kernelBuilder.AddAzureOpenAIChatCompletion(config["AZURE_OPENAI_MODEL"]!, config["AZURE_OPENAI_ENDPOINT"]!, config["AZURE_OPENAI_KEY"]!);

var evalFactory = new Mapache.PromptEvalDotNet.EvaluatorFactory("MyProjectEvals", builder.Build());

var coherence = await evalFactory.Coherence("blog-posts").GetScore(prompt, answer);

Explanation

1. Kernel Setup: Begin by setting up the kernel using Kernel.CreateBuilder().

2. Integration with Azure OpenAI Chat Completion: Use kernelBuilder.AddAzureOpenAIChatCompletion to integrate with Azure OpenAI Chat Completion, providing the necessary configuration parameters.

3. Evaluator Factory Initialization: Create an instance of Mapache.PromptEvalDotNet.EvaluatorFactory by providing a unique project name and the built kernel.

4. Evaluate Coherence: Utilize the evalFactory.Coherence method to evaluate coherence for a specific context, such as "blog-posts."

5. Get Score: Obtain the coherence score by calling GetScore with the prompt and answer.

Contributing

See the project repo for more information.