KubeOps.Operator.Web 9.1.5

KubeOps Operator Web

The KubeOps Operator Web package provides a webserver to enable webhooks for your Kubernetes operator.

Usage

To enable webhooks and external access to your operator, you need to use ASP.net. The project file needs to reference Microsoft.NET.Sdk.Web instead of Microsoft.NET.Sdk and the Program.cs needs to be changed.

To allow webhooks, the MVC controllers need to be registered and mapped.

The basic Program.cs setup looks like this:

using KubeOps.Operator;

var builder = WebApplication.CreateBuilder(args);
builder.Services
    .AddKubernetesOperator()
    .RegisterComponents();

builder.Services
    .AddControllers();

var app = builder.Build();

app.UseRouting();
app.MapControllers();

await app.RunAsync();

Note the .AddControllers and .MapControllers call. Without them, your webhooks will not be reachable.

Validation Hooks

To create a validation webhook, first create a new class that implements the ValidationWebhook<T> base class. Then decorate the webhook with the ValidationWebhookAttribute to set the route correctly.

After that setup, you may overwrite any of the following methods:

• Create
• CreateAsync
• Update
• UpdateAsync
• Delete
• DeleteAsync

The async methods take precedence over the sync methods.

An example of such a validation webhook looks like:

[ValidationWebhook(typeof(V1TestEntity))]
public class TestValidationWebhook : ValidationWebhook<V1TestEntity>
{
    public override ValidationResult Create(V1TestEntity entity, bool dryRun)
    {
        if (entity.Spec.Username == "forbidden")
        {
            return Fail("name may not be 'forbidden'.", 422);
        }

        return Success();
    }

    public override ValidationResult Update(V1TestEntity oldEntity, V1TestEntity newEntity, bool dryRun)
    {
        if (newEntity.Spec.Username == "forbidden")
        {
            return Fail("name may not be 'forbidden'.");
        }

        return Success();
    }
}

To create the validation results, use the protected methods (Success and Fail) like "normal" IActionResult creation methods.

Mutation Hooks

To create a mutation webhook, first create a new class that implements the MutationWebhook<T> base class. Then decorate the webhook with the MutationWebhookAttribute to set the route correctly.

After that setup, you may overwrite any of the following methods:

• Create
• CreateAsync
• Update
• UpdateAsync
• Delete
• DeleteAsync

The async methods take precedence over the sync methods.

An example of such a mutation webhook looks like:

[MutationWebhook(typeof(V1TestEntity))]
public class TestMutationWebhook : MutationWebhook<V1TestEntity>
{
    public override MutationResult<V1TestEntity> Create(V1TestEntity entity, bool dryRun)
    {
        if (entity.Spec.Username == "overwrite")
        {
            entity.Spec.Username = "random overwritten";
            return Modified(entity);
        }

        return NoChanges();
    }
}

To create the mutation results, use the protected methods (NoChanges, Modified, and Fail) like "normal" IActionResult creation methods.

Conversion Hooks

[!CAUTION] Conversion webhooks are not stable yet. The API may change in the future without a new major version. All code related to conversion webhooks are attributed with the RequiresPreviewFeatures attribute. To use the features, you need to enable the preview features in your project file with the <EnablePreviewFeatures>true</EnablePreviewFeatures> property.

A conversion webhook is a special kind of webhook that allows you to convert Kubernetes resources between versions. The webhooks are installed in CRDs and are called for all objects that need conversion (i.e. to achieve the stored version state).

A conversion webhook is separated to the webhook itself (the MVC controller that registers its route within ASP.NET) and the conversion logic.

The following example has two versions of the "TestEntity" (v1 and v2) and implements a conversion webhook to convert from v1 to v2 and vice versa.

// The Kubernetes resources
[KubernetesEntity(Group = "webhook.dev", ApiVersion = "v1", Kind = "TestEntity")]
public partial class V1TestEntity : CustomKubernetesEntity<V1TestEntity.EntitySpec>
{
    public override string ToString() => $"Test Entity v1 ({Metadata.Name}): {Spec.Name}";

    public class EntitySpec
    {
        public string Name { get; set; } = string.Empty;
    }
}

[KubernetesEntity(Group = "webhook.dev", ApiVersion = "v2", Kind = "TestEntity")]
public partial class V2TestEntity : CustomKubernetesEntity<V2TestEntity.EntitySpec>
{
    public override string ToString() => $"Test Entity v2 ({Metadata.Name}): {Spec.Firstname} {Spec.Lastname}";

    public class EntitySpec
    {
        public string Firstname { get; set; } = string.Empty;

        public string Lastname { get; set; } = string.Empty;
    }
}

The v1 of the resource has first and lastname in the same field, while the v2 has them separated.

public class V1ToV2 : IEntityConverter<V1TestEntity, V2TestEntity>
{
    public V2TestEntity Convert(V1TestEntity from)
    {
        var nameSplit = from.Spec.Name.Split(' ');
        var result = new V2TestEntity { Metadata = from.Metadata };
        result.Spec.Firstname = nameSplit[0];
        result.Spec.Lastname = string.Join(' ', nameSplit[1..]);
        return result;
    }

    public V1TestEntity Revert(V2TestEntity to)
    {
        var result = new V1TestEntity { Metadata = to.Metadata };
        result.Spec.Name = $"{to.Spec.Firstname} {to.Spec.Lastname}";
        return result;
    }
}

The conversion logic is implemented in the IEntityConverter interface. Each converter has a "convert" (from → to) and a "revert" (to → from) method.

[ConversionWebhook(typeof(V2TestEntity))]
public class TestConversionWebhook : ConversionWebhook<V2TestEntity>
{
    protected override IEnumerable<IEntityConverter<V2TestEntity>> Converters => new IEntityConverter<V2TestEntity>[]
    {
        new V1ToV2(), // other versions...
    };
}

The webhook the registers the list of possible converters and calls the converter upon request.

[!NOTE] There needs to be a conversion between ALL versions to the stored version (newest version). If there is no conversion, the webhook will fail and the resource is not stored. So if there exist a v1, v2, and v3, there needs to be a converter for v1 → v3 and v2 → v3 (when v3 is the stored version).

Installing In The Cluster

When creating an operator with webhooks, certain special resources must be provided to run in the cluster. When this package is referenced and KubeOps.Cli is installed, these resources should be generated automatically. Basically, instead of generating a dockerfile with dotnet:runtime as final image, you'll need dotnet:aspnet and the operator needs a service and the certificates for the HTTPS connection since webhooks only operate over HTTPS.

With the KubeOps.Cli package you can generate the required resources or let the customized Build targets do it for you.

The targets create a CA certificate and a server certificate (with respective keys), a service, and the webhook registrations required for you.

[!WARNING] The generated certificate has a validity of 5 years. After that time, the certificate needs to be renewed. For now, there is no automatic renewal process.

Webhook Development

The Operator Web package can be configured to generate self-signed certificates on startup, and create/update your webhooks in the Kubernetes cluster to point to your development machine. To use this feature, use the CertificateGenerator class and UseCertificateProvider() operator builder extension method. An example of what this might look like in Main:

var builder = WebApplication.CreateBuilder(args);
string ip = "192.168.1.100";
ushort port = 443;

using CertificateGenerator generator = new CertificateGenerator(ip);
using X509Certificate2 cert = generator.Server.CopyServerCertWithPrivateKey();
// Configure Kestrel to listen on IPv4, use port 443, and use the server certificate
builder.WebHost.ConfigureKestrel(serverOptions =>
{
    serverOptions.Listen(System.Net.IPAddress.Any, port, listenOptions =>
    {
        listenOptions.UseHttps(cert);
    });
});
 builder.Services
     .AddKubernetesOperator()
     // Create the development webhook service using the cert provider
     .UseCertificateProvider(port, ip, generator)
     // More code for generation, controllers, etc.

The UseCertificateProvider method takes an ICertificateProvider interface, so it can be used to implement your own certificate generator/loader for development if necessary.