CanSupportMe.Operator 1.0.0

CanSupportMe.Operator

Overview

What is this?

This package will allow you to create a simple Kubernetes Operator in .NET.
It can pick up changes to CRDs, ConfigMaps or Secrets and say if it was added, modified or deleted. It can check if a CRD exists as well as retrieve, create, replace and delete configmaps and secrets. It can also update the status of any manifest that supports it.

What isn't this?

It is not a full blown Kubernetes Operator SDK. These exist and, if desired, this package can be used in conjunction with them. It has been developed to do what the project required and nothing more however this will likely fit the requirements of many similar projects too.

Getting Started

Prerequisites

• .NET 8 or later
• A Kubernetes cluster (either local or remote)
• Sufficient access to the cluster to read and watch secrets (for the demo)
• The ability to create a service account and cluster role binding with access to the resources you will be watching or creating
• A suitable pair of test secrets, created using the following bash script:

  kubectl apply -f - <<EOF
  apiVersion: v1
  kind: Secret
  metadata:
    name: test-secret-with-label
    namespace: default
    labels:
      app.kubernetes.io/managed-by: DemoOperator
  stringData:
    notImportant: SomeValue
  type: Opaque
  ---
  apiVersion: v1
  kind: Secret
  metadata:
    name: test-secret-without-label
    namespace: default
  stringData:
    notImportant: SomeValue
  type: Opaque
  EOF
  

Create a new .NET console application

Create a new .NET 8 console application and call it CanSupportMe.Operator.Demo.

Add the following packages from NuGet:

• Microsoft.Extensions.Configuration.Abstractions
• Microsoft.Extensions.Hosting
• Microsoft.Extensions.Hosting.Abstractions
• Serilog.AspNetCore

Note that the Serilog.AspNetCore package is not required but is used in this demo to show how to produce nicer logs.

Install the package

dotnet add package CanSupportMe.Operator

Create a simple operator

Replace the contents of your Program.cs with the following:

using CanSupportMe.Operator.Extensions;
using CanSupportMe.Operator.Options;
using Microsoft.Extensions.Hosting;
using Serilog;
using Serilog.Events;

// const string FAILOVER_TOKEN = "<YOUR_TOKEN_GOES_HERE>";

Log.Logger = new LoggerConfiguration()
	.MinimumLevel.Debug()
	.MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
	.MinimumLevel.Override("System", LogEventLevel.Warning)
	.WriteTo.Console()
	.CreateLogger();

try
{
	Log.Information("Application starting up");

	IHost host = Host.CreateDefaultBuilder(args)
		.ConfigureServices((context, services) =>
		{
			services.AddOperator(options =>
			{
				options.Group = "";
				options.Kind = "Secret";
				options.Version = "v1";
				options.Plural = "secrets";
				options.Scope = ResourceScope.Namespaced;
				options.LabelFilters.Add("app.kubernetes.io/managed-by", "SeqApiKeyOperator");

				options.OnAdded = (kind, name, @namespace, item, dataObjectManager) =>
				{
					Log.Information("On {Kind} Add: {Name} to {Namespace} which is of type {Type} with {ItemCount} item(s)", kind, name, @namespace, item.Type, item.Data?.Count);
				};

				// options.FailoverToken = FAILOVER_TOKEN;
			});
		})
		.UseSerilog()
		.Build();

	host.Run();
}
catch (Exception ex)
{
	Log.Fatal(ex, "Application start failed");
}
finally
{
	Log.CloseAndFlush();
}

Running the project will show it connecting to the cluster and watching for changes to secrets. It will then show test-secret-with-label as being added and not mention test-secret-without-label.

Retrieve service token

If a options.FailoverToken is required, one can be generated using the following bash script:

export SERVICEACCOUNT="cansupportme-operator"

kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: ${SERVICEACCOUNT}-token
  namespace: default
  annotations:
    kubernetes.io/service-account.name: $SERVICEACCOUNT
type: kubernetes.io/service-account-token
EOF

export TOKEN=$(kubectl -n default get secret ${SERVICEACCOUNT}-token -o jsonpath='{.data.token}' | base64 --decode)
echo $TOKEN

Uncomment the two commented out lines of code in the above snippet and paste in the key generated above into the FAILOVER_TOKEN constant.

Listening for Custom Resource Definition (CRD) Changes

The more likely scenario for an operator is to listen out for CRD changes. The following example shows the simple file you need to create and how to configure the operator. To set up a CRD, apply the following CRD manifest to your cluster:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  # name must match the spec fields below, and be in the form: <plural>.<group>
  name: apikeys.demo.cansupport.me
spec:
  # group name to use for REST API: /apis/<group>/<version>
  group: demo.cansupport.me
  # list of versions supported by this CustomResourceDefinition
  versions:
    - name: v1alpha1
      # Each version can be enabled/disabled by Served flag.
      served: true
      # One and only one version must be marked as the storage version.
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                secretName:
                  type: string
                  x-kubernetes-validations:
                    - rule: self == oldSelf
                      message: "The secret name cannot be changed once set"
              required:
                - secretName
            status:
              type: object
              properties:
                status:
                  type: string
                  default: "New"
                  enum:
                    - "Ready"
                    - "Reconciling"
                    - "Failed"
                    - "New"
                    - "Deleting"
      additionalPrinterColumns:
        - name: Secret Name
          type: string
          description: The name of the secret that contains the API Key
          jsonPath: .spec.secretName
        - name: Status
          type: string
          description: The status of the Seq API key being reconciled
          jsonPath: .status.status
  # either Namespaced or Cluster
  scope: Namespaced
  names:
    # plural name to be used in the URL: /apis/<group>/<version>/<plural>
    plural: apikeys
    # singular name to be used as an alias on the CLI and for display
    singular: apikey
    # kind is normally the CamelCased singular type. Your resource manifests use this.
    kind: ApiKey
    # shortNames allow shorter string to match your resource on the CLI
    shortNames:
    - ak

Create a new file called MyCrd.cs in the root of the project and add the following:

using CanSupportMe.Operator.Interfaces;
using CanSupportMe.Operator.Models;
using System.Text.Json.Serialization;

namespace CanSupportMe.Operator.Demo;

public class MyCrd : KubernetesResource<MySpec, MyStatus>
{
}

public class MySpec : IKubernetesSpec
{
	[JsonPropertyName("secretName")]
	public required string SecretName { get; set; }
}

public class MyStatus : IKubernetesStatus
{
	[JsonPropertyName("status")]
	public required string Status { get; set; }
}

Then, in Program.cs add the following:

services.AddOperator<MySpec, MyStatus>(options =>
{
	options.OperatorName = "SeqApiKeyOperator";

	options.Group = "demo.cansupport.me";
	options.Kind = "ApiKey";
	options.Version = "v1alpha1";
	options.Plural = "apikeys";
	options.Scope = ResourceScope.Namespaced;

	options.OnReconcile = async (kind, name, @namespace, item, dataObjectManager) =>
	{
		Log.Information("On {kind} reconcile: {Name} to {Namespace} to create secret called {SecretName}", kind, name, @namespace, item.Spec.SecretName);

		var existingSecret = await dataObjectManager.Get("Secret", item.Spec.SecretName, @namespace);
		if (existingSecret.IsSuccess)
		{
			Log.Debug("Secret {SecretName} already exists in {Namespace} and has a value of {ExistingValue}", 
				existingSecret.Value.Metadata.Name, 
				existingSecret.Value.Metadata.Namespace,
				existingSecret.Value.Data["myApiKey"]);
		}
		else
		{
			Log.Debug("Secret {SecretName} does not exist in {Namespace}", item.Spec.SecretName, @namespace);

			await dataObjectManager.Create("Secret", item.Spec.SecretName, @namespace, new Dictionary<string, string>
			{
				{ "myApiKey", $"Some random string at {DateTime.UtcNow}" }
			}, labels: new()
			{
				{ "cansupport.me/owned-by", name }
			});
		}
	};

	options.FailoverToken = FAILOVER_TOKEN;
});