Behaviour of CAPI KamajiControlPlane provider when no infrastructure is used

[joseluisgonzalezca]

Hello everyone!

As I stated in this discussion, #88 (reply in thread), I’m trying to understand how CAPI works and the building blocks that compose it while using Kamaji provider.

My idea is to use the CAPI operator to create a Kamaji control plane. I’m going to leave the infrastructure part empty because I want to manage the workers independently.

In my management cluster, I installed the CAPI operator using Helm and then installed the provider using this YAML manifest:

apiVersion: operator.cluster.x-k8s.io/v1alpha2
kind: ControlPlaneProvider
metadata:
  name: kamaji
  namespace: capi-operator-system
spec:
  manager:
    featureGates:
      SkipInfraClusterPatch: true

It’s important to highlight the fact that I used the feature gate to be able to deploy the control plane without the obligation to specify an infrastructure provider (see PR #147). Once that part is working correctly, I created a Cluster resource that uses a KamajiControlPlane resource as a reference for the control plane. Here’s the example manifest:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: test-kamaji
  namespace: default
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
        - 10.244.0.0/16
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1alpha1
    kind: KamajiControlPlane
    name: test-kamaji-control-plane
---
apiVersion: controlplane.cluster.x-k8s.io/v1alpha1
kind: KamajiControlPlane
metadata:
  name: test-kamaji-control-plane
  namespace: default
spec:
  addons:
    coreDNS: {}
  controlPlaneEndpoint:
    host: 172.22.3.30
    port: 6443
  dataStoreName: default
  kubelet:
    cgroupfs: systemd
    preferredAddressTypes:
    - InternalIP
  network:
    serviceType: ClusterIP
  replicas: 3
  version: 1.30.0

As far as I understand, once the KamajiControlPlane is deployed, Cluster resource should be marked as Provisioned because no infra is required. However, it's stuck at Provisioning state because of this:

capiv1beta1.Cluster Control Plane endpoint still unprocessed, enqueuing back

As the feature gate is enabled, Cluster resource is not patched accordingly and this error is received. If I configure controlPlaneEndpoint param in the Cluster resource instead of in the KamajiControlPlane resource, Cluster state is updated correctly to Provisioned.

After seeing that, I don't see benefit of PR #137. What problem does solve including spec.controlPlaneEndpoint parameter in KamajiControlPlane resource?

Sorry for my ignorance. I just started learning about CAPI and some of the concepts are a little bit confusing for me.

Thanks for your help and time!

[prometherion]

Managing worker nodes still requires an Infrastructure Cluster, AFAIK.

Have you already picked up an infrastructure provider for worker nodes? According to your needs, Cluster API could be too much of overhead since it's designed to manage the entire lifecycle of a cluster.

[joseluisgonzalezca]

Yeah, I know CAPI will be a little overkill in this situation. I'm only using it for the sake of learning how it works. What I meant about managing worker nodes externally it's that i already have some tools to provision and configure them. I'm mainly interesed in using Kamaji together with CAPI for the controlplane part.

As I mentioned earlier with #147 and #137 PRs I understand that, although CAPI is mainly used to manage K8S cluster with all their components, I should be able to utilize it only to handle controlplane generation.

Correct me if I'm mistaken. As I said before, I'm starting to use this kind of solutions.

Thank you for your time

[prometherion]

Starting with Kamaji and CAPI could be a bit cumbersome.

I would suggest you to start simple, by picking up an infrastructure of your wish, creating nodes, and then decoupling the Control Plane.

Bare Cluster API objects are useless, unless you have a very specific and advanced approach where you manage those on your own (#147).

[joseluisgonzalezca]

I understand that it's a little bit strange to create CAPI objects to manage only the control plane part. That's why I don't see clearly what the goal of PR #147 is in this context. Could you please explain it in more detail?

[prometherion]

Could you please explain it in more detail?

As described earlier, it's useful if you have your own custom infrastructure provider and rely on retrieving the Control Plane provider. It's an advanced flag for Cluster API infrastructure developers.