Is cross-platform deployment between Kamaji and Vsphere possible?

[Rezkmike]

Objective

I am currently exploring the feasibility of deploying a Kubernetes Control Plane using the Kamaji Control Plane on an EKS (Elastic Kubernetes Service) management cluster. The main goal is to set up the control plane on EKS and subsequently add a worker node hosted on a vSphere platform, leveraging Cluster API (CAPI).

Details

Primary Task:
Deploy the Control Plane using Kamaji on an AWS EKS management cluster.

Secondary Task:
Add a worker node to this setup, but the worker node should be on a vSphere platform.

Tools/Platforms Used:

• AWS Elastic Kubernetes Service (EKS) for the management cluster.
• Kamaji for deploying the Control Plane.
• vSphere platform for the worker node.
• Cluster API (CAPI) for orchestrating the worker node setup.

Questions

Q1 - Is this combination of technologies (EKS with Kamaji for the control plane and vSphere for worker nodes via CAPI) feasible and supported?

Q2 - If feasible, are there any specific configurations or considerations required to integrate these platforms effectively?

Q3 - Are there any known limitations or issues with this setup that I should be aware of?

Q4 - Any documentation or resources that can guide the implementation of such a setup would be highly appreciated.

---

Sample Output from CAPI

When trying to add Worker node on Vsphere and point to Kamaji Control Plane. The Worker node stuck at pending state

[prometherion]

@Rezkmike thanks for the questions, I just moved here as a discussion since it's not a bug report (for now 😆).

1. I never tested the infrastructure providers combination but Kamaji has been designed to support mixed infrastructure. When providing demos, the Tenant control planes are deployed in an AKS cluster, and the worker nodes are EC2 instances (AWS). Unless something is broken, it's expected to work since the worker nodes just need to access the Control Plane endpoint, nothing else.
2. Partially answered in the following bullet point, since you're expected to run on AWS, this blog post could be interesting for you, which explains all the required steps to get Kamaji working on the AWS infrastructure (such as the management cluster).
3. We discussed publicly the potential fallacies of this design here, although CLASTIX (the company behind the project) has a customer doing it as mentioned in this blog post: till you know what you're doing, it's safe.
4. Documentation is like love: the more, the better. We hadn't the time to write down this publicly but it should be "just" a matter of combining CAPI objects together, given for granted you have vSphere API accessibility from the Cluster API management cluster.

About the CAPI issues you reported, it seems the TCP is unreachable, a clusterctl describe cluster output would be more helpful, as well as the extended YAML output of all the objects involved.

tl;dr; I'm pretty sure you're facing the dilemma of the ELB, please, follow the blog post at point nr. 2 and you're going to solve it!

[Rezkmike]

01 - So mean the current combination/mixed infrastructure, the worker nodes joined manually using a token and not using CAPI?

[prometherion]

Both strategies, with Cluster API, or manually.

[prometherion]

A small update on this: such scenario can be covered with the ExternalClusterReference as discussed on #105.