Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit 47e7ee1

Browse files
authoredFeb 27, 2024··
Merge branch 'master' into 414-Allow-kubeApiServer-extraArgs-override
2 parents cee6cba + eff68db commit 47e7ee1

9 files changed

+183
-42
lines changed
 

‎ADOPTERS.md

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Adopters
2+
3+
This is a list of companies that have adopted Kamaji.
4+
Feel free to open a Pull-Request to get yours listed.
5+
6+
### Adopter list (alphabetically)
7+
8+
| Type | Name | Since | Website | Use-Case |
9+
|:-|:-|:-|:-|:-|
10+
| Vendor | Ænix | 2023 | [link](https://aenix.io/) | Ænix provides consulting services for cloud providers and uses Kamaji for running Kubernetes-as-a-Service in free PaaS platform [Cozystack](https://cozystack.io). |
11+
| Vendor | Netsons | 2023 | [link](https://www.netsons.com) | Netsons is an Italian hosting and cloud provider and uses Kamaji in its [Managed Kubernetes](https://www.netsons.com/kubernetes) offering. |
12+
13+
### Adopter Types
14+
15+
**End-user**: The organization runs Kamaji in production in some way.
16+
17+
**Integration**: The organization has a product that integrates with Kamaji, but does not contain Kamaji.
18+
19+
**Vendor**: The organization packages Kamaji in their product and sells it as part of their product.

‎README.md

+130-19
Original file line numberDiff line numberDiff line change
@@ -3,45 +3,156 @@
33
<p align="left">
44
<img src="https://img.shields.io/github/license/clastix/kamaji"/>
55
<img src="https://img.shields.io/github/go-mod/go-version/clastix/kamaji"/>
6-
<a href="https://github.com/clastix/kamaji/releases">
7-
<img src="https://img.shields.io/github/v/release/clastix/kamaji"/>
8-
<img src="https://goreportcard.com/badge/github.com/clastix/kamaji">
9-
</a>
6+
<a href="https://github.com/clastix/kamaji/releases"><img src="https://img.shields.io/github/v/release/clastix/kamaji"/></a>
7+
<img src="https://goreportcard.com/badge/github.com/clastix/kamaji">
8+
<a href="https://kubernetes.slack.com/archives/C03GLTTMWNN"><img alt="#kamaji on Kubernetes Slack" src="https://img.shields.io/badge/slack-@kubernetes/kamaji-blue.svg?logo=slack"/></a>
109
</p>
1110

1211
![Logo](assets/logo-black.png#gh-light-mode-only)
1312
![Logo](assets/logo-white.png#gh-dark-mode-only)
1413

15-
**Kamaji** is a **Kubernetes Control Plane Manager**. It operates Kubernetes at scale with a fraction of the operational burden. Kamaji is special because the Control Plane components are running inside pods instead of dedicated machines. This solution makes running multiple Control Planes cheaper and easier to deploy and operate.
14+
### 🤔 What is Kamaji?
1615

17-
<img src="docs/content/images/architecture.png" width="600">
16+
**Kamaji** is a **Kubernetes Control Plane Manager** leveraging on the concept of [**Hosted Control Plane**](https://clastix.io/post/the-raise-of-hosted-control-plane-in-kubernetes/).
1817

19-
## Main Features
18+
Kamaji's approach is based on running the Kubernetes Control Plane components in Pods instead of dedicated machines.
19+
This allows operating Kubernetes clusters at scale, with a fraction of the operational burden.
20+
Thanks to this approach, running multiple Control Planes can be cheaper and easier to deploy and operate.
2021

21-
- **Multi-cluster Management:** centrally manage multiple Kubernetes clusters from a single Management Cluster.
22+
_Kamaji is like a fleet of Site Reliability Engineers with expertise codified into its logic, working 24/7 to keep up and running your Control Planes._
23+
24+
<img src="docs/content/images/architecture.png" width="600" style="display: block; margin: 0 auto">
25+
26+
### 📖 How it works
27+
28+
Kamaji is extending the Kubernetes API capabilities thanks to [Custom Resource Definitions](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions).
29+
30+
By installing Kamaji, two pairs of new APIs will be available:
31+
32+
- `TenantControlPlane`, the instance definition of your desired Kubernetes Control Plane
33+
- `Datastore`, the backing store used by one (or more) `TenantControlPlane`
34+
35+
The `TenantControlPlane` (short-named as `tcp`) objects are Namespace-scoped and allows configuring every aspect of your desired Control Plane.
36+
Besides the Kubernetes configuration values, you can specify the Pod options such as limit, request, tolerations, node selector, etc.,
37+
as well as how these should be exposed (e.g.: using a `ClusterIP`, a `LoadBalancer`, or a `NodePort`).
38+
39+
The `TenantControlPlane` is the stateless definition of the Control Plane allowing to set up the required components for a full-fledged Kubernetest cluster.
40+
The state is managed by the `Datastore` API, a cluster-scoped resource which can hold the data of one or more Kubernetes clusters.
41+
42+
> For further information about the API specifications and all the available options,
43+
> refer to the official [API reference](https://kamaji.clastix.io/reference/api/#tenantcontrolplane).
44+
45+
### ⭐️ Main features
46+
47+
- **Fast provisioning time**: depending on the infrastructure, Tenant Control Planes are up and ready to serve traffic in **16 seconds**.
48+
- **Streamlined update**: the rollout to a new Kubernetes version for a given Tenant Control Plane takes just **10 seconds**, with a Blue/Green deployment to avoid serving mixed Kubernetes versions.
49+
- **Resource optimization**: thanks to the Datastore decoupling, there's no need of odd number instances (e.g.: RAFT consensus) by allowing to save up to 60% of HW resources.
50+
- **Scale from zero to the moon**: scale down the instance when there's no usage, or automatically scale to support the traffic spikes reusing the Kubernetes patterns.
51+
- **Declarative approach, constant reconciliation**: thanks to the Operator pattern, drift detection happens in real-time, maintaining the desired state.
52+
- **Automated certificates management**: Kamaji leverages on `kubeadm` and the certificates are automatically created and rotated for you.
53+
- **Managing core addons**: Kamaji allows configuring automatically `kube-proxy`, `CoreDNS`, and `konnectivity`, with automatic remediation in case of user errors (e.g.: deleting the `CoreDNS` deployment).
54+
- **Auto Healing**: the `TenantControlPlane` objects in the management cluster are tracked by Kamaji, in case of deletion of those, everything is created in an idempotent way.
55+
- **Datastore multi-tenancy**: optionally, Kamaji allows running multiple Control Planes on the same _Datastore_ instance leveraging on the multi-tenancy of each driver, decreasing operations and optimizing costs.
56+
- **Overcoming `etcd` limitations**: optionally, Kamaji allows using a different _Datastore_ thanks to [`kine`](https://github.com/k3s-io/kine) by supporting `MySQL` or `PostgreSQL` as an alternative.
57+
- **Simplifying mixed-networks setup**: thanks to [`Konnectivity`](https://kubernetes.io/docs/tasks/extend-kubernetes/setup-konnectivity/),
58+
the Tenant Control Plane is connected to the worker nodes hosted in a different network, overcoming the no-NAT availability when dealing with nodes with a non routable IP address
59+
(e.g.: worker nodes in a different infrastructure).
60+
61+
### 🚀 Use cases
62+
63+
- [**Creating a private Managed Kubernetes Service**](https://clastix.io/post/netsons-builds-a-managed-kubernetes-service-with-kamaji-and-open-stack/)
64+
- [**Building a Platform as a Service**](https://aenix.io/cozystack/)
65+
- [**Overcoming public Managed Kubernetes Services**](https://clastix.io/post/overcoming-eks-limitations-with-kamaji-on-aws/) such as EKS
66+
- [**Hybrid infrastructures**](https://clastix.io/post/bridging-the-gap-hybrid-kubernetes-clusters-with-remote-control-planes/):
67+
host the Control Plane on the Cloud and worker nodes on prem or vice-versa, according to your needs.
68+
- [**Kubernetes at the edge**](https://clastix.io/post/edgevolution-unleashing-the-power-of-kubernetes-clusters-for-a-revolutionary-edge-computing-experience/):
69+
take full advantage of the _Kubernetes API Server as a service_ paradigm.
70+
- **Kubernetes Control Plane as a Service:** centrally manage multiple Kubernetes clusters from a single management point (_Multi-Cluster management_).
2271
- **High-density Control Plane:** place multiple control planes on the same infrastructure, instead of having dedicated machines for each control plane.
2372
- **Strong Multi-tenancy:** leave users to access the control plane with admin permissions while keeping them isolated at the infrastructure level.
2473
- **Kubernetes Inception:** use Kubernetes to manage Kubernetes with automation, high-availability, fault tolerance, and autoscaling out of the box.
25-
- **Bring Your Own Device:** keep the control plane isolated from data plane. Worker nodes can join and run consistently everywhere: cloud, edge, and data-center.
74+
- **Bring Your Own Device:** keep the control plane isolated from data plane. Worker nodes can join and run consistently from everywhere: cloud, edge, and data-center.
2675
- **Full CNCF compliant:** all clusters are built with upstream Kubernetes binaries, resulting in full CNCF compliant Kubernetes clusters.
2776

28-
## Roadmap
77+
> 🤔 You'd like to do the same but don't know how?
78+
> 💡 [CLASTIX](https://clastix.io/) can help you with your needs!
79+
80+
### 🧑‍💻‍ Production grade
81+
82+
Kamaji is empowering several businesses, and it counts public adopters.
83+
Check out the [adopters](./ADOPTERS.md) file to learn more.
84+
85+
> 🤗 If you're using Kamaji, share your love by opening a PR!
86+
87+
### 🍦 Vanilla Kubernetes clusters
88+
89+
Kamaji is **not** yet-another-Kubernetes distribution: you have full freedom on the technology stack to provide to end users.
90+
Kamaji is a perfect fit for Platform Engineering, hiding the complexity of the Control Plane management to developers and DevOps engineers.
91+
92+
The provided Kubernetes Control Planes are [CNCF compliant clusters](https://kamaji.clastix.io/reference/conformance/).
93+
94+
<img src="https://raw.githubusercontent.com/cncf/artwork/master/projects/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" style="display: block; width: 75px; margin: 0 auto">
95+
96+
### 🐢 Cluster API support
97+
98+
Kamaji is **not** a [Cluster API](https://cluster-api.sigs.k8s.io/) replacement, rather, it plays very well with it.
99+
100+
Since Kamaji is just focusing on the Control Plane a [Kamaji's Cluster API Control Plane provider](https://github.com/clastix/cluster-api-control-plane-provider-kamaji) has been developed.
101+
102+
### 🛣️ Roadmap
29103

30104
- [x] Dynamic address on Load Balancer
31105
- [x] Zero Downtime Tenant Control Plane upgrade
32-
- [x] Join worker nodes from anywhere
33-
- [x] Alternative datastore MySQL and PostgreSQL
34-
- [x] Pool of multiple datastores
35-
- [x] Seamless migration between datastores
106+
- [x] [Join worker nodes from anywhere thanks to Konnectivity](https://kamaji.clastix.io/concepts/#konnectivity)
107+
- [x] [Alternative datastore MySQL and PostgreSQL](https://kamaji.clastix.io/guides/alternative-datastore/)
108+
- [x] [Pool of multiple datastores](https://kamaji.clastix.io/concepts/#datastores)
109+
- [x] [Seamless migration between datastores](https://kamaji.clastix.io/guides/datastore-migration/)
36110
- [ ] Automatic assignment to a datastore
37111
- [ ] Autoscaling of Tenant Control Plane
38-
- [x] Provisioning through Cluster APIs
112+
- [x] [Provisioning through Cluster APIs](https://github.com/clastix/cluster-api-control-plane-provider-kamaji)
39113
- [ ] Terraform provider
40114
- [ ] Custom Prometheus metrics
41115

116+
### 🎥 Multimedia
117+
118+
- YouTube ▶️ [Hands-on introduction to Kamaji](https://www.youtube.com/watch?v=HhevxwQWQ88)
119+
- YouTube ▶️ [Scaling Kubernetes up to 1,000 Control Planes](https://www.youtube.com/watch?v=W_HXRXJh96U)
120+
- YouTube ▶️ [Equinix, Kamaji, and Cluster API](https://www.youtube.com/watch?v=TLBTqROj_wA)
121+
- YouTube ▶️ [Rancher & Kamaji: solving multitenancy challenges in the Kubernetes world](https://www.youtube.com/watch?v=VXHNrMmlF8U)
122+
- YouTube ▶️ [Enabling Self-Service Kubernetes clusters with Kamaji and Paralus](https://www.youtube.com/watch?v=JWA2LwZazM0)
123+
124+
### 🏷️ Versioning
125+
126+
Versioning adheres to the [Semantic Versioning](http://semver.org/) principles.
127+
A full list of the available releases is available in the GitHub repository's [**Release** section](https://github.com/clastix/kamaji/releases).
128+
129+
### 📄 Documentation
130+
131+
Further documentation can be found on the official [Kamaji documentation website](https://kamaji.clastix.io/).
132+
133+
### 🤝 Contributions
134+
135+
Contributions are highly appreciated and very welcomed!
136+
137+
In case of bugs, please, check if the issue has been already opened by checking the [GitHub Issues](https://github.com/clastix/kamaji/issues) section.
138+
In case it isn't, you can open a new one: a detailed report will help us to replicate it, assess it, and work on a fix.
139+
140+
You can express your intention in working on the fix on your own.
141+
The commit messages are checked according to the described [semantics](https://github.com/projectcapsule/capsule/blob/main/CONTRIBUTING.md#semantics).
142+
Commits are used to generate the changelog, and their author will be referenced in it.
143+
144+
In case of **✨ Feature Requests** please use the [Discussion's Feature Request section](https://github.com/clastix/kamaji/discussions/categories/feature-requests).
145+
146+
### 📝 License
147+
148+
The Kamaji Cluster API Control Plane provider is licensed under Apache 2.0.
149+
The code is provided as-is with no warranties.
150+
151+
### 🛟 Commercial Support
152+
153+
![CLASTIX](https://avatars.githubusercontent.com/u/39170129?s=50&v=4) [CLASTIX](https://clastix.io/) is the commercial company behind Kamaji and the Cluster API Control Plane provider.
42154

43-
## Documentation
44-
Please, check the project's [documentation](https://kamaji.clastix.io/) for getting started with Kamaji.
155+
If you're looking to run Kamaji in production and would like to learn more, **CLASTIX** can help by offering [Open Source support plans](https://clastix.io/support),
156+
as well as providing a comprehensive Enterprise Platform named [CLASTIX Enterprise Platform](https://clastix.cloud/), built on top of the Kamaji and [Capsule](https://capsule.clastix.io/) project (now donated to CNCF as a Sandbox project).
45157

46-
## Contributions
47-
Kamaji is Open Source with Apache 2 license and any contribution is welcome. Open an issue or suggest an enhancement on the GitHub [project's page](https://github.com/clastix/kamaji). Join the [Kubernetes Slack Workspace](https://slack.k8s.io/) and the [`#kamaji`](https://kubernetes.slack.com/archives/C03GLTTMWNN) channel to meet end-users and contributors.
158+
Feel free to get in touch with the provided [Contact form](https://clastix.io/contact).

‎charts/kamaji/Chart.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ name: kamaji
1515
sources:
1616
- https://github.com/clastix/kamaji
1717
type: application
18-
version: 0.14.1
18+
version: 0.15.0
1919
annotations:
2020
catalog.cattle.io/certified: partner
2121
catalog.cattle.io/release-name: kamaji

‎charts/kamaji/README.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# kamaji
22

3-
![Version: 0.14.1](https://img.shields.io/badge/Version-0.14.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.4.1](https://img.shields.io/badge/AppVersion-v0.4.1-informational?style=flat-square)
3+
![Version: 0.15.0](https://img.shields.io/badge/Version-0.15.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.4.1](https://img.shields.io/badge/AppVersion-v0.4.1-informational?style=flat-square)
44

55
Kamaji is a Kubernetes Control Plane Manager.
66

@@ -66,6 +66,8 @@ Here the values you can override:
6666
| Key | Type | Default | Description |
6767
|-----|------|---------|-------------|
6868
| affinity | object | `{}` | Kubernetes affinity rules to apply to Kamaji controller pods |
69+
| cfssl.image.repository | string | `"cfssl/cfssl"` | |
70+
| cfssl.image.tag | string | `"latest"` | |
6971
| datastore.basicAuth.passwordSecret.keyPath | string | `nil` | The Secret key where the data is stored. |
7072
| datastore.basicAuth.passwordSecret.name | string | `nil` | The name of the Secret containing the password used to connect to the relational database. |
7173
| datastore.basicAuth.passwordSecret.namespace | string | `nil` | The namespace of the Secret containing the password used to connect to the relational database. |

‎charts/kamaji/templates/etcd_job_preinstall.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ spec:
1919
restartPolicy: Never
2020
initContainers:
2121
- name: cfssl
22-
image: cfssl/cfssl:latest
22+
image: "{{ .Values.cfssl.image.repository }}:{{ .Values.cfssl.image.tag }}"
2323
command:
2424
- bash
2525
- -c

‎charts/kamaji/values.yaml

+5
Original file line numberDiff line numberDiff line change
@@ -214,3 +214,8 @@ datastore:
214214
namespace:
215215
# -- Key of the Secret which contains the content of the private key.
216216
keyPath:
217+
218+
cfssl:
219+
image:
220+
repository: cfssl/cfssl
221+
tag: latest

‎controllers/certificate_lifecycle_controller.go

+9-6
Original file line numberDiff line numberDiff line change
@@ -40,12 +40,16 @@ func (s *CertificateLifecycle) Reconcile(ctx context.Context, request reconcile.
4040
logger.Info("starting CertificateLifecycle handling")
4141

4242
secret := corev1.Secret{}
43-
if err := s.client.Get(ctx, request.NamespacedName, &secret); err != nil {
44-
if k8serrors.IsNotFound(err) {
45-
logger.Info("resource may have been deleted, skipping")
43+
err := s.client.Get(ctx, request.NamespacedName, &secret)
44+
if k8serrors.IsNotFound(err) {
45+
logger.Info("resource have been deleted, skipping")
4646

47-
return reconcile.Result{}, nil
48-
}
47+
return reconcile.Result{}, nil
48+
}
49+
if err != nil {
50+
logger.Error(err, "cannot retrieve the required resource")
51+
52+
return reconcile.Result{}, err
4953
}
5054

5155
checkType, ok := secret.GetLabels()[constants.ControllerLabelResource]
@@ -56,7 +60,6 @@ func (s *CertificateLifecycle) Reconcile(ctx context.Context, request reconcile.
5660
}
5761

5862
var crt *x509.Certificate
59-
var err error
6063

6164
switch checkType {
6265
case "x509":

‎controllers/datastore_controller.go

+7-5
Original file line numberDiff line numberDiff line change
@@ -39,12 +39,14 @@ func (r *DataStore) Reconcile(ctx context.Context, request reconcile.Request) (r
3939
log := log.FromContext(ctx)
4040

4141
ds := &kamajiv1alpha1.DataStore{}
42-
if err := r.Client.Get(ctx, request.NamespacedName, ds); err != nil {
43-
if k8serrors.IsNotFound(err) {
44-
return reconcile.Result{}, nil
45-
}
42+
err := r.Client.Get(ctx, request.NamespacedName, ds)
43+
if k8serrors.IsNotFound(err) {
44+
log.Info("resource have been deleted, skipping")
4645

47-
log.Error(err, "unable to retrieve the request")
46+
return reconcile.Result{}, nil
47+
}
48+
if err != nil {
49+
log.Error(err, "cannot retrieve the required resource")
4850

4951
return reconcile.Result{}, err
5052
}

‎controllers/tenantcontrolplane_controller.go

+8-9
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ import (
1515
batchv1 "k8s.io/api/batch/v1"
1616
corev1 "k8s.io/api/core/v1"
1717
networkingv1 "k8s.io/api/networking/v1"
18-
apimachineryerrors "k8s.io/apimachinery/pkg/api/errors"
18+
k8serrors "k8s.io/apimachinery/pkg/api/errors"
1919
k8stypes "k8s.io/apimachinery/pkg/types"
2020
"k8s.io/client-go/util/workqueue"
2121
"k8s.io/utils/clock"
@@ -84,16 +84,15 @@ func (r *TenantControlPlaneReconciler) Reconcile(ctx context.Context, req ctrl.R
8484
defer cancelFn()
8585

8686
tenantControlPlane, err := r.getTenantControlPlane(ctx, req.NamespacedName)()
87-
if err != nil {
88-
if apimachineryerrors.IsNotFound(err) {
89-
log.Info("resource may have been deleted, skipping")
90-
91-
return ctrl.Result{}, nil
92-
}
87+
if k8serrors.IsNotFound(err) {
88+
log.Info("resource have been deleted, skipping")
9389

94-
log.Error(err, "cannot retrieve the required instance")
90+
return reconcile.Result{}, nil
91+
}
92+
if err != nil {
93+
log.Error(err, "cannot retrieve the required resource")
9594

96-
return ctrl.Result{}, err
95+
return reconcile.Result{}, err
9796
}
9897

9998
releaser, err := mutex.Acquire(r.mutexSpec(tenantControlPlane))

0 commit comments

Comments
 (0)
Please sign in to comment.