Kubernetes turns 12 this 12 months. In that point, it’s gone from a Google aspect mission to the working system of recent infrastructure operating in every single place from mainframes to GPUs, throughout multi-cloud, hybrid, on-prem, and edge environments. The CNCF panorama has grown alongside it, filling within the gaps that Kubernetes left open.
This weblog isn’t about all of these gaps. It’s about one particular intersection: light-weight Kubernetes with K3s on an on-premise infrastructure (on this case Proxmox) and declarative multi-cluster administration with k0rdent.
For those who’ve run Kubernetes on on-prem infrastructure, you already know the ache:
- Guide VM creation
- Bash scripts that solely you perceive
- Clusters that work as soon as, after which grow to be untouchable
We wished one thing declarative, repeatable, and clear, however nonetheless pleasant to an on-prem setup. That’s the place k0rdent, Proxmox, and K3s got here collectively.
On this weblog, we’ll stroll by way of how we curated a use case for provisioning and used k0rdent to provision a K3s cluster on an On-premise surroundings by writing our personal Helm charts and utilizing k0rdent’s Convey Your Personal Template (BYOT) method. This isn’t a theoretical put up, that is precisely how our cluster will get created.
The Large Image
Right here’s what we got down to construct:
- Many end-users host their infrastructure layer on-premise, allow us to construct that for you.
- Present VM templates as an alternative of constructing photos each time
- k0rdent managing the total cluster lifecycle
- K3s because the Kubernetes bootstrap
At a excessive stage, the movement seems to be like this:
Person → k0rdent
↓
Proxmox Infrastructure (BYOT VMs)
↓
Management Airplane Supplier
↓
Bootstrap Supplier (K3s)
↓
Operating Kubernetes Cluster
Every layer does one job, and does it properly.
Why On-Prem + k0rdent Makes Sense
Proxmox is likely one of the examples of a self-hosted surroundings, however Kubernetes on-prem usually finally ends up being hand-crafted, exhausting to scale, and more durable to breed.
k0rdent adjustments that mindset. As a substitute of scripting how issues ought to occur, you describe what you need and let reconciliation do the work.
k0rdent gives a transparent separation of considerations:
- Infrastructure — provision the VMs
- Management Airplane — configure the cluster topology
- Bootstrap — set up and initialise Kubernetes
That separation made Proxmox integration surprisingly clear.
Step 1: Infrastructure Supplier (BYOT)
k0rdent doesn’t ship with a local Proxmox supplier, so we constructed one.
We created a customized Helm chart that acts as an Infrastructure Supplier for Proxmox.
Why Convey Your Personal Template?
As a substitute of dynamically constructing VM photos on each provision, I used present Proxmox VM templates. My templates already had cloud-init enabled, SSH entry configured, and base OS packages put in.
This gave me:
- Quicker VM provisioning — no picture builds within the essential path
- Higher management over OS hardening — managed exterior the Kubernetes workflow
- Simpler debugging — when one thing goes unsuitable, the VM layer is a identified amount
What the Helm Chart Does
The Proxmox infrastructure chart is deliberately scoped — it solely handles infrastructure:
- Talks to the Proxmox API
- Clones VMs from a template
- Units CPU, reminiscence, and networking
- Injects SSH keys
- Outputs VM metadata again to k0rdent
No Kubernetes logic right here. Simply clear VM provisioning.
Step 2: Management Airplane Supplier
Helm charts for the Proxmox supplier can be found right here:
As soon as the VMs exist, the Management Airplane Supplier takes over. Its job is to:
- Resolve which nodes are management airplane nodes
- Apply cluster-level configuration
- Coordinate with the bootstrap supplier
In our setup, management airplane nodes run on Proxmox VMs, with VM particulars flowing immediately from the infrastructure supplier.
Roles are assigned declaratively, the cluster feels intentional, not unintentional.
Step 3: Bootstrapping Kubernetes with K3s
For bootstrapping, I selected K3s — and it suits completely right here.
Why K3s? It’s light-weight, quick to put in, has minimal dependencies, and is purpose-built for on-prem and edge environments.
Right here’s what the supplier definitions seem like:
apiVersion: operator.cluster.x-k8s.io/v1alpha2
form: BootstrapProvider
metadata:
identify: k3s
spec:
model: v0.3.0
fetchConfig:
url:
.Values.configSecret.namespace
configSecret:
identify: .Values.configSecret.namespace
namespace: trunc 63
{ default .Launch.Namespace }—
apiVersion: operator.cluster.x-k8s.io/v1alpha2
form: ControlPlaneProvider
metadata:
identify: k3s
spec:
model: v0.3.0
fetchConfig:
url:
{{- if .Values.configSecret.identify }}
configSecret:
identify: {{ .Values.configSecret.identify }}
namespace: { trunc 63 }
{{- finish }}What the Bootstrap Supplier Does
The Bootstrap Supplier Helm chart handles the K3s lifecycle:
- Installs K3s on the primary management airplane node
- Extracts the cluster token
- Joins further management airplane and employee nodes
- Generates kubeconfig entry
As soon as this step completes, Kubernetes is up and operating.
How k0rdent Ties It All Collectively
By utilizing this methodology, the system constantly reconciles the specified state:
- Provisions Proxmox VMs
- Waits for them to grow to be reachable
- Passes VM knowledge to the management airplane supplier
- Triggers the K3s bootstrap course of
- Retains watching — and corrects drift
The consequence: a absolutely declarative and managed K3s cluster in your on-premise surroundings.
What You Finish Up With
After all the things settles, you have got:
- A working K3s cluster operating on On-prem VMs
- Management airplane and employee nodes provisioned declaratively
- A setup you possibly can recreate, scale, or tear down at any time
Scaling the cluster is now not a weekend mission. Rebuilding it’s now not terrifying. It’s simply configuration.
Operating Kubernetes on-prem doesn’t need to imply duct tape and bash scripts. With k0rdent’s BYOT method, your On-prem infrastructure turns into a first-class citizen; declarative, reproducible, and handled as a first-class citizen.. In case your infrastructure isn’t supported out of the field, construct the template. That’s the entire level.
