Skip to content

Changelog ​

Every release's changes, newest first. Versions follow Semantic Versioning; before 1.0.0 a minor version may change behaviour.

[0.1.0] - unreleased ​

The first release.

  • Appliance. One script on a Proxmox VE 9 node installs a small unprivileged container running Kubehenge's web interface. Users sign in with their Proxmox VE accounts, any realm, and act with their own Proxmox VE permissions. The appliance upgrades itself from a button, or automatically within a major version, and installs only releases signed with the project's release key. Its certificate is signed by the Proxmox VE cluster's CA, or is one of your own; it can carry a DNS name and drop it again, is renewed from the Proxmox VE shell, and the interface warns before it runs out. A Proxmox VE API whose certificate comes from another CA works too, with that CA given to the installer.
  • Management cluster. Set up from the interface (or with the OpenTofu module in management-cluster/): a Talos Linux cluster of one or three VMs, spread over the Proxmox VE nodes chosen, running Cluster API, the Proxmox VE provider, the kubeadm, RKE2 and k3s providers, IP address management and the Helm add-on provider. The Proxmox VE provider runs as Kubehenge's own reproducible, signed build of an upstream commit after its v0.9.1 release, which carries the fix kubeadm upgrades need, until a release of it does. A distribution's controllers run only while a cluster uses it. The controllers hold their leader leases through API stalls of up to 40 seconds, as when Proxmox VE clones a disk on the same storage, instead of restarting, and every controller has a memory request, so none is the first a node short of memory stops.
  • Workload clusters from three ClusterClasses, kubeadm, RKE2 and k3s: create, scale, upgrade by one minor version at a time, replace failed nodes, delete. Each cluster gets the Proxmox VE cloud controller manager, the Proxmox CSI plugin, a control-plane virtual IP with kube-vip, and a CNI: the distribution's own, or Cilium or Calico chosen at creation. Pod and service ranges, OpenID Connect sign-in and automatic upgrades within windows are set per cluster. While a cluster has workers, its control-plane nodes are tainted in every distribution and run only what has to run there; a cluster without workers runs its workloads on them.
  • Placement. A cluster's VMs spread over every online Proxmox VE node, or over the nodes chosen by hand or from one of Proxmox VE's HA node-affinity rules, and the choice can change later. A node that goes down is left out within about 15 seconds, so the VMs replacing those it ran go elsewhere. The cluster page shows where each VM runs and warns when control-plane VMs share a node. Where the administrator switches on Proxmox VE HA, it keeps every cluster's control-plane VMs on separate nodes and restarts them elsewhere after a node fails, for clusters on shared storage that do not opt out.
  • Isolation between clusters. Each cluster has its own Proxmox VE pool, user and API token with rights on its own VMs only, and a VM that owns its persistent volumes, so a compromised cluster cannot reach another's VMs or volumes. Only the cloud controller manager and the CSI plugin's controller hold the token, on control-plane nodes only.
  • Add-ons. Headlamp, per cluster or shared in the appliance behind its sign-in; Rancher, installed into a Kubehenge cluster, with the other clusters registered in it automatically.
  • Proxmox VE integration. An optional entry in the Proxmox VE interface leading to the appliance (appliance/pve-entry/), added if the administrator says yes when installing.

Apache-2.0. Not affiliated with or endorsed by Proxmox Server Solutions GmbH; Proxmox is its registered trademark.