Testing

Four tiers. Only the last needs Proxmox VE.

Tier	What it checks	Needs Proxmox VE
Lint and static	formatting, lint and types of Go, the frontend, shell, Terraform/OpenTofu, YAML, Markdown, the workflows and the e2e image's Dockerfile, each at its strictest settings; the charts' values against their schemas	no
Unit	the interface's backend and frontend, and the template rendering	no
Integration	the Helm charts and Kubehenge's controllers against a real Kubernetes API server carrying the CRDs of the Cluster API releases the management cluster installs, and of the Cluster API operator: every object against its schema, server-side apply, finalizers, status and scale	no
End to end	real clusters of VMs on Proxmox VE: create, scale workers up and down, upgrade Kubernetes by one minor version with nodes replaced one at a time, replace a failed node, delete; for each distribution and add-on	yes

Running them

make test runs make lint and make unit, and make integration runs on its own; their steps run side by side, one per CPU; each step can also run on its own, such as make unit-go or make lint-web (the targets are in the Makefile). The frontend's packages are installed again only when web/package-lock.json changes, and the frontend is built again only when one of its sources does.

Command	Tier	Needs
make lint	lint and static: Go for Linux and Windows (golangci-lint with every linter but those that contradict Go's conventions, gofumpt), the frontend (oxfmt, type-aware oxlint, TypeScript) and the Proxmox VE entry's script (oxfmt, oxlint), shell (ShellCheck with every optional check, shfmt), Terraform/OpenTofu (fmt, validate, TFLint), the Helm charts with their values, the workflows (actionlint, zizmor), YAML (yamllint), Markdown (markdownlint) and the e2e image's Dockerfile (hadolint); make fmt rewrites what the format checks refuse	the tools from scripts/tools.sh, Node.js, uv
make unit	unit: the Go packages under the race detector (the interface's API against a fake Proxmox VE and management cluster), the frontend (Vitest), and each Terraform/OpenTofu configuration against mocked providers	Go, Node.js, OpenTofu
make integration	integration: install the providers' chart with the module's values (the module's tests hold the two to the same file) and Kubehenge's chart, then for each distribution start its providers for the first cluster, apply its templates once they serve, remove them and scale the providers to none after the last cluster; take a cluster through Kubehenge's controller (finalizer, credentials owned by the cluster, the add-ons' objects) and delete it, its Proxmox VE objects removed only once its last machine is gone (Proxmox VE itself is faked)	Linux or Windows on amd64; Go, Helm, curl; envtest's kube-apiserver and etcd and the providers' CRDs are downloaded on the first run and checked against their checksums (test/integration/assets.sh)
make e2e-templates	end to end: create each distribution's template, find it again instead of rebuilding it, clone and boot it as Cluster API does and check it from inside, delete it, check nothing is left	a Proxmox VE host; settings in KUBEHENGE_PROXMOX_* and KUBEHENGE_E2E_*
make e2e-rancher	end to end: a one-node RKE2 cluster installs Rancher; a one-node k3s cluster registers with it on its own (agent pinned to Rancher's CA), leaves it with Rancher cleaning up inside it, and registers again; Rancher's cluster is held at the Kubernetes versions Rancher runs on; deleting the k3s cluster removes it from Rancher first; nothing is left	a running management cluster (KUBECONFIG), the RKE2 and k3s templates, a hostname resolving to the Rancher cluster's virtual IP (default <ip>.sslip.io); settings in test/e2e/rancher.sh
make e2e-interface	end to end: make the web interface's kubeconfig from the management cluster as the appliance does after setting it up, and check its rights: it sees clusters, machines and Kubehenge's ClusterClasses everywhere, but creates, changes and deletes clusters and reads Secrets only in its own namespace, and nothing else	a running management cluster with an admin kubeconfig in KUBECONFIG, its API in KUBEHENGE_E2E_API_ENDPOINT
make e2e-appliance RELEASE=<archive> NEWER=<archive> SETTINGS=<file>	end to end: install the appliance from a release archive with appliance/install.sh, on a certificate the Proxmox VE cluster's CA signed for its address; sign in with Proxmox VE users, a wrong password and a user without privileges refused; set the management cluster up from the interface, a second setup and an upgrade from the shell refused while it runs; reach the shared Headlamp behind the interface's sign-in; upgrade the appliance to the newer release under the running management cluster, and refuse the older one; remove the management cluster from the interface, refused unconfirmed or with a wrong passphrase, then the appliance; check that Proxmox VE is left exactly as it was	root on a Proxmox VE node, jq; two archives from make dist with different VERSIONs; the management cluster's settings as a JSON file, as the interface's setup sends them; settings in KUBEHENGE_E2E_* (test/e2e/appliance.sh)
make e2e-volumes	end to end: as Kubehenge's controller, create a cluster's pool, user, token and volume owner VM; make volumes as the CSI plugin does on every node's local storage; delete one through the owner VM with the cluster's token, the owner moving to the volume's node; remove the cluster and check every volume, the owner, the pool and the user are gone	a Proxmox VE host with the management cluster's Proxmox VE objects (the controller's token); settings in KUBEHENGE_PROXMOX_* and KUBEHENGE_E2E_* (pkg/provisioner/e2e_test.go); two or more nodes to check the move
make e2e-management-cluster DIR=<dir>	end to end: apply the management cluster, check every provider is ready, destroy, check nothing is left	a Proxmox VE host, kubectl, jq, curl; <dir> is a root configuration applying management-cluster/, and TF_VAR_proxmox_api_token is set
make e2e-management-cluster-upgrade DIR=<dir>	end to end: create the management cluster one step behind on Talos, Kubernetes, Cluster API and capmox, upgrade it in place to the tested set, check the same VMs run the new versions, destroy	as above, and <dir> passes version_overrides through
make e2e-workload-cluster CLASS=<class> VERSION=<v> UPGRADE=<v>	end to end: create a cluster from a ClusterClass trusting an OpenID issuer the test serves, check that the issuer's ID tokens sign in with prefixed user and groups while tokens for another audience or from another key do not, and ServiceAccount tokens still work (again after the upgrade); check that the kubernetes Service and every pod take their addresses from the cluster's ranges; check that the CNI chosen for it (KUBEHENGE_E2E_CNI, else the distribution's own) is the one running, with flannel off beside Cilium or Calico, and that it enforces a NetworkPolicy; check its own Proxmox VE pool, user and token scope, its nodes initialised by the cloud controller manager and its control-plane nodes tainted once its worker has joined, a persistent volume created and deleted; Headlamp switched on (reachable through the API server, without rights of its own, at Pod Security's restricted level) and off; scale workers to two and back, upgrade by one minor version (on RKE2 and k3s with the first new server refused by the old ones until its cloud-init has ended, so that its first start fails, which the roll survives), replace a stopped worker, delete, check nothing is left	a running management cluster (KUBECONFIG), the class's VM templates, kubectl, jq, curl, openssl, python3; an address of the test machine the cluster's nodes reach (KUBEHENGE_E2E_OIDC_ADDRESS); settings in KUBEHENGE_PROXMOX_* and KUBEHENGE_E2E_* (test/e2e/workload-cluster.sh)

End to end without a Proxmox VE host

CI runs the end-to-end tier with Proxmox VE in a container (.github/workflows/e2e.yml), one test per machine. A pull request runs the tests its changes need (test/e2e/ci/scope.sh: none for documentation alone, all of them for a change it cannot place); the merge queue runs all of them on the tree that would land on main, so nothing reaches main untested; and the nightly run adds the management cluster's upgrade and Rancher, builds Proxmox VE's image afresh and keeps it, with Go's build cache, for the day's runs. Any Linux machine with Docker, /dev/kvm and about 14 GB of free memory can do the same:

scripts/tools.sh ~/.local/kubehenge-tools && export PATH=~/.local/kubehenge-tools:$PATH
test/e2e/ci/up.sh /tmp/e2e          # Proxmox VE in a container, a registry beside it
test/e2e/ci/run.sh /tmp/e2e rke2    # or management-cluster, management-cluster-upgrade, appliance, kubeadm, k3s, rancher

up.sh builds test/e2e/pve, a one-node Proxmox VE on Debian 13 that runs on the machine's own kernel and starts VMs on its KVM, so the clusters' VMs are nested one level deeper than the machine itself. It gives the VMs the bridge 172.20.99.0/24 and the containers 172.21.99.0/24, routes this machine to the bridge (with sudo), and writes the settings run.sh reads. run.sh builds the controller from the working tree, pushes it to the registry, applies the management cluster (test/e2e/ci/main.tf) with the container's CA trusted by its own processes only (SSL_CERT_FILE; the machine's trust store is left as it is), creates the templates the test needs meanwhile and runs it; management-cluster runs the templates and volumes tests while the management cluster is up, before it destroys it; for appliance it builds two releases instead and runs the test in the container, whose appliance sets the management cluster up itself. Remove everything with docker rm -f kubehenge-e2e-pve kubehenge-e2e-registry && docker network rm kubehenge-e2e and sudo ip route del 172.20.99.0/24.

Supported means the end-to-end suite passed. A ClusterClass or add-on is listed as supported only once the end-to-end tier has run it in CI against a Proxmox VE host of the current release.