When I use to build Hypervisor (i.e. VMWare, Hyper-V, etc.) clusters, ALL the hardware and software has to be exactly the same. Otherwise I ran the possibility that the workload could run into a 'conflict' (i.e. the VM won't run because the hardware or OS is different), if there is a failure on one of the nodes.
If you are building a kubernetes clusters from miscellaneous (i.e. legacy) hardware sitting around the server room (i.e. different vendors [Dell, HPE, etc.], different CPU types [i.e. AMD, Intel, etc.], different BIOS versions, memory size, etc.). Do kubernetes worker nodes have to exactly the same hardware for the workload to balance across correctly the cluster (i.e. distribute the workload across the nodes).
I would guess everything from the OS (i.e. distro/kernel, libraries, modules, and services) up would have to be same. I am just trying to find a general answer to a question that I have not seen a good answer to?
Generally, it is ok to run heterogenous hardware in Kubernetes. If all of you boxes are X86 machines there is nothing to worry as your docker images should run everywhere. For instance it is common to mix different types of spot instances in the cloud and this works fine.
However, if you are mixing architecture (i.e. arm and x86) or operating systems (i.e. windows and linux) it generally makes sense to add a label indicating that. Those are typical labels in Kubernetes 1.15+:
$ kubectl describe node xxxxx
Name: xxxxx
Roles: node
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/instance-type=dell_xyz
beta.kubernetes.io/os=linux
[...]
You can then use those labels in your node selector in a pod:
apiVersion: v1
kind: Pod
metadata:
name: x86-pod
spec:
containers:
- name: x86-test
image: "yourrepo.io/test_repo"
nodeSelector:
beta.kubernetes.io/arch: amd64Kubernetes worker nodes can be heterogeneous both from hardware and software point of view. Only kubernetes related services (kubelet, container runtime, CNI) have to be compatible with control plane version.
You can even run mixture of windows and linux nodes.