When deploying pods on our k8s rancher cluster, if the chosen node has more than 59 pods running already, the new pod will come up but very slowly. Sometimes it takes 3-5 minutes to start a pod with the image already present on the machine.

To get around this we've tried cordoning nodes with pod numbers > 59 but this is not a long term solution.

Our nodes have plenty of resources.

We've considered putting the cap on --max-nodes at 59 but k8s is supposed to support 100 pods per node.

Is there something I'm missing in my cluster setup? We have 3 etc nodes that are also control planes.