We have a deployment with replicas: 1 We deploy it in a 3 agent node k8s cluster (k8s 1.8.13) and it gets deployed to a node (say agent node-0). When I shutdown node-0, the rs does not get rescheduled (its been more than an hour now). I have checked that the selector labels are correct and we have plenty of capacity in the cluster (and also we don’t specify resource requests). Also I checked that our node selectors are just checking for agent nodes and there are 2 other agent nodes available. Is there any special treatments around this shutdown scenario that k8s does ?
That's the pod that get's re-scheduled, not the replicas set. If you would be doing rolling updates, based on the image version, for example, every time a new image would be available, controller manager would take the number of desired and available pods in the replica set to 0 and would create a new one.
But when you shut down a node, and the pod get's re-scheduled, keeping the same replica set, then your cluster is working fine.