Whenever I shutdown a worker node, the pods that were running on the node are stuck in a "terminating 1/1" state. After the default 5 minutes for the probe check, the pods are redeployed onto healthy worker nodes, but the pods from the previous shut down node are still showing as 1/1 and terminating. It stays in this state indefinitely. Is there any way to make this process cleaner, so that whenever the pods are redeployed to new worker nodes, the old pods are removed and not in the terminating state?
this is expected behavior: the pods need to stay in terminating state until the node comes back, so that the master remembers to tell the node to stop these pods and to gather all remaining logs, events etc. This is because a node can go to unready state not only because of shutdown, but for example also because of temporary network fragmentation in which case after the link is back these pods would still be there also.