Is it ever safe to force-delete a kubernetes pod?

1/15/2019

https://kubernetes.io/docs/concepts/workloads/pods/pod/#force-deletion-of-pods

This section of the kubernetes documentation points out that "Force deletions can be potentially dangerous for some pods", but doesn't really go into detail on the dangers.

I understand that force deleting a pod will immediately "deregister" the pod from the API before the kubelet container confirms the underlying container is actually deleted, which could lead to a bunch of orphaned containers running if the kubelet fails to delete them. However, I don't know how to tell if a pod is "dangerous" to force-delete before I do so, or if there is even a way to predict this.

Are there any guidelines on safely force-deleting a pod? Or is this just an inherently unsafe operation?

-- Garrett Bates
kubelet
kubernetes
pod

1 Answer

1/15/2019

It really depends on what point of view.

From the K8s master and etcd which keeps the state in K8s it's safe as the entry is deleted in etcd.

However, the kube-scheduler tells the kubelet on the node to kill the pod and sometimes the kubelet might not be able to kill it (Most of the times it is).

A reason why it might not be able to kill the pod is if something like docker or your runtime isn't responding or a Linux system resource is not being released which could be anything like a deadlock, hardware failure, etc.

So most of the times it's safe but there might be a few specific cases where it's not due to the nature of your application and the state of your system.

-- Rico
Source: StackOverflow