Reasons of Pod Status Failed

4/8/2019

If Pod's status is Failed, Kubernetes will try to create new Pods until it reaches terminated-pod-gc-threshold in kube-controller-manager. This will leave many Failed Pods in a cluster and need to be cleaned up.

Are there other reasons except Evicted that will cause Pod Failed?

-- flyer
kube-controller-manager
kubelet
kubernetes

2 Answers

4/8/2019

There can be many causes for the POD status to be FAILED. You just need to check for problems(if there exists any) by running the command

kubectl -n <namespace> describe pod <pod-name>

Carefully check the EVENTS section where all the events those occurred during POD creation are listed. Hopefully you can pinpoint the cause of failure from there.

However there are several reasons for POD failure, some of them are the following:

  • Wrong image used for POD.
  • Wrong command/arguments are passed to the POD.
  • Kubelet failed to check POD liveliness(i.e., liveliness probe failed).
  • POD failed health check.
  • Problem in network CNI plugin (misconfiguration of CNI plugin used for networking).
-- Arbaaz
Source: StackOverflow

4/8/2019

PODs will not survive scheduling failures, node failures, or other evictions, such as lack of resources, or in the case of node maintenance. Pods should not be created manually but almost always via controllers like Deployments (self-healing, replication etc).

Reason why pod failed or was terminated can be obtain by

kubeclt describe pod <pod_name>

Others situation I have encountered when pod Failed:

  • Issues with image (not existing anymore)
  • When pod is attempting to access i.e ConfigMap or Secrets but it is not found in namespace.
  • Liveness Probe Failure
  • Persistent Volume fails to mount
  • Validation Error

In addition, eviction is based on resources - EvictionPolicy

It can be also caused by DRAINing the Node/Pod. You can read about DRAIN here.

-- PjoterS
Source: StackOverflow