K
Q

Question

kubectl get pods returns inconsistent results

6/15/2019

When I execute kubectl get pods, I get different output for the same pod.

For example:

$ kubectl get pods -n ha-rabbitmq
NAME            READY   STATUS    RESTARTS   AGE
rabbitmq-ha-0   1/1     Running   0          85m
rabbitmq-ha-1   1/1     Running   9          84m
rabbitmq-ha-2   1/1     Running   0          50m

After that I execute the same command and here is the different result:

$ kubectl get pods -n ha-rabbitmq
NAME            READY   STATUS             RESTARTS   AGE
rabbitmq-ha-0   0/1     CrashLoopBackOff   19         85m
rabbitmq-ha-1   1/1     Running            9          85m
rabbitmq-ha-2   1/1     Running            0          51m

I have 2 master nodes and 5 worker nodes initialized with kubeadm. Each master node has one instance of built-in etcd pod running on them.

Result of kubectl get nodes:

$ kubectl get nodes -owide
NAME              STATUS   ROLES    AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
k8s-meb-master1   Ready    master   14d     v1.14.3   10.30.29.11    <none>        Ubuntu 18.04.2 LTS   4.15.0-51-generic   docker://18.9.5
k8s-meb-master2   Ready    master   14d     v1.14.3   10.30.29.12    <none>        Ubuntu 18.04.2 LTS   4.15.0-51-generic   docker://18.9.6
k8s-meb-worker1   Ready    <none>   14d     v1.14.3   10.30.29.13    <none>        Ubuntu 18.04.2 LTS   4.15.0-51-generic   docker://18.9.5
k8s-meb-worker2   Ready    <none>   14d     v1.14.3   10.30.29.14    <none>        Ubuntu 18.04.2 LTS   4.15.0-51-generic   docker://18.9.5
k8s-meb-worker3   Ready    <none>   14d     v1.14.3   10.30.29.15    <none>        Ubuntu 18.04.2 LTS   4.15.0-51-generic   docker://18.9.5
k8s-meb-worker4   Ready    <none>   14d     v1.14.2   10.30.29.16    <none>        Ubuntu 18.04.2 LTS   4.15.0-51-generic   docker://18.9.5
k8s-meb-worker5   Ready    <none>   5d19h   v1.14.2   10.30.29.151   <none>        Ubuntu 18.04 LTS     4.15.0-20-generic   docker://18.9.5

Can this issue be related to unsynchronized contents for the /var/lib/etcd/ in the master nodes ?

-- Cemal Unal

kubectl

kubernetes

1 Answer

6/15/2019

Your pods are in CrashLoopBackoff state. That means that some containers inside the pod are exiting (the main process exits) and the pod gets restarted over and over again.

Depending when you run the get po command, you might see your pod as Running (the process didn't exit yet) or CrashLoopBackoff (kubernetes is waiting before restarting your pod.

You can confirm this is the case by looking at the Restarts counter in the output.

I suggest you have a look at the restarting pods logs to get an idea why they're failing.

-- whites11

Source: StackOverflow

KQ

kubectl get pods returns inconsistent results

Similar Questions

1 Answer

K
Q