Kubernetes coredns pods not running in every pods

11/5/2020

With new installation of Kubernetes on Ubuntu with one master and two nodes,

root@master1# kubectl get nodes -o wide
NAME      STATUS   ROLES    AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION       CONTAINER-RUNTIME
master1   Ready    master   10h   v1.19.3   10.10.10.216   <none>        Ubuntu 18.04.5 LTS   4.15.0-122-generic   docker://19.3.13
worker1   Ready    <none>   10h   v1.19.3   10.10.10.211   <none>        Ubuntu 18.04.5 LTS   4.15.0-122-generic   docker://19.3.13
worker2   Ready    <none>   10h   v1.19.3   10.10.10.212   <none>        Ubuntu 18.04.5 LTS   4.15.0-122-generic   docker://19.3.13

i check if all pods in namespace kube-system works.

root@master1:# kubectl get pods -A -o wide
NAMESPACE     NAME                              READY   STATUS    RESTARTS   AGE   IP             NODE      NOMINATED NODE   READINESS GATES
kube-system   coredns-f9fd979d6-cggnh           1/1     Running   0          10h   10.244.0.2     master1   <none>           <none>
kube-system   coredns-f9fd979d6-tnm7c           1/1     Running   0          10h   10.244.0.3     master1   <none>           <none>
kube-system   etcd-master1                      1/1     Running   0          10h   10.10.10.216   master1   <none>           <none>
kube-system   kube-apiserver-master1            1/1     Running   0          10h   10.10.10.216   master1   <none>           <none>
kube-system   kube-controller-manager-master1   1/1     Running   0          10h   10.10.10.216   master1   <none>           <none>
kube-system   kube-flannel-ds-9ph5c             1/1     Running   0          10h   10.10.10.216   master1   <none>           <none>
kube-system   kube-flannel-ds-fjkng             1/1     Running   0          10h   10.10.10.212   worker2   <none>           <none>
kube-system   kube-flannel-ds-rfkqd             1/1     Running   0          9h    10.10.10.211   worker1   <none>           <none>
kube-system   kube-proxy-j7s2m                  1/1     Running   0          10h   10.10.10.216   master1   <none>           <none>
kube-system   kube-proxy-n7279                  1/1     Running   0          10h   10.10.10.212   worker2   <none>           <none>
kube-system   kube-proxy-vkb66                  1/1     Running   0          9h    10.10.10.211   worker1   <none>           <none>
kube-system   kube-scheduler-master1            1/1     Running   0          10h   10.10.10.216   master1   <none>           <none>

I see that coredns is work only in master with two pods

How i should do to replicate coredns in all my 3 vm (master + 2 nodes)

this is the description of coredns deployment

root@master1:# kubectl describe deployment coredns -n kube-system
Name:                   coredns
Namespace:              kube-system
CreationTimestamp:      Wed, 04 Nov 2020 20:32:10 +0000
Labels:                 k8s-app=kube-dns
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               k8s-app=kube-dns
Replicas:               2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  1 max unavailable, 25% max surge
Pod Template:
  Labels:           k8s-app=kube-dns
  Service Account:  coredns
  Containers:
   coredns:
    Image:       k8s.gcr.io/coredns:1.7.0
    Ports:       53/UDP, 53/TCP, 9153/TCP
    Host Ports:  0/UDP, 0/TCP, 0/TCP
    Args:
      -conf
      /etc/coredns/Corefile
    Limits:
      memory:  170Mi
    Requests:
      cpu:        100m
      memory:     70Mi
    Liveness:     http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
    Readiness:    http-get http://:8181/ready delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /etc/coredns from config-volume (ro)
  Volumes:
   config-volume:
    Type:               ConfigMap (a volume populated by a ConfigMap)
    Name:               coredns
    Optional:           false
  Priority Class Name:  system-cluster-critical
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   coredns-f9fd979d6 (2/2 replicas created)
Events:          <none>

also , the logs and the stats of deployment

root@master1# kubectl logs deployment/coredns -n kube-system
Found 2 pods, using pod/coredns-f9fd979d6-cggnh
.:53
[INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7
CoreDNS-1.7.0
linux/amd64, go1.14.4, f59c03d


root@master1:# kubectl get deployment -o wide -n kube-system
NAME      READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES                     SELECTOR
coredns   2/2     2            2           10h   coredns      k8s.gcr.io/coredns:1.7.0   k8s-app=kube-dns
-- user14581886
coredns
kubernetes

1 Answer

11/5/2020

Andre, you can add podAntiAffinity to your coredns definition:

        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                k8s-app: kube-dns
            topologyKey: kubernetes.io/hostname

This will let your coredns replicas scheduling to different nodes.

-- Kun Li
Source: StackOverflow