coredns in crashloop because of failed connection to kubernetes api

2/28/2020

I've got a fresh install of Kubernetes cluster on centos 7 and Docker version 19.03.6

but coredns pods fail to start, this is the full log:

2020-02-28T12:55:56.541Z [INFO] plugin/reload: Running configuration MD5 = 7755bc9b35751c5434bff19f80f6324e
E0228 12:56:21.542876       1 reflector.go:134] github.com/coredns/coredns/plugin/kubernetes/controller.go:322: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E0228 12:56:21.542876       1 reflector.go:134] github.com/coredns/coredns/plugin/kubernetes/controller.go:322: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
log: exiting because of error: log: cannot create log: open /tmp/coredns.coredns-584795fc57-cgh67.unknownuser.log.ERROR.20200228-125621.1: no such file or directory

these are my pods

enter image description here

and these are the services

enter image description here

I tried with most of the common solutions like:

  • disable firewall

  • disable SELinux

  • allowPrivilegeEscalation to true

  • resetting cluster and trying again

found the solution at the end

  • one solution was changing pod_network_cidr: "192.168.0.0/16" to pod_network_cidr: "192.168.0.200/24"

  • second solution (and preferred one, I think) was setting up calico right after initializing cluster and before binding the nodes and installing docker on them etc

-- Nemanja Žunić
centos
coredns
docker
kubernetes
networking

1 Answer

2/28/2020

found the solution after all:

  • one solution was changing pod_network_cidr: "192.168.0.0/16" to pod_network_cidr: "192.168.0.200/24"

  • second solution (and preferred one, I think) was setting up calico right after initializing cluster (kubeadm init ...) and before binding the nodes to cluster (kubeadm join ...) and installing docker on the nodes etc

-- Nemanja Žunić
Source: StackOverflow