kube-dns kubedns/dnsmasq/sidecar fails to start

8/3/2017

This is a really odd issue I've started to experience. Everything was working with out issue, however, now when I startup a cluster (kubeadm), setup flannel, kube-dns never starts up. Eventually, it errors out with the following output from kubectl describe

Error: failed to start container "sidecar": Error response from daemon: {"message":"invalid header field value \"oci runtime error: container_linux.go:240: creating new parent process caused \\\"container_linux.go:1245: running lstat on namespace path \\\\\\\"/proc/7420/ns/ipc\\\\\\\" caused \\\\\\\"lstat /proc/7420/ns/ipc: no such file or directory\\\\\\\"\\\"\\n\""}

Any ideas what this error really means? I get the same looking error for dnsmasq and kubedns as well.

I am using the switch "--pod-network-cidr 10.244.0.0/16" as always. As I said ,this was working, and then a few days later, it's not....

Here's the get pods output:

NAMESPACE     NAME                                 READY     STATUS              RESTARTS   AGE
kube-system   etcd-machiato-0                      1/1       Running             0          3m
kube-system   kube-apiserver-machiato-0            1/1       Running             0          3m
kube-system   kube-controller-manager-machiato-0   1/1       Running             0          2m
kube-system   kube-dns-2258483030-pd8qj            0/3       ContainerCreating   0          3m
kube-system   kube-flannel-ds-0z0dd                2/2       Running             0          1m
kube-system   kube-flannel-ds-3dccg                2/2       Running             0          1m
kube-system   kube-proxy-gc8ft                     1/1       Running             0          3m
kube-system   kube-proxy-tjgzn                     1/1       Running             0          1m
kube-system   kube-scheduler-machiato-0            1/1       Running             0          3m

Eventually, "ContainerCreating" switches to "CrashLoopBackOff" then I see the lstat error above.

-- Andi Jay
kubernetes

1 Answer

8/4/2017

most likely its overlay network issue. can you check the dns pods log message and see any error message?

 kubectl logs -n kube-system  kube-dns-2258483030-pd8qj -c kubedns
 kubectl logs -n kube-system  kube-dns-2258483030-pd8qj -c dnsmasq 
 kubectl logs -n kube-system  kube-dns-2258483030-pd8qj -c sidecar
-- sfgroups
Source: StackOverflow