This is a really odd issue I've started to experience. Everything was working with out issue, however, now when I startup a cluster (kubeadm), setup flannel, kube-dns never starts up. Eventually, it errors out with the following output from kubectl describe
Error: failed to start container "sidecar": Error response from daemon: {"message":"invalid header field value \"oci runtime error: container_linux.go:240: creating new parent process caused \\\"container_linux.go:1245: running lstat on namespace path \\\\\\\"/proc/7420/ns/ipc\\\\\\\" caused \\\\\\\"lstat /proc/7420/ns/ipc: no such file or directory\\\\\\\"\\\"\\n\""}
Any ideas what this error really means? I get the same looking error for dnsmasq and kubedns as well.
I am using the switch "--pod-network-cidr 10.244.0.0/16" as always. As I said ,this was working, and then a few days later, it's not....
Here's the get pods output:
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-machiato-0 1/1 Running 0 3m
kube-system kube-apiserver-machiato-0 1/1 Running 0 3m
kube-system kube-controller-manager-machiato-0 1/1 Running 0 2m
kube-system kube-dns-2258483030-pd8qj 0/3 ContainerCreating 0 3m
kube-system kube-flannel-ds-0z0dd 2/2 Running 0 1m
kube-system kube-flannel-ds-3dccg 2/2 Running 0 1m
kube-system kube-proxy-gc8ft 1/1 Running 0 3m
kube-system kube-proxy-tjgzn 1/1 Running 0 1m
kube-system kube-scheduler-machiato-0 1/1 Running 0 3m
Eventually, "ContainerCreating" switches to "CrashLoopBackOff" then I see the lstat error above.
most likely its overlay network issue. can you check the dns pods log message and see any error message?
kubectl logs -n kube-system kube-dns-2258483030-pd8qj -c kubedns
kubectl logs -n kube-system kube-dns-2258483030-pd8qj -c dnsmasq
kubectl logs -n kube-system kube-dns-2258483030-pd8qj -c sidecar