I have three node cluster and somehow it is ending up in a situation where pod is always in ContainerCreating state.

NAME                          READY     STATUS              RESTARTS   AGE
i-have-no-sa                  0/1       ContainerCreating   0          7h
prometheus-5464597859-5kfkd   0/1       ContainerCreating   0          7m
test-site-5768cf4dd5-zdjms    0/1       ContainerCreating   0          8m

If i look at logs of any of these pod i can see following

   Normal   SandboxChanged          4m (x7 over 4m)  kubelet, host2 Pod sandbox changed, it will be killed and re-created.
  Warning  FailedCreatePodSandBox  4m (x8 over 4m)  kubelet, host2 Failed create pod sandbox.

I am using flannel network and all the pods are running

kube-dns-6f4fd4bdf-5d29c                   3/3       Running             0          8h
kube-flannel-ds-74krh                      1/1       Running             0          8h
kube-flannel-ds-f4x2r                      1/1       Running             0          8h
kube-flannel-ds-w8dvf                      1/1       Running             0          8h
kube-proxy-2qz57                           1/1       Running             0          8h
kube-proxy-nnk76                           1/1       Running             0          8h
kube-proxy-wzph7                           1/1       Running             0          8h

in my previous attempts kube-dns pod was also in pending state always but it is working fine now.

kubernetes v1.9.0 on master Kubernetes v1.9.2 on slave1 Kubernetes v1.9.2 on slave2