accidentally Install multiple CNI plugin in kubernets cluster, need help to clean up

12/3/2019

I am new in Kubernetes, to exploring and learning the Kubernetes I am trying to deploy the cluster, following the https://github.com/kelseyhightower/kubernetes-the-hard-way/ documentation.

in network configuration, I accidentally install kube-flannel and weave cni.

i deleted the flannel cni using kubectl delete -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

it's successfully deleted, but Not able to deploy the pod. I search in logs and find still Kubernetes trying to use the kube-flannel cni network.

Dec  3 08:25:29 worker-3 kubelet[1153]: E1203 08:25:29.019612    1153 remote_runtime.go:105] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to setup network for sandbox "1cf12a571f811a39ca725443a2a2f482d564a9f8113b6e8e7b69a9f05ab49d49": open /run/flannel/subnet.env: no such file or directory
Dec  3 08:25:29 worker-3 kubelet[1153]: E1203 08:25:29.019612    1153 kuberuntime_sandbox.go:68] CreatePodSandbox for pod "nginx-5b8c4fbff7-lhln9_default(2ec5091d-dced-42a2-b3ed-e4392cc03b3c)" failed: rpc error: code = Unknown desc = failed to setup network for sandbox "1cf12a571f811a39ca725443a2a2f482d564a9f8113b6e8e7b69a9f05ab49d49": open /run/flannel/subnet.env: no such file or directory
Dec  3 08:25:29 worker-3 kubelet[1153]: E1203 08:25:29.019612    1153 kuberuntime_manager.go:692] createPodSandbox for pod "nginx-5b8c4fbff7-lhln9_default(2ec5091d-dced-42a2-b3ed-e4392cc03b3c)" failed: rpc error: code = Unknown desc = failed to setup network for sandbox "1cf12a571f811a39ca725443a2a2f482d564a9f8113b6e8e7b69a9f05ab49d49": open /run/flannel/subnet.env: no such file or directory
Dec  3 08:25:29 worker-3 kubelet[1153]: E1203 08:25:29.019613    1153 pod_workers.go:190] Error syncing pod 2ec5091d-dced-42a2-b3ed-e4392cc03b3c ("nginx-5b8c4fbff7-lhln9_default(2ec5091d-dced-42a2-b3ed-e4392cc03b3c)"), skipping: failed to "CreatePodSandbox" for "nginx-5b8c4fbff7-lhln9_default(2ec5091d-dced-42a2-b3ed-e4392cc03b3c)" with CreatePodSandboxError: "CreatePodSandbox for pod \"nginx-5b8c4fbff7-lhln9_default(2ec5091d-dced-42a2-b3ed-e4392cc03b3c)\" failed: rpc error: code = Unknown desc = failed to setup network for sandbox \"1cf12a571f811a39ca725443a2a2f482d564a9f8113b6e8e7b69a9f05ab49d49\": open /run/flannel/subnet.env: no such file or directory"
Dec  3 08:25:41 worker-3 kubelet[1153]: I1203 08:25:41.006764    1153 kuberuntime_manager.go:404] No sandbox for pod "nginx-5b8c4fbff7-lhln9_default(2ec5091d-dced-42a2-b3ed-e4392cc03b3c)" can be found. Need to start a new one
Dec  3 08:25:41 worker-3 containerd[1138]: time="2019-12-03T08:25:41.008529346+05:30" level=info msg="RunPodsandbox for &PodSandboxMetadata{Name:nginx-5b8c4fbff7-lhln9,Uid:2ec5091d-dced-42a2-b3ed-e4392cc03b3c,Namespace:default,Attempt:0,}"
Dec  3 08:25:41 worker-3 kernel: [ 2267.354749] IPVS: Creating netns size=2192 id=175
Dec  3 08:25:41 worker-3 containerd[1138]: time="2019-12-03T08:25:41.017324873+05:30" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:nginx-5b8c4fbff7-lhln9,Uid:2ec5091d-dced-42a2-b3ed-e4392cc03b3c,Namespace:default,Attempt:0,} failed, error" error="failed to setup network for sandbox "49cc0e01b8d00d53c98937ae0408a4eaf2f72c7c3dc3dd887e9ed125af459fcd": open /run/flannel/subnet.env: no such file or directory"
Dec  3 08:25:41 worker-3 kubelet[1153]: E1203 08:25:41.018658    1153 remote_runtime.go:105] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to setup network for sandbox "49cc0e01b8d00d53c98937ae0408a4eaf2f72c7c3dc3dd887e9ed125af459fcd": open /run/flannel/subnet.env: no such file or directory

Need your help to fix this problem?

here is my cluster status

NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-2               Healthy   {"health":"true"}   
etcd-1               Healthy   {"health":"true"}   
etcd-0               Healthy   {"health":"true"}   
☁  kthw  kubectl get pods -n kube-system -o wide
NAME              READY   STATUS    RESTARTS   AGE   IP              NODE       NOMINATED NODE   READINESS GATES
weave-net-dfzxr   2/2     Running   2          53m   192.168.56.14   worker-2   <none>           <none>
weave-net-knqk4   2/2     Running   2          53m   192.168.56.13   worker-1   <none>           <none>
weave-net-nnmp8   2/2     Running   2          53m   192.168.56.15   worker-3   <none>           <none>
☁  kthw  kubectl get nodes
NAME       STATUS   ROLES    AGE   VERSION
worker-1   Ready    <none>   84m   v1.15.3
worker-2   Ready    <none>   84m   v1.15.3
worker-3   Ready    <none>   84m   v1.15.3
-- Ankur Gupta
docker
google-kubernetes-engine
kubectl
kubernetes
kubernetes-pod

0 Answers