flannel pods in CrashLoopBackoff Error in kubernetes

3/27/2018

Using flannel as a CNI in kubernetes i am trying to implement a network for pod to pod communication spread on different vagrant vms. I am using this https://raw.githubusercontent.com/coreos/flannel/v0.9.0/Documentation/kube-flannel.yml to create flannel pods. But the kube-flannel pods go in CrashLoopBackOff error and do not start.

[root@flnode-04 ~]# kubectl get pods -o wide --all-namespaces
NAMESPACE         NAME                      READY     STATUS             RESTARTS   AGE       IP              NODE
diamanti-system   collectd-v0.5-flnode-04   1/1       Running            0          3h        192.168.30.14   flnode-04
diamanti-system   collectd-v0.5-flnode-05   1/1       Running            0          3h        192.168.30.15   flnode-05
diamanti-system   collectd-v0.5-flnode-06   1/1       Running            0          3h        192.168.30.16   flnode-06
diamanti-system   provisioner-d4kvf         1/1       Running            0          3h        192.168.30.16   flnode-06
kube-system       kube-flannel-ds-2kqpv     0/1       CrashLoopBackOff   1          18m       192.168.30.14   flnode-04
kube-system       kube-flannel-ds-xgqdm     0/1       CrashLoopBackOff   1          18m       192.168.30.16   flnode-06
kube-system       kube-flannel-ds-z59jz     0/1       CrashLoopBackOff   1          18m       192.168.30.15   flnode-05

here are the logs of one pod

[root@flnode-04 ~]# kubectl logs kube-flannel-ds-2kqpv --namespace=kube-system 
I0327 10:28:44.103425       1 main.go:483] Using interface with name mgmt0 and address 192.168.30.14
I0327 10:28:44.105609       1 main.go:500] Defaulting external address to interface address (192.168.30.14)
I0327 10:28:44.138132       1 kube.go:130] Waiting 10m0s for node controller to sync
I0327 10:28:44.138213       1 kube.go:283] Starting kube subnet manager
I0327 10:28:45.138509       1 kube.go:137] Node controller sync successful
I0327 10:28:45.138588       1 main.go:235] Created subnet manager: Kubernetes Subnet Manager - flnode-04
I0327 10:28:45.138596       1 main.go:238] Installing signal handlers
I0327 10:28:45.138690       1 main.go:348] Found network config - Backend type: vxlan
I0327 10:28:45.138767       1 vxlan.go:119] VXLAN config: VNI=1 Port=0 GBP=false DirectRouting=false
panic: assignment to entry in nil map
goroutine 1 [running]:
github.com/coreos/flannel/subnet/kube.(*kubeSubnetManager).AcquireLease(0xc420010cd0, 0x7f5314399bd0, 0xc420347880, 0xc4202213e0, 0x6, 0xf54, 0xc4202213e0)
    /go/src/github.com/coreos/flannel/subnet/kube/kube.go:239 +0x1f7
github.com/coreos/flannel/backend/vxlan.(*VXLANBackend).RegisterNetwork(0xc4200b3480, 0x7f5314399bd0, 0xc420347880, 0xc420010c30, 0xc4200b3480, 0x0, 0x0, 0x4d0181)
    /go/src/github.com/coreos/flannel/backend/vxlan/vxlan.go:141 +0x44e
main.main()
    /go/src/github.com/coreos/flannel/main.go:278 +0x8ae

What exactly is the reason for flannel pods going into CrashLoopBackoff and what is the solution ?

-- Yashgiri Goswami
flannel
kubernetes

2 Answers

2/23/2019

If you deploy cluster with kubeadm init --pod-network-cidr network/mask, this network/mask should match the ConfigMap in kube-flannel.yaml

My ConfigMap looks like:

kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  data:
  ...
    net-conf.json: |
      {
        "Network": "10.244.0.0/16",
        "Backend": {
          "Type": "vxlan"
        }
      }

So the network/mask should equal 10.244.0.0/16

-- menya
Source: StackOverflow

3/28/2018

I was able to solve the problem by running the command

kubectl annotate node appserv9 flannel.alpha.coreos.com/public-ip=10.10.10.10 --overwrite=true

Reason for bug : nil map in the code(no key available)

it doesn't matter what ip you give but this command has to be run individually on all the nodes so that the above error does not have to assign to a nil map.

-- Yashgiri Goswami
Source: StackOverflow