kubernetes dashboard CrashLoopBackOff

8/5/2020

Brand new to kubernetes, but managed to install kubernetes, ubuntu 20.04 LTS, but having issues with the dashboard. followed the procedure, using flannel as CNF.

The log states issues with connection to 10.96.0.1:443, but telnet seems to work? Any suggestion how to getting further ?

bwa@prod3:~$ kubectl get pods -o wide --all-namespaces
NAMESPACE              NAME                                         READY   STATUS             RESTARTS   AGE   IP             NODE    NOMINATED NODE   READINESS GATES
kube-system            coredns-66bff467f8-jgmpl                     0/1     Running            1          27h   10.244.0.6     prod3   <none>           <none>
kube-system            coredns-66bff467f8-ldr9d                     0/1     Running            1          27h   10.244.0.9     prod3   <none>           <none>
kube-system            etcd-prod3                                   1/1     Running            1          27h   192.168.0.93   prod3   <none>           <none>
kube-system            kube-apiserver-prod3                         1/1     Running            1          27h   192.168.0.93   prod3   <none>           <none>
kube-system            kube-controller-manager-prod3                1/1     Running            1          27h   192.168.0.93   prod3   <none>           <none>
kube-system            kube-flannel-ds-amd64-xm26h                  1/1     Running            2          27h   192.168.0.93   prod3   <none>           <none>
kube-system            kube-proxy-7lk5d                             1/1     Running            1          27h   192.168.0.93   prod3   <none>           <none>
kube-system            kube-scheduler-prod3                         1/1     Running            1          27h   192.168.0.93   prod3   <none>           <none>
kubernetes-dashboard   dashboard-metrics-scraper-6b4884c9d5-xrdbh   1/1     Running            1          27h   10.244.0.7     prod3   <none>           <none>
kubernetes-dashboard   kubernetes-dashboard-7f99b75bf4-lfqtf        0/1     CrashLoopBackOff   310        27h   10.244.0.8     prod3   <none>           <none>


bwa@prod3:~$ kubectl logs kubernetes-dashboard-7f99b75bf4-lfqtf --namespace=kubernetes-dashboard --tail=100
    2020/08/05 12:02:31 Starting overwatch
    2020/08/05 12:02:31 Using namespace: kubernetes-dashboard
    2020/08/05 12:02:31 Using in-cluster config to connect to apiserver
    2020/08/05 12:02:31 Using secret token for csrf signing
    2020/08/05 12:02:31 Initializing csrf token from kubernetes-dashboard-csrf secret
    panic: Get "https://10.96.0.1:443/api/v1/namespaces/kubernetes-dashboard/secrets/kubernetes-dashboard-csrf": dial tcp 10.96.0.1:443: i/o timeout
    
    goroutine 1 [running]:
    github.com/kubernetes/dashboard/src/app/backend/client/csrf.(*csrfTokenManager).init(0xc00000c640)
            /home/runner/work/dashboard/dashboard/src/app/backend/client/csrf/manager.go:41 +0x446
    github.com/kubernetes/dashboard/src/app/backend/client/csrf.NewCsrfTokenManager(...)
            /home/runner/work/dashboard/dashboard/src/app/backend/client/csrf/manager.go:66
    github.com/kubernetes/dashboard/src/app/backend/client.(*clientManager).initCSRFKey(0xc00044f800)
            /home/runner/work/dashboard/dashboard/src/app/backend/client/manager.go:501 +0xc6
    github.com/kubernetes/dashboard/src/app/backend/client.(*clientManager).init(0xc00044f800)
            /home/runner/work/dashboard/dashboard/src/app/backend/client/manager.go:469 +0x47
    github.com/kubernetes/dashboard/src/app/backend/client.NewClientManager(...)
            /home/runner/work/dashboard/dashboard/src/app/backend/client/manager.go:550
    main.main()
            /home/runner/work/dashboard/dashboard/src/app/backend/dashboard.go:105 +0x20d
    bwa@prod3:~$ telnet 10.96.0.1 443
    Trying 10.96.0.1...
    Connected to 10.96.0.1.
    Escape character is '^]'.
    ^CConnection closed by foreign host.
    bwa@prod3:~$
-- bwa
kubernetes
kubernetes-dashboard

1 Answer

5/11/2021

By the looks of that cluster, you do not have a networking plugin (CNI) installed. I do not see any flannel pods in the kube-system namespace, and the coredns pods are not starting.

This would also explain why the dashboard panics, as it is unable to reach the K8s API server via the 10.96.0.1 service.

Can you check the flannel installation (or just reinstall flannel on the cluster)?

-- Bogd
Source: StackOverflow