kubeadm install flannel get error, what's wrong?

2/11/2019

I install kubeadm (version : v1.13.2 ), after init, I install flannel, it fails, install command:

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml

error is like below.

Error from server (Forbidden): error when retrieving current configuration of: Resource: "rbac.authorization.k8s.io/v1beta1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1beta1, Kind=ClusterRole" Name: "flannel", Namespace: "" Object: &{map["apiVersion":"rbac.authorization.k8s.io/v1beta1" "kind":"ClusterRole" "metadata":map["name":"flannel" "annotations":map["kubectl.kubernetes.io/last-applied-configuration":""]] "rules":[map["apiGroups":[""] "resources":["pods"] "verbs":["get"]] map["apiGroups":[""] "resources":["nodes"] "verbs":["list" "watch"]] map["apiGroups":[""] "resources":["nodes/status"] "verbs":["patch"]]]]} from server for: "kube-flannel.yml": clusterroles.rbac.authorization.k8s.io "flannel" is forbidden: User "system:node:node1" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope Error from server (Forbidden): error when retrieving current configuration of: Resource: "rbac.authorization.k8s.io/v1beta1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1beta1, Kind=ClusterRoleBinding" Name: "flannel", Namespace: "" Object: &{map["subjects":[map["kind":"ServiceAccount" "name":"flannel" "namespace":"kube-system"]] "apiVersion":"rbac.authorization.k8s.io/v1beta1" "kind":"ClusterRoleBinding" "metadata":map["name":"flannel" "annotations":map["kubectl.kubernetes.io/last-applied-configuration":""]] "roleRef":map["apiGroup":"rbac.authorization.k8s.io" "kind":"ClusterRole" "name":"flannel"]]} from server for: "kube-flannel.yml": clusterrolebindings.rbac.authorization.k8s.io "flannel" is forbidden: User "system:node:node1" cannot get resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope Error from server (Forbidden): error when retrieving current configuration of: Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount" Name: "flannel", Namespace: "kube-system" Object: &{map["kind":"ServiceAccount" "metadata":map["name":"flannel" "namespace":"kube-system" "annotations":map["kubectl.kubernetes.io/last-applied-configuration":""]] "apiVersion":"v1"]} from server for: "kube-flannel.yml": serviceaccounts "flannel" is forbidden: User "system:node:node1" cannot get resource "serviceaccounts" in API group "" in the namespace "kube-system": can only create tokens for individual service accounts Error from server (Forbidden): error when retrieving current configuration of: Resource: "/v1, Resource=configmaps", GroupVersionKind: "/v1, Kind=ConfigMap" Name: "kube-flannel-cfg", Namespace: "kube-system" Object: &{map["kind":"ConfigMap" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["app":"flannel" "tier":"node"] "name":"kube-flannel-cfg" "namespace":"kube-system"] "apiVersion":"v1" "data":map["cni-conf.json":"{\n \"name\": \"cbr0\",\n \"plugins\": [\n {\n \"type\": \"flannel\",\n \"delegate\": {\n \"hairpinMode\": true,\n \"isDefaultGateway\": true\n }\n },\n {\n \"type\": \"portmap\",\n \"capabilities\": {\n \"portMappings\": true\n }\n }\n ]\n}\n" "net-conf.json":"{\n \"Network\": \"10.244.0.0/16\",\n \"Backend\": {\n \"Type\": \"vxlan\"\n }\n}\n"]]} from server for: "kube-flannel.yml": configmaps "kube-flannel-cfg" is forbidden: User "system:node:node1" cannot get resource "configmaps" in API group "" in the namespace "kube-system": no path found to object Error from server (Forbidden): error when retrieving current configuration of: Resource: "extensions/v1beta1, Resource=daemonsets", GroupVersionKind: "extensions/v1beta1, Kind=DaemonSet" Name: "kube-flannel-ds-amd64", Namespace: "kube-system" Object: &{map["apiVersion":"extensions/v1beta1" "kind":"DaemonSet" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["app":"flannel" "tier":"node"] "name":"kube-flannel-ds-amd64" "namespace":"kube-system"] "spec":map["template":map["metadata":map["labels":map["app":"flannel" "tier":"node"]] "spec":map["tolerations":[map["effect":"NoSchedule" "operator":"Exists"]] "volumes":[map["hostPath":map["path":"/run"] "name":"run"] map["hostPath":map["path":"/etc/cni/net.d"] "name":"cni"] map["configMap":map["name":"kube-flannel-cfg"] "name":"flannel-cfg"]] "containers":[map["name":"kube-flannel" "resources":map["limits":map["cpu":"100m" "memory":"50Mi"] "requests":map["cpu":"100m" "memory":"50Mi"]] "securityContext":map["privileged":%!q(bool=true)] "volumeMounts":[map["mountPath":"/run" "name":"run"] map["mountPath":"/etc/kube-flannel/" "name":"flannel-cfg"]] "args":["--ip-masq" "--kube-subnet-mgr"] "command":["/opt/bin/flanneld"] "env":[map["name":"POD_NAME" "valueFrom":map["fieldRef":map["fieldPath":"metadata.name"]]] map["name":"POD_NAMESPACE" "valueFrom":map["fieldRef":map["fieldPath":"metadata.namespace"]]]] "image":"quay.io/coreos/flannel:v0.10.0-amd64"]] "hostNetwork":%!q(bool=true) "initContainers":[map["args":["-f" "/etc/kube-flannel/cni-conf.json" "/etc/cni/net.d/10-flannel.conflist"] "command":["cp"] "image":"quay.io/coreos/flannel:v0.10.0-amd64" "name":"install-cni" "volumeMounts":[map["mountPath":"/etc/cni/net.d" "name":"cni"] map["mountPath":"/etc/kube-flannel/" "name":"flannel-cfg"]]]] "nodeSelector":map["beta.kubernetes.io/arch":"amd64"] "serviceAccountName":"flannel"]]]]} from server for: "kube-flannel.yml": daemonsets.extensions "kube-flannel-ds-amd64" is forbidden: User "system:node:node1" cannot get resource "daemonsets" in API group "extensions" in the namespace "kube-system" Error from server (Forbidden): error when retrieving current configuration of: Resource: "extensions/v1beta1, Resource=daemonsets", GroupVersionKind: "extensions/v1beta1, Kind=DaemonSet" Name: "kube-flannel-ds-arm64", Namespace: "kube-system" Object: &{map["apiVersion":"extensions/v1beta1" "kind":"DaemonSet" "metadata":map["name":"kube-flannel-ds-arm64" "namespace":"kube-system" "annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "labels":map["app":"flannel" "tier":"node"]] "spec":map["template":map["spec":map["containers":[map["resources":map["limits":map["cpu":"100m" "memory":"50Mi"] "requests":map["cpu":"100m" "memory":"50Mi"]] "securityContext":map["privileged":%!q(bool=true)] "volumeMounts":[map["mountPath":"/run" "name":"run"] map["mountPath":"/etc/kube-flannel/" "name":"flannel-cfg"]] "args":["--ip-masq" "--kube-subnet-mgr"] "command":["/opt/bin/flanneld"] "env":[map["valueFrom":map["fieldRef":map["fieldPath":"metadata.name"]] "name":"POD_NAME"] map["name":"POD_NAMESPACE" "valueFrom":map["fieldRef":map["fieldPath":"metadata.namespace"]]]] "image":"quay.io/coreos/flannel:v0.10.0-arm64" "name":"kube-flannel"]] "hostNetwork":%!q(bool=true) "initContainers":[map["command":["cp"] "image":"quay.io/coreos/flannel:v0.10.0-arm64" "name":"install-cni" "volumeMounts":[map["mountPath":"/etc/cni/net.d" "name":"cni"] map["mountPath":"/etc/kube-flannel/" "name":"flannel-cfg"]] "args":["-f" "/etc/kube-flannel/cni-conf.json" "/etc/cni/net.d/10-flannel.conflist"]]] "nodeSelector":map["beta.kubernetes.io/arch":"arm64"] "serviceAccountName":"flannel" "tolerations":[map["effect":"NoSchedule" "operator":"Exists"]] "volumes":[map["hostPath":map["path":"/run"] "name":"run"] map["hostPath":map["path":"/etc/cni/net.d"] "name":"cni"] map["configMap":map["name":"kube-flannel-cfg"] "name":"flannel-cfg"]]] "metadata":map["labels":map["app":"flannel" "tier":"node"]]]]]} from server for: "kube-flannel.yml": daemonsets.extensions "kube-flannel-ds-arm64" is forbidden: User "system:node:node1" cannot get resource "daemonsets" in API group "extensions" in the namespace "kube-system" Error from server (Forbidden): error when retrieving current configuration of: Resource: "extensions/v1beta1, Resource=daemonsets", GroupVersionKind: "extensions/v1beta1, Kind=DaemonSet" Name: "kube-flannel-ds-arm", Namespace: "kube-system" Object: &{map["apiVersion":"extensions/v1beta1" "kind":"DaemonSet" "metadata":map["labels":map["app":"flannel" "tier":"node"] "name":"kube-flannel-ds-arm" "namespace":"kube-system" "annotations":map["kubectl.kubernetes.io/last-applied-configuration":""]] "spec":map["template":map["metadata":map["labels":map["app":"flannel" "tier":"node"]] "spec":map["hostNetwork":%!q(bool=true) "initContainers":[map["args":["-f" "/etc/kube-flannel/cni-conf.json" "/etc/cni/net.d/10-flannel.conflist"] "command":["cp"] "image":"quay.io/coreos/flannel:v0.10.0-arm" "name":"install-cni" "volumeMounts":[map["mountPath":"/etc/cni/net.d" "name":"cni"] map["mountPath":"/etc/kube-flannel/" "name":"flannel-cfg"]]]] "nodeSelector":map["beta.kubernetes.io/arch":"arm"] "serviceAccountName":"flannel" "tolerations":[map["effect":"NoSchedule" "operator":"Exists"]] "volumes":[map["hostPath":map["path":"/run"] "name":"run"] map["name":"cni" "hostPath":map["path":"/etc/cni/net.d"]] map["configMap":map["name":"kube-flannel-cfg"] "name":"flannel-cfg"]] "containers":[map["name":"kube-flannel" "resources":map["limits":map["cpu":"100m" "memory":"50Mi"] "requests":map["cpu":"100m" "memory":"50Mi"]] "securityContext":map["privileged":%!q(bool=true)] "volumeMounts":[map["mountPath":"/run" "name":"run"] map["mountPath":"/etc/kube-flannel/" "name":"flannel-cfg"]] "args":["--ip-masq" "--kube-subnet-mgr"] "command":["/opt/bin/flanneld"] "env":[map["name":"POD_NAME" "valueFrom":map["fieldRef":map["fieldPath":"metadata.name"]]] map["name":"POD_NAMESPACE" "valueFrom":map["fieldRef":map["fieldPath":"metadata.namespace"]]]] "image":"quay.io/coreos/flannel:v0.10.0-arm"]]]]]]} from server for: "kube-flannel.yml": daemonsets.extensions "kube-flannel-ds-arm" is forbidden: User "system:node:node1" cannot get resource "daemonsets" in API group "extensions" in the namespace "kube-system" Error from server (Forbidden): error when retrieving current configuration of: Resource: "extensions/v1beta1, Resource=daemonsets", GroupVersionKind: "extensions/v1beta1, Kind=DaemonSet" Name: "kube-flannel-ds-ppc64le", Namespace: "kube-system" Object: &{map["spec":map["template":map["metadata":map["labels":map["tier":"node" "app":"flannel"]] "spec":map["containers":[map["command":["/opt/bin/flanneld"] "env":[map["name":"POD_NAME" "valueFrom":map["fieldRef":map["fieldPath":"metadata.name"]]] map["name":"POD_NAMESPACE" "valueFrom":map["fieldRef":map["fieldPath":"metadata.namespace"]]]] "image":"quay.io/coreos/flannel:v0.10.0-ppc64le" "name":"kube-flannel" "resources":map["requests":map["cpu":"100m" "memory":"50Mi"] "limits":map["cpu":"100m" "memory":"50Mi"]] "securityContext":map["privileged":%!q(bool=true)] "volumeMounts":[map["mountPath":"/run" "name":"run"] map["name":"flannel-cfg" "mountPath":"/etc/kube-flannel/"]] "args":["--ip-masq" "--kube-subnet-mgr"]]] "hostNetwork":%!q(bool=true) "initContainers":[map["args":["-f" "/etc/kube-flannel/cni-conf.json" "/etc/cni/net.d/10-flannel.conflist"] "command":["cp"] "image":"quay.io/coreos/flannel:v0.10.0-ppc64le" "name":"install-cni" "volumeMounts":[map["mountPath":"/etc/cni/net.d" "name":"cni"] map["mountPath":"/etc/kube-flannel/" "name":"flannel-cfg"]]]] "nodeSelector":map["beta.kubernetes.io/arch":"ppc64le"] "serviceAccountName":"flannel" "tolerations":[map["effect":"NoSchedule" "operator":"Exists"]] "volumes":[map["hostPath":map["path":"/run"] "name":"run"] map["hostPath":map["path":"/etc/cni/net.d"] "name":"cni"] map["configMap":map["name":"kube-flannel-cfg"] "name":"flannel-cfg"]]]]] "apiVersion":"extensions/v1beta1" "kind":"DaemonSet" "metadata":map["labels":map["app":"flannel" "tier":"node"] "name":"kube-flannel-ds-ppc64le" "namespace":"kube-system" "annotations":map["kubectl.kubernetes.io/last-applied-configuration":""]]]} from server for: "kube-flannel.yml": daemonsets.extensions "kube-flannel-ds-ppc64le" is forbidden: User "system:node:node1" cannot get resource "daemonsets" in API group "extensions" in the namespace "kube-system" Error from server (Forbidden): error when retrieving current configuration of: Resource: "extensions/v1beta1, Resource=daemonsets", GroupVersionKind: "extensions/v1beta1, Kind=DaemonSet" Name: "kube-flannel-ds-s390x", Namespace: "kube-system" Object: &{map["apiVersion":"extensions/v1beta1" "kind":"DaemonSet" "metadata":map["labels":map["app":"flannel" "tier":"node"] "name":"kube-flannel-ds-s390x" "namespace":"kube-system" "annotations":map["kubectl.kubernetes.io/last-applied-configuration":""]] "spec":map["template":map["metadata":map["labels":map["app":"flannel" "tier":"node"]] "spec":map["nodeSelector":map["beta.kubernetes.io/arch":"s390x"] "serviceAccountName":"flannel" "tolerations":[map["effect":"NoSchedule" "operator":"Exists"]] "volumes":[map["hostPath":map["path":"/run"] "name":"run"] map["hostPath":map["path":"/etc/cni/net.d"] "name":"cni"] map["configMap":map["name":"kube-flannel-cfg"] "name":"flannel-cfg"]] "containers":[map["env":[map["name":"POD_NAME" "valueFrom":map["fieldRef":map["fieldPath":"metadata.name"]]] map["name":"POD_NAMESPACE" "valueFrom":map["fieldRef":map["fieldPath":"metadata.namespace"]]]] "image":"quay.io/coreos/flannel:v0.10.0-s390x" "name":"kube-flannel" "resources":map["limits":map["cpu":"100m" "memory":"50Mi"] "requests":map["cpu":"100m" "memory":"50Mi"]] "securityContext":map["privileged":%!q(bool=true)] "volumeMounts":[map["mountPath":"/run" "name":"run"] map["name":"flannel-cfg" "mountPath":"/etc/kube-flannel/"]] "args":["--ip-masq" "--kube-subnet-mgr"] "command":["/opt/bin/flanneld"]]] "hostNetwork":%!q(bool=true) "initContainers":[map["args":["-f" "/etc/kube-flannel/cni-conf.json" "/etc/cni/net.d/10-flannel.conflist"] "command":["cp"] "image":"quay.io/coreos/flannel:v0.10.0-s390x" "name":"install-cni" "volumeMounts":[map["mountPath":"/etc/cni/net.d" "name":"cni"] map["mountPath":"/etc/kube-flannel/" "name":"flannel-cfg"]]]]]]]]} from server for: "kube-flannel.yml": daemonsets.extensions "kube-flannel-ds-s390x" is forbidden: User "system:node:node1" cannot get resource "daemonsets" in API group "extensions" in the namespace "kube-system"

-- D.TS
kubernetes

1 Answer

2/11/2019

D.TS the problem seems to be in the fact that the previous kube.config is still present on the cluster. Try to run:

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config #confirm the overwrite with yes + enter.
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    kubectl get nodes

and then try to apply Flannel again. You can find more steps required for Kubeadm in this answer in case you did not prepare the environment properly.

-- aurelius
Source: StackOverflow