I am trying to setup a new worker-node on CentOS-7.9 with following commands.
# setenforce 0
# sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
# firewall disabled already.
# swapoff -a
# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
# yum install kubeadm -y
# systemctl enable kubelet
# systemctl start kubelet
But kubelet service
status shows below error.
# systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Wed 2020-12-02 16:49:22 IST; 3s ago
Docs: https://kubernetes.io/docs/
Process: 4442 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255)
Main PID: 4442 (code=exited, status=255)
Dec 02 16:49:22 k8s-node-01 systemd[1]: Unit kubelet.service entered failed state.
Dec 02 16:49:22 k8s-node-01 kubelet[4442]: /workspace/anago-v1.19.4-rc.0.51+5f1e5cafd33a88/src/k8s.io/kubernetes/_output/dockerized/go/src/...81 +0x4f
Dec 02 16:49:22 k8s-node-01 kubelet[4442]: created by k8s.io/kubernetes/vendor/k8s.io/component-base/logs.InitLogs
Dec 02 16:49:22 k8s-node-01 kubelet[4442]: /workspace/anago-v1.19.4-rc.0.51+5f1e5cafd33a88/src/k8s.io/kubernetes/_output/dockerized/go/src/...58 +0x8a
Dec 02 16:49:22 k8s-node-01 kubelet[4442]: goroutine 47 [select]:
Dec 02 16:49:22 k8s-node-01 kubelet[4442]: k8s.io/kubernetes/vendor/go.opencensus.io/stats/view.(*worker).start(0xc000050be0)
Dec 02 16:49:22 k8s-node-01 kubelet[4442]: /workspace/anago-v1.19.4-rc.0.51+5f1e5cafd33a88/src/k8s.io/kubernetes/_output/dockerized/go/src/...4 +0x105
Dec 02 16:49:22 k8s-node-01 kubelet[4442]: created by k8s.io/kubernetes/vendor/go.opencensus.io/stats/view.init.0
Dec 02 16:49:22 k8s-node-01 kubelet[4442]: /workspace/anago-v1.19.4-rc.0.51+5f1e5cafd33a88/src/k8s.io/kubernetes/_output/dockerized/go/src/...32 +0x57
Dec 02 16:49:22 k8s-node-01 systemd[1]: kubelet.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
I have below Kubernetes & docker version installed.
# kubelet --version
Kubernetes v1.19.4
# docker --version
Docker version 19.03.14, build 5eb3275d40
Also tried to join but even this fails.
# kubeadm join 65.66.67.68:6443 --token tu7qvt.1rfzhnxevg8m792h --discovery-token-ca-cert-hash sha256:48109668a4eadfs3c0c13a04d24a99bd82ff2eredefab6be6b78aadeead358074ee
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 65.66.67.55:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 65.66.67.55:10248: connect: connection refused.
error execution phase kubelet-start: error uploading crisocket: timed out waiting for the condition
To see the stack trace of this error execute with --v=5 or higher
With -v=9
option:-
# kubeadm join 65.66.67.68:6443 --token tu7qvt.1rfzhnxevg8m792h --discovery-token-ca-cert-hash sha256:48109668a4eadfs3c0c13a04d24a99bd82ff2eredefab6be6b78aadeead358074ee -v=9
I1203 10:25:29.374052 11716 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.19.4 (linux/amd64) kubernetes/d360454" 'https://65.66.67.68:6443/api/v1/nodes/k8s-node-01?timeout=10s'
I1203 10:25:29.376358 11716 round_trippers.go:443] GET https://65.66.67.68:6443/api/v1/nodes/k8s-node-01?timeout=10s 404 Not Found in 2 milliseconds
I1203 10:25:29.376406 11716 round_trippers.go:449] Response Headers:
I1203 10:25:29.376411 11716 round_trippers.go:452] Content-Type: application/json
I1203 10:25:29.376415 11716 round_trippers.go:452] Content-Length: 192
I1203 10:25:29.376419 11716 round_trippers.go:452] Date: Thu, 03 Dec 2020 04:55:29 GMT
I1203 10:25:29.376423 11716 round_trippers.go:452] Cache-Control: no-cache, private
I1203 10:25:29.376443 11716 request.go:1097] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodes \"k8s-node-01\" not found","reason":"NotFound","details":{"name":"k8s-node-01","kind":"nodes"},"code":404}
timed out waiting for the condition
error uploading crisocket
What could be the wrong in installation? Any direction would be helpful.
Node has joined the cluster after commenting the entries from /etc/resolv.conf
file then once node has joined to the cluster successfully again Un-commented
. Now on my master all the namespaces and nodes are running fine.