I'm trying to build a self managed kubernetes cluster on AWS/EC2 using Ubuntu VM's running 18.04 (so not EKS). I've managed to get the Master built which is integrated with ELB/Classic LB (i couldnt get this working with NLB) to allow me to expose services via type=LoadBalancer before moving over to an ingress controller such as nginx or istio to do more L7 stuff.

The Master is healthy and in Ready status running K8's version 1.20.5

I've managed to join a worker node to the cluster.

If I run kubectl get node on the master both the master and worker node are showing as Ready.

But as the worker node is joining the cluster i see an error relating to the below.

Seems to be an error relating to error uploading crisocket.

Anyone got any ideas why? I dont want to move on before clearning the error even though both my master and worker nodes are 'Ready'. Thanks!

error uploading crisocket: timed out waiting for the condition

This is the debug from the joining process

I0326 11:53:48.564188    4751 join.go:395] [preflight] found NodeName empty; using OS hostname as NodeName
I0326 11:53:48.564426    4751 initconfiguration.go:104] detected and using CRI socket: /var/run/dockershim.sock
[preflight] Running pre-flight checks
I0326 11:53:48.564662    4751 preflight.go:90] [preflight] Running general checks
I0326 11:53:48.564821    4751 checks.go:249] validating the existence and emptiness of directory /etc/kubernetes/manifests
I0326 11:53:48.564946    4751 checks.go:286] validating the existence of file /etc/kubernetes/kubelet.conf
I0326 11:53:48.565004    4751 checks.go:286] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I0326 11:53:48.565050    4751 checks.go:102] validating the container runtime
I0326 11:53:48.623727    4751 checks.go:128] validating if the "docker" service is enabled and active
I0326 11:53:48.694853    4751 checks.go:335] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I0326 11:53:48.695050    4751 checks.go:335] validating the contents of file /proc/sys/net/ipv4/ip_forward
I0326 11:53:48.695164    4751 checks.go:649] validating whether swap is enabled or not
I0326 11:53:48.695282    4751 checks.go:376] validating the presence of executable conntrack
I0326 11:53:48.695382    4751 checks.go:376] validating the presence of executable ip
I0326 11:53:48.695487    4751 checks.go:376] validating the presence of executable iptables
I0326 11:53:48.695608    4751 checks.go:376] validating the presence of executable mount
I0326 11:53:48.695691    4751 checks.go:376] validating the presence of executable nsenter
I0326 11:53:48.695805    4751 checks.go:376] validating the presence of executable ebtables
I0326 11:53:48.695874    4751 checks.go:376] validating the presence of executable ethtool
I0326 11:53:48.695961    4751 checks.go:376] validating the presence of executable socat
I0326 11:53:48.696007    4751 checks.go:376] validating the presence of executable tc
I0326 11:53:48.696101    4751 checks.go:376] validating the presence of executable touch
I0326 11:53:48.696213    4751 checks.go:520] running all checks
I0326 11:53:48.766440    4751 checks.go:406] checking whether the given node name is reachable using net.LookupHost
I0326 11:53:48.767324    4751 checks.go:618] validating kubelet version
I0326 11:53:48.858929    4751 checks.go:128] validating if the "kubelet" service is enabled and active
I0326 11:53:48.871674    4751 checks.go:201] validating availability of port 10250
I0326 11:53:48.871944    4751 checks.go:286] validating the existence of file /etc/kubernetes/pki/ca.crt
I0326 11:53:48.872045    4751 checks.go:432] validating if the connectivity type is via proxy or direct
I0326 11:53:48.872194    4751 join.go:465] [preflight] Discovering cluster-info
I0326 11:53:48.872309    4751 token.go:78] [discovery] Created cluster-info discovery client, requesting info from "internal-k8-lb-1843285331.eu-west-1.elb.amazonaws.com:6443"
I0326 11:53:48.901218    4751 token.go:116] [discovery] Requesting info from "internal-k8-lb-1843285331.eu-west-1.elb.amazonaws.com:6443" again to validate TLS against the pinned public key
I0326 11:53:48.913626    4751 token.go:133] [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "internal-k8-lb-1843285331.eu-west-1.elb.amazonaws.com:6443"
I0326 11:53:48.913749    4751 discovery.go:51] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
I0326 11:53:48.913840    4751 join.go:479] [preflight] Fetching init configuration
I0326 11:53:48.913948    4751 join.go:517] [preflight] Retrieving KubeConfig objects
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
I0326 11:53:48.929632    4751 interface.go:400] Looking for default routes with IPv4 addresses
I0326 11:53:48.929749    4751 interface.go:405] Default route transits interface "eth0"
I0326 11:53:48.930180    4751 interface.go:208] Interface eth0 is up
I0326 11:53:48.930365    4751 interface.go:256] Interface "eth0" has 2 addresses :[172.31.27.238/20 fe80::47a:b6ff:fe55:969d/64].
I0326 11:53:48.930482    4751 interface.go:223] Checking addr  172.31.27.238/20.
I0326 11:53:48.930569    4751 interface.go:230] IP found 172.31.27.238
I0326 11:53:48.930674    4751 interface.go:262] Found valid IPv4 address 172.31.27.238 for interface "eth0".
I0326 11:53:48.930758    4751 interface.go:411] Found active IP 172.31.27.238
I0326 11:53:48.940030    4751 preflight.go:101] [preflight] Running configuration dependant checks
I0326 11:53:48.940151    4751 controlplaneprepare.go:211] [download-certs] Skipping certs download
I0326 11:53:48.940238    4751 kubelet.go:110] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I0326 11:53:48.941312    4751 kubelet.go:118] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
I0326 11:53:48.942266    4751 kubelet.go:139] [kubelet-start] Checking for an existing Node in the cluster with name "ip-172-31-27-238" and status "Ready"
I0326 11:53:48.946297    4751 kubelet.go:153] [kubelet-start] Stopping the kubelet
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
I0326 11:53:54.169977    4751 kubelet.go:188] [kubelet-start] preserving the crisocket information for the node
I0326 11:53:54.170123    4751 patchnode.go:30] [patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "ip-172-31-27-238" as an annotation
I0326 11:53:54.170218    4751 cert_rotation.go:137] Starting client certificate rotation controller
[kubelet-check] Initial timeout of 40s passed.
timed out waiting for the condition
error uploading crisocket