K
Q

Question

Can't join cluster: connection refused during kubeadm join

7/10/2018

I am getting the following error when I try to join the cluster

sudo kubeadm join 10.1.1.150:6443 --token ypcdg7.w6pun0nd31c4q5c2 --discovery-token-ca-cert-hash sha256:1ac79447f8dee9d90d592a3ead3d6c54ce7046dcd0b3854917c93cf6bbff7894
[preflight] running pre-flight checks
    [WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh] or no builtin kernel ipvs support: map[ip_vs:{} ip_vs_rr:{} ip_vs_wrr:{} ip_vs_sh:{} nf_conntrack_ipv4:{}]
you can solve this problem with following methods:
 1. Run 'modprobe -- ' to load missing kernel modules;
2. Provide the missing builtin kernel ipvs support

I0710 13:39:56.080260    2415 kernel_validator.go:81] Validating kernel version
I0710 13:39:56.080423    2415 kernel_validator.go:96] Validating kernel config
[discovery] Trying to connect to API Server "10.1.1.150:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://10.1.1.150:6443"
[discovery] Failed to request cluster info, will try again: [Get https://10.1.1.150:6443/api/v1/namespaces/kube-public/configmaps/cluster-info: dial tcp 10.1.1.150:6443: connect: connection refused]

I get the same error with curl on the machine that I want to join the cluster. But if I run this

curl -k https://10.1.1.150:6443/api/v1/namespaces/kube-public/configmaps/cluster-info

in any machine in the local net (10.1.1.0/24) I get a good JSON response.

Some usefull info:

I can ping my master node (10.1.1.150) 6443 is open on my master node, this is netstat on my master

jp@tensor3:~$ sudo netstat -tulpn
Connessioni Internet attive (solo server)
Proto CodaRic CodaInv Indirizzo locale        Indirizzo remoto       Stato       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1110/sshd       
tcp        0      0 127.0.0.1:10248         0.0.0.0:*               LISTEN      6783/kubelet    
tcp        0      0 127.0.0.1:10249         0.0.0.0:*               LISTEN      7375/kube-proxy 
tcp        0      0 127.0.0.1:10251         0.0.0.0:*               LISTEN      7120/kube-scheduler
tcp        0      0 127.0.0.1:2379          0.0.0.0:*               LISTEN      7065/etcd       
tcp        0      0 127.0.0.1:10252         0.0.0.0:*               LISTEN      7058/kube-controlle
tcp        0      0 127.0.0.1:2380          0.0.0.0:*               LISTEN      7065/etcd       
tcp        0      0 127.0.0.1:37491         0.0.0.0:*               LISTEN      6783/kubelet    
tcp6       0      0 :::22                   :::*                    LISTEN      1110/sshd       
tcp6       0      0 :::10250                :::*                    LISTEN      6783/kubelet    
tcp6       0      0 :::6443                 :::*                    LISTEN      7130/kube-apiserver
tcp6       0      0 :::10256                :::*                    LISTEN      7375/kube-proxy 
udp        0      0 0.0.0.0:68              0.0.0.0:*                           980/dhclient    
udp        0      0 0.0.0.0:8472            0.0.0.0:*                           -

I'm using flannel for network communication

ifconfig and route on my master

jp@tensor3:~$ ifconfig -a
cni0      Link encap:Ethernet  IndirizzoHW 0a:58:0a:f4:00:01  
          indirizzo inet:10.244.0.1  Bcast:0.0.0.0  Maschera:255.255.255.0
          indirizzo inet6: fe80::ec67:aeff:fe90:4fbc/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:31641 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31925 errors:0 dropped:0 overruns:0 carrier:0
          collisioni:0 txqueuelen:1000 
          Byte RX:2032440 (2.0 MB)  Byte TX:11726575 (11.7 MB)

docker0   Link encap:Ethernet  IndirizzoHW 02:42:44:f4:4f:cb  
          indirizzo inet:172.17.0.1  Bcast:0.0.0.0  Maschera:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisioni:0 txqueuelen:0 
          Byte RX:0 (0.0 B)  Byte TX:0 (0.0 B)

ens160    Link encap:Ethernet  IndirizzoHW 00:50:56:3d:73:84  
          indirizzo inet:10.1.1.150  Bcast:10.1.1.255  Maschera:255.255.255.0
          indirizzo inet6: fe80::250:56ff:fe3d:7384/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:294081 errors:0 dropped:0 overruns:0 frame:0
          TX packets:74724 errors:0 dropped:0 overruns:0 carrier:0
          collisioni:0 txqueuelen:1000 
          Byte RX:297445831 (297.4 MB)  Byte TX:5900453 (5.9 MB)

flannel.1 Link encap:Ethernet  IndirizzoHW 06:a7:4c:1d:6f:cd  
          indirizzo inet:10.244.0.0  Bcast:0.0.0.0  Maschera:255.255.255.255
          indirizzo inet6: fe80::4a7:4cff:fe1d:6fcd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:8 overruns:0 carrier:0
          collisioni:0 txqueuelen:0 
          Byte RX:0 (0.0 B)  Byte TX:0 (0.0 B)

lo        Link encap:Loopback locale  
          indirizzo inet:127.0.0.1  Maschera:255.0.0.0
          indirizzo inet6: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:1219503 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1219503 errors:0 dropped:0 overruns:0 carrier:0
          collisioni:0 txqueuelen:1 
          Byte RX:271937625 (271.9 MB)  Byte TX:271937625 (271.9 MB)

vethd451ffcc Link encap:Ethernet  IndirizzoHW 8a:f1:f0:05:80:f3  
          indirizzo inet6: fe80::88f1:f0ff:fe05:80f3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:8799 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8860 errors:0 dropped:0 overruns:0 carrier:0
          collisioni:0 txqueuelen:0 
          Byte RX:687962 (687.9 KB)  Byte TX:3256670 (3.2 MB)

vethed2073fb Link encap:Ethernet  IndirizzoHW ea:9e:43:5e:4e:30  
          indirizzo inet6: fe80::e89e:43ff:fe5e:4e30/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:8800 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8887 errors:0 dropped:0 overruns:0 carrier:0
          collisioni:0 txqueuelen:0 
          Byte RX:688228 (688.2 KB)  Byte TX:3258477 (3.2 MB)


jp@tensor3:~$ route -n
Tabella di routing IP del kernel
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.1.1.29       0.0.0.0         UG    0      0        0 ens160
10.1.1.0        0.0.0.0         255.255.255.0   U     0      0        0 ens160
10.244.0.0      0.0.0.0         255.255.255.0   U     0      0        0 cni0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0

ifconfig and route on my node

jp@tensor2:~$ ifconfig
docker0   Link encap:Ethernet  IndirizzoHW 02:42:4e:2f:0e:97  
          indirizzo inet:172.17.0.1  Bcast:0.0.0.0  Maschera:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisioni:0 txqueuelen:0 
          Byte RX:0 (0.0 B)  Byte TX:0 (0.0 B)

ens160    Link encap:Ethernet  IndirizzoHW 00:50:56:8c:0a:cd  
          indirizzo inet:10.1.1.151  Bcast:10.1.1.255  Maschera:255.255.255.0
          indirizzo inet6: fe80::250:56ff:fe8c:acd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:17470 errors:0 dropped:0 overruns:0 frame:0
          TX packets:913 errors:0 dropped:0 overruns:0 carrier:0
          collisioni:0 txqueuelen:1000 
          Byte RX:1511461 (1.5 MB)  Byte TX:101801 (101.8 KB)

lo        Link encap:Loopback locale  
          indirizzo inet:127.0.0.1  Maschera:255.0.0.0
          indirizzo inet6: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:160 errors:0 dropped:0 overruns:0 frame:0
          TX packets:160 errors:0 dropped:0 overruns:0 carrier:0
          collisioni:0 txqueuelen:1 
          Byte RX:11840 (11.8 KB)  Byte TX:11840 (11.8 KB)

jp@tensor2:~$ route -n
Tabella di routing IP del kernel
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.1.1.29       0.0.0.0         UG    0      0        0 ens160
10.1.1.0        0.0.0.0         255.255.255.0   U     0      0        0 ens160
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0

and this is the status of my cluster

jp@tensor3:~$ sudo kubectl get all --namespace=kube-system
[sudo] password di jp: 
NAME                                  READY     STATUS    RESTARTS   AGE
pod/coredns-78fcdf6894-bblcs          1/1       Running   0          1h
pod/coredns-78fcdf6894-wrmj4          1/1       Running   0          1h
pod/etcd-tensor3                      1/1       Running   0          1h
pod/kube-apiserver-tensor3            1/1       Running   0          1h
pod/kube-controller-manager-tensor3   1/1       Running   0          1h
pod/kube-flannel-ds-amd64-p7jmq       1/1       Running   0          1h
pod/kube-proxy-qg7jj                  1/1       Running   0          1h
pod/kube-scheduler-tensor3            1/1       Running   0          1h

NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
service/kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP   1h

NAME                                     DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR                     AGE
daemonset.apps/kube-flannel-ds-amd64     1         1         1         1            1           beta.kubernetes.io/arch=amd64     1h
daemonset.apps/kube-flannel-ds-arm       0         0         0         0            0           beta.kubernetes.io/arch=arm       1h
daemonset.apps/kube-flannel-ds-arm64     0         0         0         0            0           beta.kubernetes.io/arch=arm64     1h
daemonset.apps/kube-flannel-ds-ppc64le   0         0         0         0            0           beta.kubernetes.io/arch=ppc64le   1h
daemonset.apps/kube-flannel-ds-s390x     0         0         0         0            0           beta.kubernetes.io/arch=s390x     1h
daemonset.apps/kube-proxy                1         1         1         1            1           beta.kubernetes.io/arch=amd64     1h

NAME                      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/coredns   2         2         2            2           1h

NAME                                 DESIRED   CURRENT   READY     AGE
replicaset.apps/coredns-78fcdf6894   2         2         2         1h

On the same machine I'm using:

docker version 1.13.1

k8s version 1.11

-- Antonio Caristia

kubernetes

1 Answer

9/7/2018

I met the similar behaviour on Ubuntu 16.04 after reboot.

I did sudo kubeadm join ... It crashed like described. Then I did: sudo su kubeadm init exit

I tried again and that second time it was successful.

-- Stanislav Chlebec

Source: StackOverflow

KQ

Can't join cluster: connection refused during kubeadm join

Similar Questions

1 Answer

K
Q