In GCP We are setting up kubernetes 1.14. HA as Stacked etcd topology.

We have created a image where kubernetes binaries are installed.

We have terrafrom script where an instance group is created with 3 master and 5 worker nodes instances using the above image.

Also, in the terrafrom script, we have created a TCP Load Balancing with 6443 port enabled.

I am able to bootstrap one master by running kubeadm init --config=. However, joining the 2nd master fails with below error.

kubeadm join XX.XX.XX.XX:6443 --token 9a08jv.c0izixklcxtmnze7 --discovery-token-ca-cert-hash sha256:73390a94962247546282a0954cb46f2a282b00534c06aff93773f3fc50aee562 --experimental-control-plane -v 8

Logs

I0423 09:50:33.623004 21078 checks.go:382] validating the presence of executable touch
I0423 09:50:33.623063 21078 checks.go:524] running all checks
I0423 09:50:33.656532 21078 checks.go:412] checking whether the given node name is reachable using net.LookupHost
I0423 09:50:33.656705 21078 checks.go:622] validating kubelet version
I0423 09:50:33.716178 21078 checks.go:131] validating if the service is enabled and active
I0423 09:50:33.723119 21078 checks.go:209] validating availability of port 10250
I0423 09:50:33.723377 21078 checks.go:439] validating if the connectivity type is via proxy or direct
I0423 09:50:33.723445 21078 join.go:441] [preflight] Fetching init configuration
I0423 09:50:33.723486 21078 join.go:474] [preflight] Retrieving KubeConfig objects
[preflight] Reading configuration from the cluster…
[preflight] FYI: You can look at this config file with ‘kubectl -n kube-system get cm kubeadm-config -oyaml’
I0423 09:50:33.725538 21078 round_trippers.go:416] GET https://XX.XX.XX.XX:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config
I0423 09:50:33.725564 21078 round_trippers.go:423] Request Headers:
I0423 09:50:33.725570 21078 round_trippers.go:426] Accept: application/json, /
I0423 09:50:33.725594 21078 round_trippers.go:426] User-Agent: kubeadm/v1.14.0 (linux/amd64) kubernetes/641856d
I0423 09:50:33.725886 21078 round_trippers.go:441] Response Status: in 0 milliseconds
I0423 09:50:33.725903 21078 round_trippers.go:444] Response Headers:

error execution phase preflight: unable to fetch the kubeadm-config ConfigMap: failed to get config map: Get https://XX.XX.XX.XX:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config: dial tcp XX.XX.XX.XX:6443: connect: connection refused

Note, we had faced the same issue in AWS wih NLB Loadbalacer, we were able to overcome the issue by using Classic Loadbalacer

Thanks in advance for your help.