I'm trying to get k3s set up with High Availability using two raspberry pi 4s, but can't get the second server to link to the first.
Grateful for any help with getting this to work.
I've tried both the Embedded DB and with an External DB (mariadb)
The first server node sets up fine using the following command:
#embedded db - server 1
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--log k3s.log --bind-address 192.168.X.XXX --write-kubeconfig-mode 644 --docker --cluster-init" sh -
#external db - server 1
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--log k3s.log --bind-address 192.168.X.XXX --write-kubeconfig-mode 644 --docker --cluster-init --datastore-endpoint mysql://username:password@tcp(192.168.X.Y:3307)/k3s" sh -Getting token from server 1 and adding on server 2
#get token from server 1
pi@rpi4:~ $ sudo cat /var/lib/rancher/k3s/server/node-token
K10b598b7a839cb4d2351f77b7d7c18f12345678bd7f68603434248b4cdf1b333fd::server:4b7b5a1b8b05271298150f008e1b804e
#on server 2...
export K3S_TOKEN="K10b598b7a839cb4d2351f77b7d7c18f12345678bd7f68603434248b4cdf1b333fd::server:4b7b5a1b8b05271298150f008e1b804e"I then try to get a second server to join the cluster using:
#embedded db - server 2
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--log k3s.log --server https://192.168.X.X:6443 --write-kubeconfig-mode 644 --docker" sh -
#external db - server 2
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--log k3s.log --server https://192.168.X.X:6443 --write-kubeconfig-mode 644 --datastore-endpoint mysql://user:password@tcp(192.168.X.X:3307)/k3s --docker" sh -The services run without error on both rpis, but second server doesn't show up as connected to the first
Edit
Had to use a bit of a variation to get it to work with the embedded db, this worked in terms of getting the 2nd/3rd server to join the initial cluster
(with K3S_TOKEN and K3S_URL set)
curl -fL https://get.k3s.io | INSTALL_K3S_EXEC="--log k3s.log --flannel-iface=eth0 --write-kubeconfig-mode 644 --no-deploy servicelb --docker" sh -s - serverhowever, pods on Server 2 and Server 3 are unable to communicate with pods connected to Server 1.
I've already raised a separate issue for this, as I initially thought it was a different problem
It seems to me you're correctly extracting the token and putting it into the 2nd server, but you're not calling it in the installation script: this way the second server doesn't even try to sync with the first, rather it just spins a new cluster and creates a new token.
Also the --server option is not needed. I also fell into this trap: it may be some legacy of previous versions, but it doesn't work. Currently, as I understand it, the server when given a token and a database, goes into the db and looks for the cluster defined by the token, if it finds one, it automatically starts as HA.
What I do is the following: I first create a token
A) master1$ NODE_TOKEN=$(echo $(hostname) $(date +%s) | shasum | base64)
master1$ echo $NODE_TOKEN > node_token
and use it to spin up the cluster on the first master
B) master1$ curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="-v 2 -l master1.log -t ${NODE_TOKEN} --flannel-iface=enp0s8 --write-kubeconfig-mode 644 --tls-san k3s-cluster-01.lan --node-taint k3s-controlplane=true:NoExecute --datastore-endpoint mysql://k3s:${MYSQL_PASSWORD}@tcp(${IPADDR}:3306)/k3s" sh -I then put it on my second master and install there too
C) master2$ curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="-v 2 -l master2.log -t ${NODE_TOKEN} --flannel-iface=enp0s8 --write-kubeconfig-mode 644 --tls-san k3s-cluster-01.lan --node-taint k3s-controlplane=true:NoExecute --datastore-endpoint mysql://k3s:${MYSQL_PASSWORD}@tcp(${IPADDR}:3306)/k3s" sh -Regarding the HA without external storage I haven't been able to make it work and eventually I gave up. So far I'm using a single instance of mysql, but that has to go eventually. Whether replaced by K3S own dqlite or etcd or whatnot I do not know yet.