Add new control plane node got failed k8s 1.21.0 [solved]

6/14/2021

I want to add a new control plane node into the cluster.

So, I run in an existing control plane server: kubeadm token create --print-join-command

I run this command in new control plane node:

kubeadm join 10.0.0.151:8443 --token m3g8pf.gdop9wz08yhd7a8a --discovery-token-ca-cert-hash sha256:634db22bc69b47b8f2b9f733d2f5e95cf8e56b349e68ac611a56d9da0cf481b8 --control-plane --apiserver-advertise-address 10.0.0.10 --apiserver-bind-port 6443 --certificate-key 33cf0a1d30da4c714755b4de4f659d6d5a02e7a0bd522af2ebc2741487e53166
  1. I got this message:
[download-certs] Downloading the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace 
error execution phase control-plane-prepare/download-certs: error downloading certs: the Secret does not include the required certificate or key - name: external-e
tcd.crt, path: /etc/kubernetes/pki/apiserver-etcd-client.crt
  1. I run in an existing production control plane node:
kubeadm init phase upload-certs --upload-certs
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
0a3f5486c3b9303a4ace70ad0a9870c2605d67eebcd500d68a5e776bbd628a3b
  1. Re-run this command in the new control plane node:
kubeadm join 10.0.0.151:8443 --token m3g8pf.gdop9wz08yhd7a8a --discovery-token-ca-cert-hash sha256:634db22bc69b47b8f2b9f733d2f5e95cf8e56b349e68ac611a56d9da0cf481b8 --control-plane --apiserver-advertise-address 10.0.0.10 --apiserver-bind-port 6443 --certificate-key 0a3f5486c3b9303a4ace70ad0a9870c2605d67eebcd500d68a5e776bbd628a3b

I got the same message:

[download-certs] Downloading the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
error execution phase control-plane-prepare/download-certs: error downloading certs: the Secret does not include the required certificate or key - name: external-etcd.crt, path: /etc/kubernetes/pki/apiserver-etcd-client.crt
To see the stack trace of this error execute with --v=5 or higher

What's I am wrong?

I have all certs in the new node installed before doing this op:

# ls /etc/kubernetes/pki/
apiserver.crt              apiserver.key                 ca.crt  front-proxy-ca.crt      front-proxy-client.key
apiserver-etcd-client.crt  apiserver-kubelet-client.crt  ca.key  front-proxy-ca.key      sa.key
apiserver-etcd-client.key  apiserver-kubelet-client.key  etcd    front-proxy-client.crt  sa.pub

I didn't see how to specify etcd certs files:

Usage:
  kubeadm init phase upload-certs [flags]

Flags:
      --certificate-key string       Key used to encrypt the control-plane certificates in the kubeadm-certs Secret.
      --config string                Path to a kubeadm configuration file.
  -h, --help                         help for upload-certs
      --kubeconfig string            The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file. (default "/etc/kubernetes/admin.conf")
      --skip-certificate-key-print   Don't print the key used to encrypt the control-plane certificates.
      --upload-certs                 Upload control-plane certificates to the kubeadm-certs Secret.

Global Flags:
      --add-dir-header           If true, adds the file directory to the header of the log messages
      --log-file string          If non-empty, use this log file
      --log-file-max-size uint   Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
      --one-output               If true, only write logs to their native severity level (vs also writing to each lower severity level)
      --rootfs string            [EXPERIMENTAL] The path to the 'real' host root filesystem.
      --skip-headers             If true, avoid header prefixes in the log messages
      --skip-log-headers         If true, avoid headers when opening log files
  -v, --v Level                  number for the log level verbosity
-- sincorchetes
etcd
kubeadm
kubernetes
ssl

1 Answer

6/22/2021

You also need to pass the --config flag to your kubeadm init phase command (use sudo if needed). So instead of:

kubeadm init phase upload-certs --upload-certs

you should for example run:

kubeadm init phase upload-certs --upload-certs --config kubeadm-config.yaml

This topic is also explained by Uploading control-plane certificates to the cluster docs.

-- WytrzymaƂy Wiktor
Source: StackOverflow