We currently having 2 Master 2 Worker node cluster on Kubernetes v1.13.4.The cluster is down as the kubelet certificate located in /var/lib/kubelet/pki/kubelet.crt has expired and the kubelet service is not running. On checking the kubelet logs I get the following error
E0808 09:49:35.126533 55154 bootstrap.go:209] Part of the existing bootstrap client certificate is expired: 2019-08-06 22:39:23 +0000 UTC
The following certificates ca.crt, apiserver-kubelet-client.crt are valid. We are unable to renew the kubelet.crt certificate manually by using the kubeadm-config.yaml. Can someone please provide the steps to renew the certificate.
We have tried setting --rotate-certificates property and also using kubeadm-config.yaml but since we are using v1.13.4 kubeadm --config flag is not present.
On checking the kubelet logs I get the following error
E0808 09:49:35.126533 55154 bootstrap.go:209] Part of the existing bootstrap client certificate is expired: 2019-08-06 22:39:23 +0000 UTC
As you mentioned that only kubelet.crt has expired and apiserver-kubelet-client.crt is valid, you can try to renew it by command kubeadm alpha certs renew based on documentation.
Second way to renew kubeadm certificates is upgrade version like in this article.
You can also try by using kubeadm init phase certs all. It was explained in this Stackoverflow case.
Let me know if that helped. If not provide more information with more logs.