x509: certificate signed by unknown authority in kubernetes

12/12/2021

Suddenly my kubernetes cluster shows this error when I deployment a new service into this cluster:

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "02ceaa702704151a69b1f9f5210d4a68956c2b41050b711319f07d38f57364af": error getting ClusterInformation: Get "https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default": x509: certificate signed by unknown authority

This cluster running fine for monthes, I have using this command to check my kubernetes cluster certificate like this:

[root@k8smasterone ~]# openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep ' Not '
            Not Before: Aug  3 13:31:21 2021 GMT
            Not After : Aug  4 04:40:53 2022 GMT

the certificate did not expire util 2022.08, why shows this error? what should I do to fix this problem? This is all the certificate expire status:

[root@k8smasterone ~]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Aug 03, 2022 13:31 UTC   234d                                    no      
apiserver                  Aug 04, 2022 04:40 UTC   234d            ca                      no      
apiserver-etcd-client      Aug 03, 2022 13:31 UTC   234d            etcd-ca                 no      
apiserver-kubelet-client   Aug 03, 2022 13:31 UTC   234d            ca                      no      
controller-manager.conf    Aug 03, 2022 13:31 UTC   234d                                    no      
etcd-healthcheck-client    Aug 03, 2022 13:31 UTC   234d            etcd-ca                 no      
etcd-peer                  Aug 03, 2022 13:31 UTC   234d            etcd-ca                 no      
etcd-server                Aug 03, 2022 13:31 UTC   234d            etcd-ca                 no      
front-proxy-client         Aug 03, 2022 13:31 UTC   234d            front-proxy-ca          no      
scheduler.conf             Aug 03, 2022 13:31 UTC   234d                                    no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Aug 01, 2031 13:31 UTC   9y              no      
etcd-ca                 Aug 01, 2031 13:31 UTC   9y              no      
front-proxy-ca          Aug 01, 2031 13:31 UTC   9y              no 

this is my kubernetes version information:

[root@k8smasterone ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.3", GitCommit:"ca643a4d1f7bfe34773c74f79527be4afd95bf39", GitTreeState:"clean", BuildDate:"2021-07-15T21:04:39Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:53:14Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}

I am sure the kube proxy running in the master node.

-- Dolphin
kubernetes

1 Answer

12/14/2021

Its not the issue with certificate expiry, it is the issue with the trust between the cluster components, seems like that trust is broken. This might be related. Please check the following, if that helps: https://discuss.kubernetes.io/t/solved-x509-certificate-error/14151 https://programmerah.com/failed-to-create-pod-sandbox-rpc-error-code-unknown-desc-failed-to-set-up-sandbox-container-28097/

-- redzack
Source: StackOverflow