Trying to build 3-node Kubernetes cluster and running into and issue on CoreOS and I cannot join any nodes to cluster API Server keeps throwing an error regarding TLS certificate.
I1128 23:08:47.715663 1 logs.go:41] http: TLS handshake error from 10.123.146.55:48344: remote error: bad certificate
I1128 23:08:47.829082 1 logs.go:41] http: TLS handshake error from 10.123.146.55:48346: remote error: bad certificate
I1128 23:08:47.881655 1 logs.go:41] http: TLS handshake error from 10.123.146.54:40896: remote error: bad certificate
I1128 23:08:47.923955 1 logs.go:41] http: TLS handshake error from 10.123.146.54:40898: remote error: bad certificate
Certificate on the worker is valid - verified using
curl --key worker-key.pem -k https://10.123.146.53/api/v1/nodes --cert worker.pem --cacert ca.pem
Only relevant error I could see is Controller is failing to start certificate controller:
I1128 22:45:12.452293 1 controllermanager.go:462] Starting certificates.k8s.io/v1alpha1 apis
I1128 22:45:12.452989 1 controllermanager.go:464] Starting certificate request controller
E1128 22:45:12.454607 1 controllermanager.go:474] Failed to start certificate controller: open /etc/kubernetes/ca/ca.pem: no such file or directory
From what i can tell /etc/kubernetes/is included from below:
$ cat /etc/kubernetes/manifests/kube-controller-manager.yaml
apiVersion: v1
kind: Pod
metadata:
name: kube-controller-manager
namespace: kube-system
spec:
hostNetwork: true
containers:
- name: kube-controller-manager
image: quay.io/coreos/hyperkube:v1.4.3_coreos.0
command:
- /hyperkube
- controller-manager
- --master=http://127.0.0.1:8080
- --leader-elect=true
- --service-account-private-key-file=/etc/kubernetes/ssl/apiserver-key.pem
- --root-ca-file=/etc/kubernetes/ssl/ca.pem
livenessProbe:
httpGet:
host: 127.0.0.1
path: /healthz
port: 10252
initialDelaySeconds: 15
timeoutSeconds: 1
volumeMounts:
- mountPath: /etc/kubernetes/ssl
name: ssl-certs-kubernetes
readOnly: true
- mountPath: /etc/ssl/certs
name: ssl-certs-host
readOnly: true
volumes:
- hostPath:
path: /etc/kubernetes/ssl
name: ssl-certs-kubernetes
- hostPath:
path: /usr/share/ca-certificates
name: ssl-certs-host
The -k
flag in curl
ignores and hides TLS/SSL warnings. If you remove -k
you will see this this is in fact a invalid TLS certificate.
From the curl
manual:
-k, --insecure Allow connections to SSL sites without certs (H)