Created a cluster in EKS (Kubernetes 1.11.5) with multiple node groups however I'm noticing that in the extension-apiserver-authentication
configmap that client-ca-file
key is missing.
I assume this is due to the way Kubernetes API service is initiated. Has anyone else come across this issue ?
I came across this problem while deploying certificate manager which queries the api server with GET https://10.100.0.1:443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication
.
In GKE this isnt a problem as extension-apiserver-authentication
configmap already includes client-ca-file
.
extension-apiserver-authentication
configmap in AWS,
apiVersion: v1
data:
requestheader-allowed-names: '["front-proxy-client"]'
requestheader-client-ca-file: |
<certificate file>
requestheader-extra-headers-prefix: '["X-Remote-Extra-"]'
requestheader-group-headers: '["X-Remote-Group"]'
requestheader-username-headers: '["X-Remote-User"]'
kind: ConfigMap
metadata:
creationTimestamp: 2019-01-14T04:56:51Z
name: extension-apiserver-authentication
namespace: kube-system
resourceVersion: "39"
selfLink: /api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication
uid: ce2b6f64-17b8-11e9-a6dd-021a269d3ce8
However in GKE,
apiVersion: v1
data:
client-ca-file: |
<client certificate file>
requestheader-allowed-names: '["aggregator"]'
requestheader-client-ca-file: |
<certificate file>
requestheader-extra-headers-prefix: '["X-Remote-Extra-"]'
requestheader-group-headers: '["X-Remote-Group"]'
requestheader-username-headers: '["X-Remote-User"]'
kind: ConfigMap
metadata:
creationTimestamp: 2018-05-24T12:06:33Z
name: extension-apiserver-authentication
namespace: kube-system
resourceVersion: "32"
selfLink: /api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication
uid: e6c0c431-5f4a-11e8-8d8c-42010a9a0191
Your assumptions might be correct, I mean your issue might really have to do with the way how api-server is initiated in EKS, and when lacking the --client-ca-file option, according the apiserver source code:
Cluster doesn't provide client-ca-file in configmap/%s in %s, so client certificate authentication to extension api-server won't work.
I`m wondering if you could create a client-ca by yourself, and update afterwards extension-apiserver-authentication ConfigMap. After all in EKS we have access to CA (the way to get it here), which can serve us to sign the client certificates, like described in GitHub page with sample api-server.
As a side comment:
I noticed, that on EKS, components like kubelet , are using aws-iam-authenticator
(content of /var/lib/kubelet/kubeconfig
on cluster node):
apiVersion: v1
kind: Config
users:
- name: kubelet
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- token
- -i
- eks-api-eval
command: /usr/bin/aws-iam-authenticator
env: null
in opposite to GKE, where client-ca is used as a way for authenticating clients:
apiVersion: v1
kind: Config
users:
- name: kubelet
user:
client-certificate: /etc/srv/kubernetes/pki/kubelet.crt
client-key: /etc/srv/kubernetes/pki/kubelet.key
I've also run into this issue while trying to use cert-manager on an AWS EKS cluster. It is possible to inject the certificate yourself using the certificate obtained from the AWS CLI. Follow these steps to address this issue:
Obtain the Certificate
The certificate is stored Base64 encoded and can be retrieved using
aws eks describe-cluster \
--region=${AWS_DEFAULT_REGION} \
--name=${CLUSTER_NAME} \
--output=text \
--query 'cluster.{certificateAuthorityData: certificateAuthority.data}' | base64 -D
Inject the Certificate
Edit configMap/extension-apiserver-authentication under the kube-system namespace: kubectl -n kube-system edit cm extension-apiserver-authentication
Under the data section, add the CA under a new config entry named client-ca-file. For example:
client-ca-file: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----