What is the recommended cert strategy for multi-master K8s clusters?

8/7/2018

Is it atypical for multi-master K8s cluster deployments to use unique certs per service, per controller node? Most guides I've seen generate unique certs per service (API, Controller, Scheduler) and then use those certs for the eponymous service on each Controller node.

Does Kubernetes disallow or discourage unique certs per service, per node? With DNS/IP SANs it should be possible to still have each service respond to a singular cluster address, so I'm curious if this decision is one for the sake of simpler instructions, or if it's actually some requirement I'm missing.

Thank you.

-- akutz
kubernetes
kubernetes-security

1 Answer

8/8/2018

Does Kubernetes disallow or discourage unique certs per service, per node? With DNS/IP SANs it should be possible to still have each service respond to a singular cluster address, so I'm curious if this decision is one for the sake of simpler instructions, or if it's actually some requirement I'm missing

When we have running Kubernetes cluster/s, we can have thousands of private and public keys, and different components usually do not know if they are valid. So there is the Certificate Authority that is a 3rd party entity which tells the interested elements "this certificate is trusted".

documentation:

Every Kubernetes cluster has a cluster root Certificate Authority (CA). The CA is generally used by cluster components to validate the API server’s certificate, by the API server to validate kubelet client certificates, etc.

This actually shows that you can have different certificates in each cluster, but it is not a requirement, you can imagine many different combinations of your CA. You can have one global CA that is responsible for signing all the keys, or one CA for each cluster, one for internal communication and one for external, etc.

Any request that presents a client certificate signed by cluster CA will be considered authenticated. In that authentication process, it should be possible to obtain a username from the Common Name field (CN) and a group from the Organization field of that certificate. So the answer would be yes, you can use different certs per service, node or any component in the cluster unless it is signed by the Certificate Authority in the cluster.

When creating certificates for the master with multiple masters (HA cluster), you have to make sure that the load-balancers IP and DNS name is a part of that certificate. Otherwise, whenever a client will try to talk through the API server through an LB, a client will complain since the common name on the certificate will be different than the one it wants to communicate with.

Going further, each of the core cluster components has his own client certificate in addition to the main certificate because each of them will have a different access level to the cluster with different common names. It is noteworthy that kubelet has a little different certificate name as each kublet will have a different identity (hostname where the kubelet is running will be a part of the certificate) it is related with other features like NodeAuthorizer and Node Restriction Admission Plugin. These features are important from the perspective of least privilege - they limit the in another case unrestricted access and interaction of the kubelet to apiserver. Using this feature, you can limit kubelet to only being able to modify its node resource instead of the whole cluster, as the same it will only be able to read its nodes secrets instead of all secrets in the cluster, etc.

EDIT - following comment discussion:

Assuming you are asking for opinion on why does more people do not use multiple certificates, I think that it is because it does not realy rise security in a significant matter. As the certs are not as important as the CA - which is a trusted guarantor that the entities can talk with each other securely. So you can make multiple CA's - the reason for that would be more of a HA approach than security. Of course if you have a trusted CA, you don't need more certificate kinds as you actually do not reach any goal by increasing the number of them.

-- aurelius
Source: StackOverflow