Why is kubeadm config's controlPlaneEndpoint necessary?

9/8/2019

It is not possible to join master nodes without having set a controlPlaneEndpoint:

error execution phase preflight:
One or more conditions for hosting a new control plane instance is not satisfied.
unable to add a new control plane instance a cluster that doesn't have a stable controlPlaneEndpoint address
Please ensure that:
* The cluster has a stable controlPlaneEndpoint address.

But if you instead join a worker node (i.e. without --control-plane), then it is not only aware of other nodes in the cluster, but also which ones are masters.

This is because the mark-control-plane phase does:

Marking the node as control-plane by adding the label "node-role.kubernetes.io/master=''" Marking the node as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]

So couldn't masters (--control-plane) join the cluster and use the role label to discover the other control plane nodes?

Is there any such plugin or other way of configuring this behaviour to avoid separate infrastructure for load balancing the API server?

-- OJFord
kubeadm
kubernetes

1 Answer

9/10/2019

Looking at the kubeadm types definition I found this nice description that clearly explains it:

ControlPlaneEndpoint sets a stable IP address or DNS name for the control plane; it can be a valid IP address or a RFC-1123 DNS subdomain, both with optional TCP port. In case the ControlPlaneEndpoint is not specified, the AdvertiseAddress + BindPort are used; in case the ControlPlaneEndpoint is specified but without a TCP port, the BindPort is used. Possible usages are: e.g. In a cluster with more than one control plane instances, this field should be assigned the address of the external load balancer in front of the control plane instances. e.g. in environments with enforced node recycling, the ControlPlaneEndpoint could be used for assigning a stable DNS to the control plane.

This also likely affects the PKI generated by kubernetes, as it will need to know a common name/IP that you will access the cluster via to include in the certs it generates for the api nodes, otherwise these won't match up correctly.

If you really didn't want to have a loadbalancer you might be able to set up a round-robin dns entry with the IP's of all the control plane nodes and try specifying this for the controlPlaneEndpoint value. However this won't help with failover and redundancy, since failed nodes won't be removed from the record, and some clients might cache the address and not try to re-resolve it, thus further prolonging any outages.

Hope this helps.

-- cewood
Source: StackOverflow