Is there a cloud-native friendly method to select a master among the replicas?

1/29/2021

Is there a way in Kubernetes to upgrade a given type of pod first when we have a deployment or stateful set with two or more replicas ( where one pod is master and others are not)?

My requirement to be specific is to ensure when calling upgrade on deployment/statefull set is to upgrade master as the last pod under a given number of replicas..

-- DT.
kubernetes

2 Answers

1/30/2021

The easiest way is probably having master in one deployment/statefulset, while followers in another deployment/statefulset. This approach ensure update is persist and can make use of update strategy in k8s.

The fact that k8s does not differentiate pod by containers nor any role specific to user application architecture ('master'); it is better to manage your own deployment when you have specific sequence that is outside of deployment/statefulset control. You can patch but change will not persist rollout restart.

-- gohm'c
Source: StackOverflow

1/30/2021

The only thing that's built into Kubernetes is the automatic sequential naming of StatefulSet pods.

If you have a StatefulSet, one of its pods is guaranteed to be named statefulsetname-0. That pod can declare itself the "master" for whatever purposes this is required. A pod can easily determine (by looking at its hostname(1)) whether it is the "master", and if it isn't, it can also easily determine what pod is. Updates happen by default in numerically reverse order, so statefulsetname-0 will be upgraded last, which matches your requirement.

StatefulSets have other properties, which you may or may not want. It's impossible for another pod to take over as the master if the first one fails; startup and shutdown happens in a fairly rigid order; if any part of your infrastructure is unstable then you may not be able to reliably scale the StatefulSet.

If you don't want a StatefulSet, you can implement your own leader election in a couple of ways (use a service like ZooKeeper or etcd that can help you maintain this state across the cluster; bring in a library for a leader-election algorithm like Raft). Kubernetes doesn't provide this on its own. The cluster will also be unaware of the "must upgrade the leader last" requirement, but if the current leader is terminated, another pod can take over the leadership.

-- David Maze
Source: StackOverflow