How would I start a specific number of replicas of the same image, when that number is defined at startup?
On startup I need to call an API endpoint which returns a number. I then want to use this number to deploy that number of replicas of a pod (with each pod being aware of what order it was started in, even after restarts, etc).
For example the API endpoint returns 15, and 15 replicas are started with each having an 'order' / index number of 1 - 15, and maintaining always having a single pod with an 'order' number for each number between 1-15.
I was thinking of using an init container to call the API endpoint, I can't find how to then start that number of replicas and pass the 'order' to the pod.
Your problem can be solved in several ways. You can use Statefulset to identify your pods, but you won't be able to number them from 1 to 15. Statefulset behaves slightly differently. Take a look at the source code:
// for any empty indices in the sequence [0,set.Spec.Replicas) create a new Pod at the correct revision
for ord := 0; ord < replicaCount; ord++ {
if replicas[ord] == nil {
replicas[ord] = newVersionedStatefulSetPod(
currentSet,
updateSet,
currentRevision.Name,
updateRevision.Name, ord)
}
}
For a StatefulSet with X replicas, the numbering will start from 0 up through X-1 (see: Ordinal Index).
I think you might be interested in using Cronjob that runs your custom script periodically on a given schedule. This script can use the Discord Gateway Bot endpoint to determine the recommended number of shards and automatically scale up your bot when Discord recommends it. A good example of such an approach is Marver - a K8s StatefulSet autoscaler. Just take into account that this was made in 2018, so it will need to change to accommodate your Kubernetes version. Additionally, in order to use Marver (or other similar tools), the following requirements must be met:
This project requires that you already be using Kubernetes, and assume you have some understand of how Kubernetes works. It also assumes that you have your bot set up to handle changes in the StatefulSet's replica count gracefully. Meaning: if we scale up, all existing shards will need to re-identify with Discord to present the new shard count, and update their local cache as necessary.
Of course you can also use an operator as Turing85 has mentioned in the comment:
Sounds like a job for an operator. I would, however, highly advice against the approach of assigning an id to each pod since this, in essence, gives each pod an identity. This would, in return, mean that we have to use a StatefulSet, not a Deployment. Remember: pods are cattle, not pets.