I have configured a CronJon in K8S that creates a pod every 10 minutes. The running container can complete in anything from 5 minutes to 2 hours, no way to tell.
Sometimes, when the computation is taking very long, i end up with far too many pods created.
Is there a way to limit the number of running pods, instances of the specific CronJob?
EDIT
To clarify, I want concurrent runs, i want to limit the number of running concurrent instances to a certain large number N.
Refer
kubectl explain cronjob.spec.jobTemplate.specThis will give you "parallelism" key, you can map the "parallelism" to the number 'N', as required by you.
This will allow 'N' number of pods to run in parallel.
I guess this helps.
Following up from @Tushar's reply, in combination with the information from this post: https://www.alibabacloud.com/blog/kubernetes-cronjobs---part-2-parallelism_595022
The solution is a combination of three settings, that are counter indicative if looked separately.
To achieve a maximum number of 10 running pods, instances of a cronjob i did the following
parallelism: 10
completions: 8
concurrencyPolicy: ForbidEach time the tick will hit, CronJob will create 10 parallel pods, but on the next tick - unless 8 of the previous pods have completed, will not trigger the run due to concurrencyPolicy: Forbid
This also means that the tick will be skipped if say only three instances are still running, but this is close enough to what i was looking for.