What is the maximum number of jobs that you can submit to a kubernetes cluster?

8/20/2019

I have a Kubernetes cluster and I have tested submitting 1,000 jobs at a time and the cluster has no problem handling this. I am interested in submitting 50,000 to 100,000 jobs and was wondering if the cluster would be able to handle this?

-- Morgan Gladden
kubernetes

1 Answer

8/21/2019

Yes you can but only if only don't run out of resources or you don't exceed this criteria regarding building large clusters.

Usually you want to limit your jobs in some way in order to better handle memory and CPU or to adjust it in any other way according to your needs.

So the best practice in your case would be to:

  • set as many jobs as you want (bear in mind the building large clusters criteria)
  • observe the resource usage
  • if needed use for example Resource Quotas in order to limit resources used by the jobs

I hope you find this helpful.

-- OhHiMark
Source: StackOverflow