I have a Kubernetes cluster executing Jobs of different types (I'm not using parallelism, these are independent and unrelated jobs).
Each Job has a label userid=xyz, that's the id of the user who requested the job (users submit these requests trough my app, not to the cluster directly).
I want to limit the number of Jobs running at any given time by tag, for example, limit the jobs with userid=foo to 10, and userid=bar to 50. Is it possible to achieve this on Kubernetes?
You could use a ResourceQuota to accomplish something similar.
There is no direct way to limit the job objects themselves using Resource Quota, but there may be other ways you can accomplish the same effect, by limiting the number of CPUs, or the number of pods.
For custom policies and logic around labels as you mentioned, admission control is a good place to enforce it. Extensible admission control is going into alpha in Kubernetes 1.7.