I have a Kubernetes Job that has, for instance, parallelism set to 4. When this job is created, I might want to scale this out to, say, 8. But it seems like edit
ing the Job and setting parallelism to 8 doesn't actually create more pods in the Job.
Am I missing something? Or is there no way to scale out a Job?
Thanks in advance!
kubectl scale
doesn't support Job
resource anymore. Here is working solution for me:
kubectl edit job [JOB_NAME]
parallelism
field to appropriate value for you.It will create new pods or will terminate existing ones.
So as per job documentation you can still scale a Job running the following command:
kubectl scale job my-job --replicas=[VALUE]
Simple test shows that this option works right now as expected, but will be really deprecated in a future
kubectl scale job is DEPRECATED and will be removed in a future version.
The ability to use kubectl scale jobs is deprecated. All other scale operations remain in place, but the ability to scale jobs will be removed in a future release.
The reason is: Deprecate kubectl scale job
Use below Job yaml as an example to create job:
apiVersion: batch/v1
kind: Job
metadata:
name: test-job
spec:
template:
spec:
containers:
- name: pi
image: perl
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2010)"]
restartPolicy: Never
completions: 1000
parallelism: 5
Now lets test behavior:
kubectl describe jobs.batch test-job
Parallelism: 5
Completions: 1000
Start Time: Fri, 17 May 2019 16:58:36 +0200
Pods Statuses: 5 Running / 21 Succeeded / 0 Failed
kubectl get pods | grep test-job | grep Running
test-job-98mlv 1/1 Running 0 13s
test-job-fs2hb 1/1 Running 0 8s
test-job-l8n6v 1/1 Running 0 16s
test-job-lbh46 1/1 Running 0 13s
test-job-m8btl 1/1 Running 0 2s
Changing parallelism with kubectl scale
:
kubectl scale jobs.batch test-job --replicas=10
kubectl describe jobs.batch test-job
Parallelism: 10
Completions: 1000
Start Time: Fri, 17 May 2019 16:58:36 +0200
Pods Statuses: 10 Running / 87 Succeeded / 0 Fail
kubectl get pods | grep test-job | grep Running
test-job-475zf 1/1 Running 0 10s
test-job-5k45h 1/1 Running 0 14s
test-job-8p99v 1/1 Running 0 22s
test-job-jtssp 1/1 Running 0 4s
test-job-ltx8f 1/1 Running 0 12s
test-job-mwnqb 1/1 Running 0 16s
test-job-n7t8b 1/1 Running 0 20s
test-job-p4bfs 1/1 Running 0 18s
test-job-vj8qw 1/1 Running 0 18s
test-job-wtjdl 1/1 Running 0 10s
And the last step that i believe will be the most interesting for you - you can always edit your job using kubectl patch command
kubectl patch job test-job -p '{"spec":{"parallelism":15}}'
kubectl describe jobs.batch test-job
Parallelism: 15
Completions: 1000
Start Time: Fri, 17 May 2019 16:58:36 +0200
Pods Statuses: 15 Running / 175 Succeeded / 0 Failed
kubectl get pods | grep test-job | grep Running | wc -l
15
There's a scale
command:
kubectl scale job my-job --replicas=[VALUE]
From docs:
kubectl scale causes the number of concurrently-running Pods to change. Specifically, it changes the value of parallelism to the [VALUE] you specify.