I'm trying to run docker image on multiple data using indexed jobs.
It seems that indexed jobs is available since Kubernetes 1.21,
but my job on google kubernetes engine doesn't recognize the variable $JOB_COMPLETION_INDEX.
my yaml file looks like this:
spec:
containers:
- args:
- --job_id="$JOB_COMPLETION_INDEX"If I set job_id=0, it works ok.
Even if I add the following in the top level spec,
spec:
completions: 3
parallelism: 3
completionMode: Indexedyaml file automatically remove the part (I'm using google cloud web console for editing yaml file).
I tried both 1.22.3-gke.700 and 1.21.5-gke.1302 with Kubernetes alpha features, but nothing worked.
Is indexed job available now in google kubernetes engine?
where did you get args? Use command instead. It will be look something like this:
apiVersion: batch/v1
kind: Job
metadata:
name: 'sample-job'
spec:
completions: 3
parallelism: 3
completionMode: Indexed
template:
spec:
restartPolicy: Never
containers:
- command:
- 'bash'
- '-c'
- 'echo "My partition: ${JOB_COMPLETION_INDEX}"'
image: 'docker.io/library/bash'
name: 'sample-load'It's from official documentation.
Or this example also works:
[...]
spec:
subdomain: my-job-svc
containers:
- name: task
image: registry.example.com/processing-image
command: ["./process", "--index", "$JOB_COMPLETION_INDEX", "--hosts-pattern", "my-job-{{.id}}.my-job-svc"]