Given 3 jobs (A,B,C) in 3 Helm charts, is it possible to run A and B jobs in parallel, then start job C as soon as both of them are finished? Think of a barrier, in which a bunch of stuff needs to be finished before moving on.
Even if I put A and B charts as sub-charts for C chart, then all 3 are started in parallel.
I already have an workaround for this: add an external check for A and B job finishing, then start C. Still, I would prefer a Helm-based solution, if it exists.
Indeed, Kubernetes seems to be quite basic when it comes to scheduling jobs. We'll move to Luigi at some point in the future for advanced job scenarios.
For now, I wrote this small awk, bash-based workaround for this. Perhaps it could help others in a similar situation.
while true; do
sleep 60
# done means output is 'SUCCESSFUL\n1' only
is_done=$(kubectl get jobs 2>&1 | awk '{print $3}' | uniq | awk 'BEGIN{no_lines=0; no_all_lines=0} {no_all_lines++; if(NR==1 && $1=="SUCCESSFUL") {no_lines++}; if(NR==2 && $1=='1') {no_lines++}} END{ if(no_lines==no_all_lines) {print "1"} else {print "0"}}'
)
if [ ${is_done} = "1" ]; then
echo "Finished all jobs"
break
else
echo "One or more jobs are still running, checking again in 1 minute"
fi
done
Kubernetes isn't a batch job framework/scheduler and does not fit your advanced batch framework requirements.
My recommendation would be to use a real batch framework like Luigi which also supports scheduling Luigi jobs on kubernetes. Look here for an example how to do this.