How to set a master pod and worker pods for remote chunking with Spring Batch in OpenShift?

8/2/2021

I'm trying to use Spring Batch to parse a large flatfile (order of 10s of millions of records) with remote partitioning and chunking. I've achieved this by following various tutorials with a single OpenShift pod. My question is if I am to horizontally scale to multiple pods, how do I set it so that only a single pod runs the master service/bean and the others all run only workers?

Right now my workaround has been having two deployments. One used the master profile with a single pod and the other uses the worker profile with multiple pods but this feels like an inelegant solution.

Thanks in advance!

-- Aurarius
chunking
kubernetes
openshift
partitioning
spring-batch

0 Answers