Hi I run the SCDF in openshift. I have two questions here.
Question 1
When I execute the task using Schedule task option once per minute from SCDF, it creates a pod for each execution in openshift. And the pods are keep running even after the job completed. Assuming there would be jobs running every 15 minutes, I guess lots of pods would be in running mode at the end of a day. I assume there should be a way to move those PODS to completed status once the job returns completed status. But I couldn't figure out how.
A simple task that I'm using
@Configuration
@EnableBatchProcessing
@EnableTask
public class Facdata_loaderLoader extends BatchJobLoader{
@Bean
public JdbcCursorItemReader<> data_loaderReader()
{
return JdbcCursorItemReader; // An instance of it
}
@Bean
public Step step01_data_loader()
{
return stepBuilderFactory
.get("S1-CU-Loaddata_loader")
.<DomainObjects, data_loader> chunk(100)
.reader(data_loader_Reader())
.writer(data_loader_writer)
.listener(listener)
.allowStartIfComplete(false)
.startLimit(2)
.build();
}
@Bean
public Job loaddata_loader()
{
return jouilderFactory
.get(data_loader_JOB_NAME)
.incrementer(new RunIdIncrementer())
.start(step_00_job_details_session())
.next(step_01_data_loader())
.listener(jobExecutionListener)
.build();
}
}
Is there any configuration I'm missing here?
Question 2
Upon running the task once per minute after few executions, the next job execution started throwing error as "job Instance already exists". Exception below.
Caused by: org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException: A job instance already exists and is complete for parameters={-spring.cloud.task.executionid=63, run.id=5, -spring.cloud.data.flow.platformname=default, -spring.datasource.username=User_name, -spring.cloud.task.name=SingleJob, -spring.datasource.password="Password", -spring.datasource.driverClassName=oracle.jdbc.OracleDriver, -spring.datasource.url="datasource_Url". If you want to run this job again, change the parameters.
at org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution(SimpleJobRepository.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
In normal spring batch , I could start my job with a jobLauncher and pass the Job_Id as parameter. Like below.
JobParameters param = new JobParametersBuilder()
.addString("JobID", String.valueOf(System.currentTimeMillis())).toJobParameters();
jobLauncher.run(facSessionsLoader.loadBBSessions(), param);
But how can I do that in Task when I schedule it through the SCDF ?