Spring Batch: different job launcher for different jobs

Question

I have 2 different jobs (actually more but for simplicity assume 2). Each job can run in parallel with the other job, but each instance of the same job should be run sequentially (otherwise the instances will cannibalize eachother's resources).

Basically I want each of these jobs to have it's own queue of job instances. I figured I could do this using two different thread pooled job launchers (each with 1 thread) and associating a job launcher with each job.

Is there a way to do this that will be respected when launching jobs from the Spring Batch Admin web UI?

Beefsteak

Answer 1

There is a way to specify a specific job launcher for a specific job, but the only way I have found to do it is through use of a JobStep.

If you have a job called "specificJob" this will create another job "queueSpecificJob" so when you launch it, either through Quartz or Spring Batch web admin, it will queue up a "specificJob" execution.

<bean id="specificJobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
    <property name="jobRepository" ref="jobRepository"/>
    <property name="taskExecutor">
        <task:executor id="singleThreadPoolExecutor" pool-size="1"/>
    </property>
</bean>

<job id="queueSpecificJob">
    <step id="specificJobStep">
        <job ref="specificJob" job-launcher="specificJobLauncher" job-parameters-extractor="parametersExtractor" />
    </step>
</job>

Sateia

Answer 2

@ ahbutfore

How are the jobs triggered? Do you use Quartz trigger by any chance? If yes, would implementing/extending the org.quartz.StatefulJob interface in all your jobs do the work for you?

See Spring beans configuration here : https://github.com/regunathb/Trooper/blob/master/examples/example-batch/src/main/resources/external/shellTaskletsJob/spring-batch-config.xml. Check source code of org.trpr.platform.batch.impl.spring.job.BatchJob

You can do more complex serialization (including across Spring batch nodes) using a suitable "Leader Election" implementation. I have used Netflix Curator (an Apache Zookeeper recipe) in my project. Some pointers here : https://github.com/regunathb/Trooper/wiki/Useful-Batch-Libraries

Eternize

Answer 3

Using a shell script you can launch different jobs parallel.

Add an '&' to the end of each command line. The shell will execute them in parallel with it's own execution.

Zing

I don't think shell scripts have anything to do with Spring Batch – Incompletion

Shell script is just for invoking the spring batch jobs as parallel jobs. – Zing

this doesn't address the question of queuing up jobs. i specifically don't want job instances of the same job to run in parallel, i want them to run sequentially. – Beefsteak