How to get access to job parameters from ItemReader, in Spring Batch?

Question

This is part of my job.xml:

<job id="foo" job-repository="job-repository">
  <step id="bar">
    <tasklet transaction-manager="transaction-manager">
      <chunk commit-interval="1"
        reader="foo-reader" writer="foo-writer"
      />
    </tasklet>
  </step>
</job>

This is the item reader:

import org.springframework.batch.item.ItemReader;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
@Component("foo-reader")
public final class MyReader implements ItemReader<MyData> {
  @Override
  public MyData read() throws Exception {
    //...
  }
  @Value("#{jobParameters['fileName']}")
  public void setFileName(final String name) {
    //...
  }
}

This is what Spring Batch is saying in runtime:

Field or property 'jobParameters' cannot be found on object of 
type 'org.springframework.beans.factory.config.BeanExpressionContext'

What's wrong here? Where I can read more about these mechanisms in Spring 3.0?

Hyperphagia

Answer 1

As was stated, your reader needs to be 'step' scoped. You can accomplish this via the @Scope("step") annotation. It should work for you if you add that annotation to your reader, like the following:

import org.springframework.batch.item.ItemReader;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

@Component("foo-reader")
@Scope("step")
public final class MyReader implements ItemReader<MyData> {
  @Override
  public MyData read() throws Exception {
    //...
  }

  @Value("#{jobParameters['fileName']}")
  public void setFileName(final String name) {
    //...
  }
}

This scope is not available by default, but will be if you are using the batch XML namespace. If you are not, adding the following to your Spring configuration will make the scope available, per the Spring Batch documentation:

<bean class="org.springframework.batch.core.scope.StepScope" />

Bilski

StepScope is now available by default – Chary

There's also a shortcut since v2.2 using @StepScope with @Bean – Anamariaanamnesis

@Scope("step") didn't work for me, whereas @StepScope did – Capstan

Spring doesn't really like final classes – Greensboro

we can annotate the @StepScope on method – Trudey

Answer 2

If you want to define your ItemReader instance and your Step instance in a single JavaConfig class. You can use the @StepScope and the @Value annotations such as:

@Configuration
public class ContributionCardBatchConfiguration {

   private static final String WILL_BE_INJECTED = null;

   @Bean
   @StepScope
   public FlatFileItemReader<ContributionCard> contributionCardReader(@Value("#{jobParameters['fileName']}")String contributionCardCsvFileName){

     ....
   }

   @Bean
   Step ingestContributionCardStep(ItemReader<ContributionCard> reader){
         return stepBuilderFactory.get("ingestContributionCardStep")
                 .<ContributionCard, ContributionCard>chunk(1)
                 .reader(contributionCardReader(WILL_BE_INJECTED))
                 .writer(contributionCardWriter())
                 .build();
    }
}

The trick is to pass a null value to the itemReader since it will be injected through the @Value("#{jobParameters['fileName']}") annotation.

Thanks to Tobias Flohre for his article : Spring Batch 2.2 – JavaConfig Part 2: JobParameters, ExecutionContext and StepScope

Bleareyed

How will we test this? – Dan

Answer 3

Pretty late, but you can also do this by annotating a @BeforeStep method:

@BeforeStep
    public void beforeStep(final StepExecution stepExecution) {
        JobParameters parameters = stepExecution.getJobExecution().getJobParameters();
        //use your parameters
}

Inquest

This is only useful, if the usage of the job-parameters are not altering the state of the ItemReader. If they are e.g. used to designate what data to read, then concurrent execution of the reader might cause incorrect behaviour. – Unbowed

@tveon, this can be solved by using StepScope when declaring the reader bean, right? – Chute

@Chute yes, and unless you reuse the reader later in the same job, then you can also solve it with JobScope – Unbowed

Answer 4

To be able to use the jobParameters I think you need to define your reader as scope 'step', but I am not sure if you can do it using annotations.

Using xml-config it would go like this:

<bean id="foo-readers" scope="step"
  class="...MyReader">
  <property name="fileName" value="#{jobExecutionContext['fileName']}" />
</bean>

See further at the Spring Batch documentation.

Perhaps it works by using @Scope and defining the step scope in your xml-config:

<bean class="org.springframework.batch.core.scope.StepScope" />

Bookcraft

I'm interested in annotations specifically. This format of configuration is old and documented. – Hyperphagia

You haven't stated that you were interested in an annotations-only solution only. Thanks for the vote anyhow. – Bookcraft

Answer 5

Complement with an additional example, you can access all job parameters in JavaConfig class:

@Bean
@StepScope
public ItemStreamReader<GenericMessage> reader(@Value("#{jobParameters}") Map<String,Object> jobParameters){
          ....
}

Overcareful

Answer 6

While executing the job we need to pass Job parameters as follows:

JobParameters jobParameters= new JobParametersBuilder().addString("file.name", "filename.txt").toJobParameters();   
JobExecution execution = jobLauncher.run(job, jobParameters);  

by using the expression language we can import the value as follows:

 #{jobParameters['file.name']}

Housewarming

Answer 7

'technical' writing.

configuration class.

    @Autowired
    @Qualifier("testReader")
    private testReader reader;


    @Bean(name = "testJob")
    public Job testJob(@Autowired @Qualifier("testStep") Step step) {
        return jobBuilderFactory
                .get("testJob")
                .incrementer(new RunIdIncrementer())
//                .listener(new JobCompletionListener())
                .start(step)
                .build();

    }

    @Bean("testStep")
    @JobScope
    public Step testStep(@Value("#{jobParameters['key']}") String key) {
        return stepBuilderFactory.get("testStep")
                .<UniqueUserVO, List<UniqueUser>>chunk(500)
                .reader(reader.setKey(key).reader())
                .processor(processor.processor())
                .writer(writer.writer())
                .build();

    }

reader interface class.

public interface Reader<T> {

    /**
     * reader 
     * @return
     */
    ItemReader<T> reader();
}

reader class

@Component
public class TestReader implements Reader<UniqueUserVO> {
    private String key;

    public TestReader setKey(String key) {
        this.key= key;
        return this;
    }
    @Override
    public ItemReader<UniqueUserVO> reader() {
       xxxxx
    }
}

Saturnalia

Answer 8

Let's consider a scenario where you need to access JobParameters in an ItemWriter or ItemReader to set values like the file path for writing. In this case, you can utilize a JobParameterExecutionContextCopyListener.

Suppose you have a non-bean class where you create your STEP reader/writer. You can then create a JobParameterExecutionContextCopyListener and specify the keys you want to extract, like myDate:

private TaskletStep internalBuild() {
    try {
        JobParameterExecutionContextCopyListener listener = new JobParameterExecutionContextCopyListener();
        listener.setKeys(new String []{"myDate"});
        return new StepBuilder("MY_STEP", jobRepository)
                .<T, T>chunk(batchSize, transactionManager)
                .reader(itemReader())
                .processor(new PassThroughItemProcessor<>())
                .writer(itemWriter())
                .listener(listener)
                .build();
    } catch (IOException e) {
        throw new SqlToCsvException(e);
    }
}

Next, in your writer (e.g., FlatFileItemWriter), if you want to set the resource based on the date obtained from the JobParameters, you can override the open method to handle this logic:

    public FlatFileItemWriter itemWriter() {
    FlatFileItemWriter writer = new FlatFileItemWriter<T>(){
        // Override the resource given from the executionContext.
        @Override
        public void open(ExecutionContext executionContext) throws ItemStreamException {
            this.setResource(new FileSystemResource(getMyFileNameFromDate((LocalDate) executionContext.get("myDate"))));
            super.open(executionContext);
        }
    };

    // Other configurations for the writer...

    return writer;
}

This approach allows you to dynamically set the resource (file path in this case) based on the JobParameter (e.g., date) provided during the job execution.

Hepato

Answer 9

This could be an easier manner to do it:

@Configuration
@Setter
@StepScope
public  class Reader extends FlatFileItemReader<Object> {

public Reader(@Value("#{jobParameters['filePath']}") String resource){
    setResource(new FileSystemResource(resource));
   }

}

Gunthar

Answer 10

You can use StepExecution context inside a method annotated with @BeforeStep annotation inside your item reader to get the Job parameters & set it to a variable, which you can use inside your read method.

In my case I've written something like this :-

@Component
@RequiredArgsConstructor
public class SpelItemReader implements ItemReader<BatchRequest>{

private String requestId;


@Override
public BatchRequest read() {
   //access the request-id variable here
}

@BeforeStep
public void beforeStep(StepExecution stepExecution) {
    requestId = stepExecution.getJobParameters().getString("requestId");
}

}

Auger

Answer 11

Did you declare the jobparameters as map properly as bean?

Or did you perhaps accidently instantiate a JobParameters object, which has no getter for the filename?

For more on expression language you can find information in Spring documentation here.

Willtrude