Transactions issues with spring scheduler & spring batch item writer

Question

I am trying to run a spring batch job using the @Scheduled annotation as follows:

@Scheduled(cron = "* * * * * ?")
public void launchMessageDigestMailing() throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException {
    jobLauncher.run(messagesDigestMailingJob, new JobParametersBuilder().addDate("execution_date", new Date()).toJobParameters());
}

I get the following error:

javax.persistence.TransactionRequiredException: Executing an update/delete query

However, this error does not occur when I launch the job from a spring mvc controller as follows:

@GetMapping("/messageDigestMailing")
public void launchMessageDigestMailing() throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException {
    jobLauncher.run(messagesDigestMailingJob, new JobParametersBuilder().addDate("execution_date", new Date()).toJobParameters());
}

I understand Spring batch manages the transactions and does not require a @Transactional / @EnableTransactionManagement. Why then do I get the above exception?

All samples I have found on the web use a ResourcelessTransactionManager (see https://www.mkyong.com/spring-batch/spring-batch-and-spring-taskscheduler-example) but I do need for my job executions to be persisted to database.

Can someone please help?

edit: Here is the stack trace (you can see the spring batch ItemWriter is referred below):

Caused by: org.springframework.dao.InvalidDataAccessApiUsageException: Executing an update/delete query; nested exception is javax.persistence.TransactionRequiredException: Executing an update/delete query
        at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:413) ~[spring-orm-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:246) ~[spring-orm-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:488) ~[spring-orm-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:59) ~[spring-tx-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213) ~[spring-tx-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:147) ~[spring-tx-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:133) ~[spring-data-jpa-1.11.3.RELEASE.jar:na]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:57) ~[spring-data-commons-1.13.3.RELEASE.jar:na]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) [spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at com.sun.proxy.$Proxy129.markMessagesAsNotificationSent(Unknown Source) ~[na:na]
        at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_102]
        at com.bignibou.writer.MessagesDigestMailerItemWriter.write(MessagesDigestMailerItemWriter.java:49) ~[main/:na]
        at org.springframework.batch.core.step.item.SimpleChunkProcessor.writeItems(SimpleChunkProcessor.java:175) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
        at org.springframework.batch.core.step.item.SimpleChunkProcessor.doWrite(SimpleChunkProcessor.java:151) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
        at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor$3.doWithRetry(FaultTolerantChunkProcessor.java:328) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
        at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:286) ~[spring-retry-1.2.0.RELEASE.jar:na]
        ... 48 common frames omitted

edit 2: When I try to enable transactions with @EnableTransactionManagement and:

@Scheduled(cron = "* * * * * ?")
@Transactional
public void launchMessageDigestMailing() throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException {
    jobLauncher.run(messagesDigestMailingJob, new JobParametersBuilder().addDate("execution_date", new Date()).toJobParameters());
}

I get the following exception:

2017-05-20 17:04:04.013 ERROR 5574 --- [pool-2-thread-1] o.s.s.s.TaskUtils$LoggingErrorHandler    : Unexpected error occurred in scheduled task.

java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove @Transactional annotations from client).
        at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:168) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at com.sun.proxy.$Proxy117.createJobExecution(Unknown Source) ~[na:na]
        at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:125) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_102]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_102]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102]
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at com.sun.proxy.$Proxy142.run(Unknown Source) ~[na:na]
        at com.bignibou.scheduler.Scheduler.launchMessageDigestMailing(Scheduler.java:33) ~[main/:na]
        at com.bignibou.scheduler.Scheduler$$FastClassBySpringCGLIB$$f0615234.invoke(<generated>) ~[main/:na]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) ~[spring-tx-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) ~[spring-tx-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) ~[spring-tx-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673) ~[spring-aop-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at com.bignibou.scheduler.Scheduler$$EnhancerBySpringCGLIB$$26d7d986.launchMessageDigestMailing(<generated>) ~[main/:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_102]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_102]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102]
        at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) ~[spring-context-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) [spring-context-4.3.8.RELEASE.jar:4.3.8.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_102]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_102]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_102]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_102]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_102]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_102]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]

edit 3: Here is my reader configuration:

@Bean
@StepScope
public static ItemReader<UserAccount> jpaPagingItemReader(EntityManagerFactory entityManagerFactory) {
    final JpaPagingItemReader<UserAccount> reader = new JpaPagingItemReader<>();
    reader.setEntityManagerFactory(entityManagerFactory);
    reader.setQueryString("SELECT ua FROM UserAccount ua JOIN FETCH ua.receivedMessages msg " +
            "WHERE msg.notificationSent = false AND msg.messageRead = false AND ua.enabled = true AND ua.emailNotification = true");
    return reader;
}

Caviness

so you get first exception when not using @Transactional on writer? – Reveille

@SabirKhan I get the first exception when I don't use @Transactional at all. And the second when I annotate the @Scheduled method with @Transactional. – Caviness

How is your reader and transaction manager configured? – Ashok

@MichaelMinella, I have not configured any transaction manager explicitly. I am adding the reader configuration as an edit. – Caviness

You need to for JPA. You need to use the JPA transaction manager. – Ashok

@MichaelMinella Thanks for your input. I forgot to mention that I use spring boot which autoconfigures a JpaTransactionManager for me. Do you confirm I am required to explicitely configure a JpaTransactionManager? Also please note that I don't get the exception above when I run the job through the controller, only when the scheduler is used... What are your thoughts about this? – Caviness

I would confirm that the correct transaction manager is being used. You can use the --debug option or the /beans accuator endpoint to verify. Using it without the scheduler, if you are using the session per request pattern, you'd have a transaction created that way anyways (aka outside of batch). – Ashok

Umm. Thanks. What you say about the transaction been created by the web layer is interesting... Bear with me whilst I confirm that the correct tx manager is used. I'll update shortly. – Caviness

@MichaelMinella, thanks a lot Michael. That was indeed the issue. I have posted an answer taking into account your comments and advice. – Caviness

Answer 1

Following Michael Minella's advice of inspecting the configured beans (/beans endpoint), I noticed that the configured & declared Transaction Manager, named transactionManager is not of type JpaTransactionManager. See below:

{
"bean": "transactionManager",
"aliases": [],
"scope": "singleton",
"type": "com.sun.proxy.$Proxy121",
"resource": "class path resource [org/springframework/batch/core/configuration/annotation/SimpleBatchConfiguration.class]",
"dependencies": [],
},

So I configured a JpaTransactionManager as follows:

@Configuration
public class TransactionManagerConfiguration {

    @Bean
    public PlatformTransactionManager transactionManager(EntityManagerFactory entityManagerFactory) {
        JpaTransactionManager transactionManager = new JpaTransactionManager();
        transactionManager.setEntityManagerFactory(entityManagerFactory);
        return transactionManager;
    }
}

and the issue was sorted. Note that I did not use any of the @Transactional or @EnableTransactionManagement in my configuration.

Caviness

Answer 2

I have not used JPA in my batch writers so I am not sure what might be causing it but two pointers,

1.Do you really need this cron expression in scheduler?

* * * * * ?

Its like telling - keep launching new job instances all the time, without any break or am I missing something?

I think, you should try to introduce some time gap in between launches and change your cron expression accordingly.

2.There are questions on SO like this and as pointed in first answer, there might be a conflict in container managed and application managed transactions. I guess, you are running your job in some container too. Try to put relevant portions of writer code throwing exception.

Reveille

Hi Sabir. The above cron expression is just here for dev/debug purposes. It won't make it to production :-) – Caviness

As far as point 2 is concerned, I find it odd that the job can be launched from the controller without problems but that the same code won't work from a @Scheduled method. Do you see my point? Any idea? – Caviness