How to add new schedule job dynamically with Spring
Asked Answered
B

3

33

I am writing a Spring Boot App

My requirements are - In the resources (src/main/resources) folder if I add new xml files... I should read those files and get some url and other specific settings from each of them. and for those urls I need to download data everyday... So a new scheduler job will start with url and some settings

The new jobs will run in different schedule time which will use cron expression present in the xml files. Also, files will be added dynamically at any point of time.

How to implement it?

Bannock answered 27/10, 2017 at 11:45 Comment(2)
by using scheduling api available in springMettah
Yes JEY.. but dynamically I will add new xml files.. for those I need to start new schedule jobs..Bannock
M
75

If you want to dynamically schedule tasks you can do it without spring by using ExecutorService in particular ScheduledThreadPoolExecutor

Runnable task  = () -> doSomething();
ScheduledExecutorService executor = Executors.newScheduledThreadPool(Runtime.getRuntime().availableProcessors());
// Schedule a task that will be executed in 120 sec
executor.schedule(task, 120, TimeUnit.SECONDS);

// Schedule a task that will be first run in 120 sec and each 120sec
// If an exception occurs then it's task executions are canceled.
executor.scheduleAtFixedRate(task, 120, 120, TimeUnit.SECONDS);

// Schedule a task that will be first run in 120 sec and each 120sec after the last execution
// If an exception occurs then it's task executions are canceled.
executor.scheduleWithFixedDelay(task, 120, 120, TimeUnit.SECONDS);

With spring you can rely on the Task and Scheduling API

public class MyBean {

    private final TaskScheduler executor;

    @Autowired
    public MyBean(TaskScheduler taskExecutor) {
        this.executor = taskExecutor;
    }

    public void scheduling(final Runnable task) {
        // Schedule a task to run once at the given date (here in 1minute)
        executor.schedule(task, Date.from(LocalDateTime.now().plusMinutes(1)
            .atZone(ZoneId.systemDefault()).toInstant()));

        // Schedule a task that will run as soon as possible and every 1000ms
        executor.scheduleAtFixedRate(task, 1000);

        // Schedule a task that will first run at the given date and every 1000ms
        executor.scheduleAtFixedRate(task, Date.from(LocalDateTime.now().plusMinutes(1)
            .atZone(ZoneId.systemDefault()).toInstant()), 1000);

        // Schedule a task that will run as soon as possible and every 1000ms after the previous completion
        executor.scheduleWithFixedDelay(task, 1000);

        // Schedule a task that will run as soon as possible and every 1000ms after the previous completion
        executor.scheduleWithFixedDelay(task, Date.from(LocalDateTime.now().plusMinutes(1)
            .atZone(ZoneId.systemDefault()).toInstant()), 1000);

        // Schedule a task with the given cron expression
        executor.schedule(task, new CronTrigger("*/5 * * * * MON-FRI"));
    }
}

And you can provide your own trigger by implementing Trigger

Don't forget to enable the scheduling by usin @EnableScheduling on configuration class.

About listening to directory content you can use WatchService. Something like:

final Path myDir = Paths.get("my/directory/i/want/to/monitor");
final WatchService watchService = FileSystems.getDefault().newWatchService();
// listen to create event in the directory
myDir.register(watchService, StandardWatchEventKinds.ENTRY_CREATE);
// Infinite loop don't forget to run this in a Thread
for(;;) {
   final WatchKey key = watchService.take();
   for (WatchEvent<?> event : key.pollEvents()) {
       WatchEvent<Path> watchEvent = (WatchEvent<Path>) event;
       Path newFilePath = myDir.resolve(watchEvent.context());
       //do something with the newFilePath
    }
    // To keep receiving event
    key.reset();
}

Take a look at this article: Watching a Directory for Changes for more details.

Mettah answered 2/11, 2017 at 16:59 Comment(6)
By what mechanism is public void scheduling called? By virtue of being a method in MyBean? Is it possible to pass parameters in addition to Task, such as a string CRON expression?Vieira
well here you want to schedule a new task dynamically so this mean that some where in your code you explicitly call the scheduling method. You can do what ever you want with the method signature. It's juste a sample of what can be done with spring and java scheduling API.Mettah
Thanks, @JEY, that makes sense. So if I use some frequent periodic task to call a method that envokes executor.schedule, I should be able to add tasks to the queue dynamically. Sound plausible?Vieira
Yes it would do it or you can directly do it in the periodic task.Mettah
Cool. I think I have a solution now. Thank you!Vieira
You can cancel the scheduled task like so: ` ScheduledFuture<?> scheduledFuture = testCustom(() -> System.out.println("HI!"), "*/5 * * * * ?"); Thread.sleep(25000); System.out.println("Now cancelling..."); scheduledFuture.cancel(false);`Edeline
P
1

Try this library with external dynamic parameters configuration, real time monitoring:

https://github.com/tyrion9/mtask

Config params in mtasks.yml

-   code: complex
    scheduled:
        period: 1000
    name: Autowired Param MTask
    className: sample.sample2.ComplexMTask
    params:
        name: HoaiPN
    autoStart: true

Dynamic params configurations on the fly:

curl -X GET http://localhost:8080/api

curl -X POST http://localhost:8080/api/helloworld/stop

curl -X POST http://localhost:8080/api/helloworld/start
Pigmy answered 16/5, 2019 at 4:34 Comment(0)
C
-2

You can do that over a spring annotation:

@Scheduled(fixedRate = 360000)
public void parseXmlFile() {
    // logic for parsing the XML file.
}

Please note that the method must be void. Furthermore, in your main class, you must enable scheduling:

@SpringBootApplication
@EnableScheduling
public class Application {

    public static void main(String[] args) throws Exception {
        SpringApplication.run(Application.class);
    }
}

Please see the full reference here: https://spring.io/guides/gs/scheduling-tasks/

Crozier answered 27/10, 2017 at 12:56 Comment(5)
This code will load a particular xml file at the start of the server.. but suppose later I am adding new xml files then it will not load those new jobs .. and one more thing the xml file is having cron expression . So how to launch a new job by taking that expressionBannock
You have to recognize the new XML files somehow. That could work over the name with a timestamp in it (e.g. dataFile_1486589279.xml). You can parse that in Java and get the file with the biggest timestamp. Regarding dynamically adding cron jobs: I don't know of any way to dynamically add cron jobs. I would doubt if that's the right approach. What if you have an error in your logic and your flooding your server with cron jobs? That will be quite hard to debug. Maybe you could tell more about your use case?Crozier
dave0688 .. I will handle the new xml files and parse data also.. that will not be a big deal so currently only concern will be how to launch those new schedule jobs.. suppose what you told about the error part is handled properlyBannock
Can you pass XML path as the program arguments, whenever it will schedule next job it will read XML from the path provided in the program argument, so when you need to change the XML, replace the XML in the path passed in the code.Reader
Amit K Bist - the files will be in resources folder .. I will maintain an in memory object (Map) which will keep track which xml file is there which isn't .. so whenever anybody adds a new xml file, from that in memory map object I will get to know whether a new scheduler needs to be added or not.. but how to launch that new schedule job that's my questionBannock

© 2022 - 2024 — McMap. All rights reserved.