What's the best way to build a queue for long-running jobs in a Grails app?

Question

I have a Grails app that has some computationally intensive optimizations with a running time of ~5 minutes (perhaps more). Currently, I'm doing these in the main request thread, i.e. it takes 5 minutes for the request to return. It works, but is of course horrible from a usability perspective.

So what's the best way to implement this in an asynchronous way? I assume a ThreadPoolExecutor would have to be involved, but how do I start and access it? Can I model it as a Grails Service? Or a Job (seems that those are only meant for recurring jobs though)?

Also, what's the best way to handle job status? Via a a flag or perhaps an entire new class in the DB? Have the browser show a spinner and keep polling until the status changes?

Moth

Answer 1

There is a grails plugin background-thread which might be just what you are looking for.

Of course it would be possible to roll your own thread pooling or use existing Java stuff.

Mauromaurois

Answer 2

I'd use the grails JMS Plugin for this.

Then you can create a service with an "onMessage" method that interacts automatically with an underlying jms provider (like OpenMQ or ActiveMQ.

It makes this kind of thing pretty easy.

Tremain

What would be the benefit compared to simply using a ThreadPoolExecutor? It sounds a lot more complex than that. – Moth

It is definitely more complex. The main benefits would be around some of the monitoring and polling capabilities that your question alluded to some requirements around. Lots of this kind of stuff is solved in the JMS world, but there is more setup/maintenance around it. It really depends on how "enterprisey" your requirements are. If you don't care much about monitoring or what happens if your container goes down while it's processing, then I'd do something simpler. Just one potential option. – Tremain

Ah, I see. Well, the only monitoring I need is for a user waiting for a specific job to complete. I don't need transactional safety either, so I'll go for a simpler approach. – Moth

Answer 3

Background-thread plugin is outdated so I wouldn't recommend using it and also the JMS seems overkill for background processing. JMS is more of a message queue rather than background processing utility.

I would recommend either using Quartz plugin or using the gpars.

Wiggins

Answer 4

In the spirit of the "simplest thing that could possibly work" I have done something like this as a simple service. (Of course it might be too simple, criticism is welcomed).

I used Groovy's feature that Thread has a static start method that takes a closure. See http://groovy.codehaus.org/groovy-jdk/java/lang/Thread.html

I implemented a method on a service that looked like:

synchronized def runThreadedTask() {
  if(taskRunning) { 
    // taskRunning is defined as a service level flag to monitor job status
    // if we are already running the task just return
    return;
  }

  Thread.start {
    taskRunning = true
    // do job processing here
    taskRunning = false
  }
}

Distinguishing

Answer 5

grails install-plugin background-thread

def backgroundService

backgroundService.execute("doing my thing", {

// do work here

});

Detention

Answer 6

It's been a while since this was asked, but since it just came up in a search I figured I'd add that there is now a Groovy threading closure in the Thread object. You just use:

Thread.start {
    // async code goes here
}

and call it a day. Your async code can call an update data method or something when it's done--you might want to synchronize that if you can have multiple threads running.

Chasechaser