Java Framework for managing Tasks

Question

my question is, whether there exists a framework in Java for managing and concurrently running Tasks that have logical dependencies.

My Task is as follows: I have a lot of independent tasks (Let's say A,B,C,D...), They are implemented as Commands (like in Command pattern). I would like to have a kind of executor which will accept all these tasks and execute them in a parallel manner. The tasks can be dependent one on another (For example, I can't run C, Before I run A), synchronous or asynchronous.

I would also like to incorporate the custom heuristics to affect the scheduler execution, for example if tasks A and B are CPU-intensive and C is, say, has high Memory consumption, It makes sense to run A and C in parallel, rather than running A and B.

Before diving into building this stuff by myself (i'm thinking about java.util.concurrent + annotation based constraints/rules), I was wondering, if someone could point me on some project that could suit my needs. Thanks a lot in advance

Fryd

Had a look at Fork/Join? – Parget

I wonder if tasks reuse each other results or not. If they are rather independent units of work with predefined sequence then scheduling and custom heuristic could be solved by quite simple custom thread pool executor. – Centrifugal

Yes, I'm also thinking about using the Executor Services from java.util.concurrent, the tasks seem to be independent in terms of reusing the results of each other. I should take a look at fork/join as Viktor Klang kindly proposed as well. – Fryd

Answer 1

I don't think that a there is a framework for managing tasks that could fulfill your requirements. You are on the right path using the Command pattern. You could take a look at the Akka framework for a simplified concurrency model. Akka is based on the Actor model:

The actor model is another very simple high level concurrency model: actors can’t respond to more than one message at a time (messages are queued into mailboxes) and can only communicate by sending messages, not sharing variables. As long as the messages are immutable data structures (which is always true in Erlang, but has to be a convention in languages without means of ensuring this property), everything is thread-safe, without need for any other mechanism. This is very similar to request cycle found in web development MVC frameworks. http://metaphysicaldeveloper.wordpress.com/2010/12/16/high-level-concurrency-with-jruby-and-akka-actors/

Akka is written in Scala but it exposes clean Java API.

Naseberry

Answer 2

I'd recommend you to examine possibility to use ant for this purpose. Although ant is known as a popular build tool it actually the XML controlled engine that runs various tasks. I think that its flag fork=true does exactly what you need: runs tasks concurrently. As any java application ant can be executed from other java application: just call its main method. In this case you can wrap your tasks using ant API, i.e. implement them as Ant tasks.

I have never try this approach but I believe it should work. I thought about it several years ago and suggested it to my management as a possible solution for problem similar to yours.

Unstable

Yes, thought about this option as well. It looks like it suits my needs, but in its own way :) Looks just like a wrong tool for solving this thing. The tasks are already written (its a part of a big production ready code). So introducing ant just to solve this sounds like an overkill. Thanks for advice though! – Fryd

Do not mention. I partially agree with you when you are saying that ant is a wrong tool. I think that using ant here is its abuse. But I believe that sometimes API abuse may be useful. I even wrote article about this: java.dzone.com/articles/useful-abuse – Unstable

Answer 3

Eclipse's job scheduling module is able to handle interdependent tasks. Take a look at http://www.eclipse.org/articles/Article-Concurrency/jobs-api.html.

Malik

Answer 4

There is a framework specifically for this purpose called dexecutor (Disclaimer : I am the owner)

Dexecutor is a very light weight framework to execute dependent/independent tasks in a reliable way, to do this it provides the minimal API.

• An API to add nodes in the graph (addDependency, addIndependent, addAsDependentOnAllLeafNodes, addAsDependencyToAllInitialNodes Later two are the hybrid version of the first two)
• and the other to execute the nodes in order.

Here is the simplest example :

DefaultDependentTasksExecutor<Integer, Integer> executor = newTaskExecutor();

executor.addDependency(1, 2);
executor.addDependency(1, 2);
executor.addDependency(1, 3);
executor.addDependency(3, 4);
executor.addDependency(3, 5);
executor.addDependency(3, 6);
//executor.addDependency(10, 2); // cycle
executor.addDependency(2, 7);
executor.addDependency(2, 9);
executor.addDependency(2, 8);
executor.addDependency(9, 10);
executor.addDependency(12, 13);
executor.addDependency(13, 4);
executor.addDependency(13, 14);
executor.addIndependent(11);

executor.execute(ExecutionBehavior.RETRY_ONCE_TERMINATING);

Here is how the dependency graph would be constructed

Tasks 1,12,11 would run in parallel, once on of these tasks finishes dependent tasks would run, for example, lets say task 1 finishes, tasks 2 and 3 would run similarly once task 12, finishes task 13 would run and so on.

Estas