Are Futures executed on a single thread? (Scala)

Question

Using the implicit execution context in Scala, will each new future be computed on a single, dedicated thread or will the computation be divided up and distributed to multiple threads in the thread pool?

I don't know if this helps, the background to this question is that I want to perform multiple concurrent operations using the HtmlUnit API. To do this, I would wrap each new WebClient instance in a Future. The only problem is that the WebClient class is not thread-safe, so I'm worried that it might break up and send to different threads.

Mccord

Answer 1

One future is executed on a single thread. Several futures might be executed on several threads. So, no more than one future can occupy one thread simultaneously.

How does it work? When you create a Future it means that you've submitted a task to your thread pool - this one task can't be implicitly parallelized so it's executed on one thread only. One or several tasks submitted to the pool are being put into the pool's queue, so the executor takes tasks from that queue one-by-one and runs each on some randomly (or intentionally) chosen threads. So several Futures may get to several threads.

About shared objects - the only way to execute operations safely for an object shared between futures is using the Executors.newFixedThreadPool(1), which will use only one thread for the whole pool. Another solution - is to clone the object for every future. Using actors (make your shared object an actor's state) should be the best option.

If you use one object per future - everything should be fine.

Note: The future's handler, like Future{ ... }.map(handler) may be executed in a different thread than the future itself, but it actually creates another Future to obtain a result. Same for flatMap. More precisely, they use onComplete which creates a CallbackRunnable to launch the handler (possibly in a different thread) after the old future succeeds - this callback just completes the newly created future, so still "no more than one thread per future".

Charleen

Thanks! That was exactly the information I was looking for. Just a quick followup question--is there any advantage to submitting multiple futures to a newFixedThreadPool with only one thread? Can computations be concurrent on a single thread? – Mccord

No performance advantages at all. The only thing it could give you is asynchoronous API (submit multiple tasks + subscribe on their completion). You can also create several pools for different kind of futures, but the actors are much better for that. – Charleen

Create an ExecutionContext from Executors.newFixedThreadPool(1) with implicit val executionContext = ExecutionContext.fromExecutorService(Executors.newFixedThreadPool(1)). – Aeschines

Answer 2

A Future[+T] cannot guarantee that it will be completed on the same thread if its composed of multiple futures. That said, it doesn't mean that you'll get a concurrent modification exception or something along those lines. You can still get asynchronous code to execute sequentially, in which case it would be safe.

As for your second question, as long as you have one instance for each future you shouldn't have any concurrency issues.

Wileywilfong

if by composing you mean something like Future.sequence - they just create a new successful future with empty body (so it executed nowhere at all) and then adding handlers (whichadd handlers to future in composition). So the future itself is executed in single thread; however handlers (Future.map) might be executed in different ones, but every new handler returns a different future – Charleen

So, sequence is just producing several new futures, but no single future from them can be assigned to more than one thread. – Charleen

I'm referring to the use of flatMap or for comprehensions on futures. – Wileywilfong

that's same as map - creates a handler and returns new future for it – Charleen

just take a look here – Charleen

... and in case it wasn't clear from what Jonathan and dk14 wrote: unless something explicitly parallelizes some big operation (unlikely in Avraham's context), only one future will be operating on a single WebClient object at a time, so everything should be fine. One caution: if the WebClient is not threadsafe in a static way, that's a different matter entirely, and you'd better restrict the threadpool to 1. – Ethel

@Charleen If you have a method which returns a Future, there's no way to know for certain if it will take 10 threads to return the final result unless you look directly at the code of that method. That's all I meant, sorry for the confusion. – Wileywilfong

@Jonathan Boudreau I think you should add this explanation right to the answer because now it looks like you're talking about one single future, which is not correct. And "completed on the same thread" may be interpreted like "on the same thread, where it's created", which is not what you've meant – Charleen