Java 8: Parallel FOR loop

Question

I have heard Java 8 provides a lot of utilities regarding concurrent computing. Therefore I am wondering what is the simplest way to parallelise the given for loop?

public static void main(String[] args)
{
    Set<Server> servers = getServers();
    Map<String, String> serverData = new ConcurrentHashMap<>();

    for (Server server : servers)
    {
        String serverId = server.getIdentifier(); 
        String data = server.fetchData();

        serverData.put(serverId, data);
    }
}

Mascagni

Answer 1

Read up on streams, they're all the new rage.

Pay especially close attention to the bit about parallelism:

"Processing elements with an explicit for-loop is inherently serial. Streams facilitate parallel execution by reframing the computation as a pipeline of aggregate operations, rather than as imperative operations on each individual element. All streams operations can execute either in serial or in parallel."

So to recap, there are no parallel for-loops, they're inherently serial. Streams however can do the job. Take a look at the following code:

    Set<Server> servers = getServers();
    Map<String, String> serverData = new ConcurrentHashMap<>();

    servers.parallelStream().forEach((server) -> {
        serverData.put(server.getIdentifier(), server.fetchData());
    });

Typical

Note that parallel streams have an overhead: it doesn't always improve performance over a serial stream (or normal for each). – Alwin

how does threads are handled using .parallelStream() ? Does it use any internal thread-pool ? – Reganregard

What if we have a million elements in servers? Will this create a million parallel streams? – Elmiraelmo

@SonuMishra, No. Parallel streams attempt to make use of multiple cores on the processer. – Talc

i think you could make it even more clear that the callbacks to the forEach method can/will be called in parallel – Misprint

Is there anything like MaxDegreeOfParallelism as in Parallel.ForEach in C#? It helps us limit the number of threads we want to run in parallel. – Wolfgang

@shashwat: does #30802963 help ? – Pothole

Answer 2

That would be using a Stream:

servers.parallelStream().forEach(server -> {
    serverData.put(server.getIdentifier(), server.fetchData());
});

I suspect a Collector can be used to greater effect here, since you use a concurrent collection.

Hort

Well, after your advise, I now can read lambda expressions as well as know some details of streams as well ;) – Shallop

Answer 3

More elegant or functional solution will be just using Collectors toMap or toConcurrentMap function, which avoid maintaining another stateful variable for ConcurrentHashMap, as following example:

final Set<Server> servers = getServers();
Map<String, String> serverData = servers.parallelStream().collect(
    toConcurrentMap(Server::getIdentifier, Server::fetchData));

Note: 1. Those functional interfaces (Server::getIdentifier or Server::fetchData) doesn't allow throw checked exception here, 2. To get the full benefits of parallel stream, the number of servers would be large and there is no I/O involved, purely data processing in those functions(getIdentifier, fetchData)

Please refer to Collectors javadoc at http://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html#toConcurrentMap

Eversion

Answer 4

Simple example to copy'n'paste (the examples above use the class Server which is a custom class written by the OP):

import java.io.Console;
import java.util.ArrayList;

ArrayList<String> list = new ArrayList<>();
list.add("Item1");
list.add("Item2");
list.parallelStream().forEach((o) -> {
    System.out.print(o);
});

Console output. The order could possibly vary as everything executes in parallel:

Item1
Item2

The .parallelStream() method was introduced in Java v8. This example was tested with JDK v1.8.0_181.

Fixture