Using multiple cores/processors when compiling Java

Question

I use a desktop with eight cores to build a Java application using Ant (through a javac target). Is there a way to speed up the compilation by using more than one thread or process?

I know I can run several Ant tasks in parallel, but I don't think this can be applied to a single compilation target, or does it?

Lebanon

See related question here: #524075 – Cobnut

Answer 1

I don't know of any way to do tell ant itself to make effective use of multiple cores. But you can tell ant to use the Eclipse Compiler, which has support for multithreaded compilation built-in.

Fellowship

Here's a nice blog article about Running Eclipse Java Compiler with Ant. – Deshabille

Using Intellij 2017.1.3 with JDK 1.8 javac ends up being faster than eclipse compiler in both instances - parallel builds turned on and when turned off. With it on our project takes ~23 seconds for a rebuild with Javac and ~27 seconds for a rebuild with eclipse copmiler - strange as it's a 4 core i7 processor. – Pokeberry

The first link is broken. – Burnside

Answer 2

As long as the javac you are calling doesn't use all the cores it doesn't really matter what you say in Ant. You can use the compiler attribute to define which java compiler should be used for the task.

If you have several build targets you can use fork=yes to execute the target(s) externally.

http://ant.apache.org/manual/Tasks/javac.html#compilervalues

Gorgonzola

+1. I just want to add that currently javac is single-threaded ( bugs.sun.com/bugdatabase/view_bug.do?bug_id=6629150 ) and it is not a trivial task to make it multi-threaded ( blogs.sun.com/jjg/entry/towards_a_multi_threaded_javac ). – Radiant

Answer 3

The documentation seems to indicate that it's unlikely to work correctly with javac.

Anyone trying to run large Ant task sequences in parallel, such as javadoc and javac at the same time, is implicitly taking on the task of identifying and fixing all concurrency bugs the tasks that they run.

Accordingly, while this task has uses, it should be considered an advanced task which should be used in certain batch-processing or testing situations, rather than an easy trick to speed up build times on a multiway CPU.

Mittel

upvote, also from the doc: Third party tasks may or may not be thread safe, and some of Ant's core tasks, such as <javac> are definitely not re-entrant. – Materi

Answer 4

You can use Buck Build to increase your build speed and utilize multiple cores.

In a nutshell:

Buck is a build system developed and used by Facebook. It encourages the creation of small, reusable modules consisting of code and resources, and supports a variety of languages on many platforms.

Buck builds independent artifacts in parallel to take advantage of multiple cores on your machine. Further, it reduces incremental build times by keeping track of unchanged modules so that the minimal set of modules is rebuilt.

Ingathering

Answer 5

Not as far as I know. The Eclipse compiler has some work done to speed up using multiple cores but it does not buy as much as you probably would like it to.

Question is, can you live with incremental compilation for development, and only recompile those that changed? The full rebuild can then be left to the build server.

Glandule

Answer 6

I assume that it might not help much because javac can pull all files in memory and if it has to do this with multiple processes it's just doubling the effort. However if you want to compile two fairly separate pieces of Java code, then you can just do:

#!/usr/bin/env bash

javac file1.java &
javac file2.java &
javac file3.java &

wait;

if the 3 files have mostly different dependencies, then it might save time, if the dependencies overlap, then it probably doesn't save much time.

Eastlake