OS specific build performance in Java
Asked Answered
U

1

6

We are currently evaluating our next-generation company-wide developer pc-configuration and have noticed something really weird.

Our rather large monolith has - on our current configuration a build time of approx. 4.5 minutes (no test, just compile).

For our next generation configuration we upgraded several components. A moderate increase in frequency and IPC with the processor, doubling the number of CPU cores and a switch from a small SATA SSD towards a NVMe SSD rated at >3GBps. Also, the next generation configuration switches from Windows 7 to Windows 10.

When executing the first tests, we noticed an almost identical build time (4.3 Minutes), which was a lot less improvement than we expected.

During our experiments we tried at one point to run the build process from within a virtual Linux machine running on the windows host. On the old configuration (Windows7) we saw a drop in build times from 4.5 to ~3.7 Minutes, on the Windows 10 Host, we saw a decrease from 4.3 to 2.3 minutes. We have ruled out things like virus scan.

We were rather astonished with these results and have tried to find another explanation than some almost-religious and insulting statements about different operation systems.

So the question is: What could we have possibly done wrong in configuring the Windows machine such that the speed is almost half of a Linux running virtualized in the very same windows host? Especially as all the hardware advancements seem to be eaten up by the switch from windows 7 to 10.

Another question is: How can we ace the javac process use up more cores, because right now, using Hotspot JDK 8 we can see at most two cores really used by the build. I've read about sjavac but that seems a rather experimental feature only available to OpenJDK9 onward, right?

Unwarranted answered 17/2, 2018 at 12:44 Comment(0)
U
7

After almost a year in experimenting we came to the conclusion, that it is indeed NTFS which is the evil-doer. If you have a ntfs user-partition with a linux host, you get somewhat similar results compared to an all-windows-setup.

We did benchmarks of gradle-build, eclipse internal build, starting up wildfly and running database-centered tests on multiple devices. All our benchmarks showed consistently a speedup of at least 100% when switching from Windows to Linux (sometimes, Windows takes 3x the amount of time in real world benchmarks than Linux, some artificial benchmarks had a speedup of 60!). Especially on notebooks we experienced much less noise, as the combined processor load of a complete build is substantial less than with windows.

Our conclusion was, to switch from Windows to Linux over the course of the last year.

Regarding the parallelisation thing, we realized, it was some form of code-entanglement. Resolving this helped gradle and javac to parallelise the build a lot (also have a look into gradle-composite-builds)

Unwarranted answered 13/2, 2019 at 13:32 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.