Jenkins does not work properly on Mac M1 agent

Question

We want to use a Mac Mini with the M1 processor as an agent in our CI pipeline to make sure that developers with new Macs can compile the project. We duplicated the setup that we have for x86, but cmake cannot find libraries such as boost:

CMake Error at /opt/homebrew/Cellar/cmake/3.19.7/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:218 (message):
  Could NOT find Boost (missing: Boost_INCLUDE_DIR container system
  date_time)

Weirdly, this is not an issue when connecting to the machine directly via ssh. Even manually setting the environment variables does not change anything.

Octonary

Answer 1

The Jenkins agent uses Java. If you install Java via brew (e.g., adoptopenjdk8), you might get an x64 version (as of March 2021) that is run using the Rosetta translation layer. You can verify this by executing sysctl sysctl.proc_translated. In the "normal" SSH connection, this should return 0, when executed as sh "sysctl sysctl.proc_translated" in Jenkins, you will see a 1.

Preferred Option (install native JDK)

Instead of using, e.g., adoptopenjdk8, you can install the zulu JDK, which is available natively, via

brew install --cask zulu

After this, you should be able to verify that proc_translated is 0 within Jenkins. You can verify that a native binary is built as described below.

Hotfix (without changing the JDK)

The following is a hotfix if you cannot install a native JDK.

We were able to force cmake to run natively (and thus to find the native libraries) by modifying our Jenkinsfile as follows:

old:

sh "cmake .."
sh "make hyriseTest"

new:

sh "PATH=/opt/homebrew/bin:$PATH arch -arm64 cmake .."
sh "arch -arm64 make hyriseTest"

Having to repeat the arch command over and over is not really pretty. Maybe someone has a better solution.

Verification of the result

We can verify that our generated binary is a native ARM binary:

sh "file hyriseTest"

Result:

+ file ./hyriseTest
./hyriseTest: Mach-O 64-bit executable arm64

Compiler warnings

Note: We are still seeing linker warnings about "ltmp3" and "ltmp4". I have not found any information about that online and it does not seem to affect the correctness of the result. Example:

ld: warning: direct access in function 'ltmp4' from file 'CMakeFiles/hyriseMvccDeletePlugin.dir/Unity/unity_0_cxx.cxx.o' to global weak symbol 

Octonary