Bazel build fails with msg "env: python: No such file or directory" (macOS Monterey)

Question

I am new to python world and I have followed a few articles to set python on my system. I need python to build my project through bazel.

When I build my project on local, I get the following error. Please note, I am able to build the project successfully on server so issue is not related to code.

env: python: No such file or directory
INFO: Elapsed time: 1.708s, Critical Path: 1.09s
INFO: 7 processes: 4 darwin-sandbox, 3 worker.
FAILED: Build did NOT complete successfully

Please HELP ME to solve this issue with my build failure.

I tried to solve this by applying the solutions online but all in vain. I have removed my complete python env and have re-configured it as well but that did not work for me either.

This is my system configuration:

Python version in env is: Python 3.9.12 but it still says "env: python: No such file or directory" in the above error.

└─(13:59:50)──> /usr/bin/env python --version    
Python 3.9.12

If I fire /usr/bin/python3 --version I get Python 3.8.9 as output

└─(13:47:35)──> /usr/bin/python3 --version                   
Python 3.8.9

But when cd into the /usr/bin/ and I fire python3 --version command I get Python 3.9.12 as output.

┌─(/usr/bin)──────────────────────────────────────────
└─(13:56:29)──> pwd                                                      
/usr/bin
┌─(/usr/bin)──────────────────────────────────────────
└─(13:56:32)──> python3 --version                                        
Python 3.9.12

Also, both /usr/local/bin/python and /usr/local/bin/python3 point to Python 3.9.12.

┌─(/usr/bin)──────────────────────────────────────────
└─(13:56:41)──> /usr/local/bin/python --version       
Python 3.9.12
┌─(/usr/bin)──────────────────────────────────────────
└─(13:57:46)──> /usr/local/bin/python3 --version      
Python 3.9.12

Choosy

Answer 1

I am having the same issue. Here is what I found. Bazel builds often use py_binary tools, which are launched with a shell script that has the shebang #!/usr/bin/env python (in versions prior to 5). But it also launches them inside a subshell that clears the PATH environment variable. (exec env - ...).

This means that there must be a python on the system path. Modifying your PATH won't help nor will using something like pyenv. To see which python is being used, run (exec env - python). In macOS 12.3, Apple removed /usr/bin/python. so this fails and prints env: python: No such file or directory.

I haven't found any way to fix this, since Apple also does not let you modify /usr/bin to add a symlink to python3 even as root (due to System Integrity Protection).

This is fixed in Bazel 5, but I am not able to upgrade to that since I need to be able to make updates on an older project that has libraries that require Bazel < 4.

Note that Github macOS runners are still on macOS 11. That may be why your project still builds on your server.

I am blocked at this point. I think I either need to find a way to add python to the system PATH (working around SIP), or I need to back-port some changes from Bazel 5 to Bazel 3 and use a custom version of Bazel. Or I need to do all development and debugging on an older version of macOS.

Tipper

I was able to work around this by getting my project to work with Bazel 4 and using the py_runtime approach described here: #67512566 – Tipper

Answer 2

You are right about your analysis @Matthew Self. The symlink cannot be create due to MacOS's SIP. Even after disabling it you cannot create a symlink in /usr/bin/... since it is read-only.

More info on this here - Cannot create a symlink inside of /usr/bin even as sudo

---

Now I am also having a similar issue while building a bazel target that is by default using /usr/bin/python for running any python files. Since my bazel verison is 3.7.1, so it is giving the similar error as yours after updating the OS Monterery.

But using ln -s /usr/bin/python3 /usr/local/bin/python solved my issue as it created a symlink in /usr/local/bin/... instead of /usr/bin/... and surprisingly bazel picked up this PATH of python by deafault.

May be because /usr/local/bin will be listed before /usr/bin, hence it was taking precedence.

Overplus

Answer 3

I just solved this problem. The approach I used is to hack bazel 4.1.0.

First, check out the bazel 4.1.0 source

By modifying the file: src/main/java/com/google/devtools/build/lib/bazel/rules/python/python_stub_template.txt

change the first line

#!/usr/bin/env python

into

#!/usr/bin/env /Library/Frameworks/Python.framework/Versions/2.7/bin/python

You can specify the python2 path here.

Last, you build the modified bazel in command line:

$ bazel build //src:bazel

You can get a modified version of bazel. By using this bazel instead of original one, the problem is solved.

Timbrel

Answer 4

This is probably not the exact same issue as you were having, but I figured I'd share it since I was also having trouble just getting an initial py_binary to run.

I was running a centos docker container and was having trouble even after having already installed python3 in my container's environment. It turned out, I needed to have the linux package which installed and included in my path, which then gets used internally by rules_python to locate the exact path of python3 or python

Pandybat