Heroku fails to bundle python-javabridge (javahome not found)

Question

I am trying to deploy my first Plotly Dash app (written in Python) to Heroku. I am having trouble with bundling up all the packages to compile the app, currently with Python-javabridge.

When I try to deploy my app, even after deleting python-javabridge from the requirements.txt, it still tries to bundle it, giving me the following error:

remote:        Collecting jeepney>=0.4.2; sys_platform == "linux"
remote:          Downloading jeepney-0.6.0-py3-none-any.whl (45 kB)
remote:        Collecting importlib-resources; python_version < "3.7"
remote:          Downloading importlib_resources-4.1.1-py3-none-any.whl (22 kB)
remote:        Collecting python-javabridge==4.0.0
remote:          Downloading python-javabridge-4.0.0.tar.gz (1.3 MB)
remote:            ERROR: Command errored out with exit status 1:
remote:             command: /app/.heroku/python/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-qllzoihk/python-javabridge/setup.py'"'"'; __file__='"'"'/tmp/pip-install-qllzoihk/python-javabridge/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-xeh4coov
remote:                 cwd: /tmp/pip-install-qllzoihk/python-javabridge/
remote:            Complete output (11 lines):
remote:            Traceback (most recent call last):
remote:              File "<string>", line 1, in <module>
remote:              File "/tmp/pip-install-qllzoihk/python-javabridge/setup.py", line 412, in <module>
remote:                ext_modules=ext_modules(),
remote:              File "/tmp/pip-install-qllzoihk/python-javabridge/setup.py", line 96, in ext_modules
remote:                java_home = find_javahome()
remote:              File "/tmp/pip-install-qllzoihk/python-javabridge/javabridge/locate.py", line 133,
in find_javahome
remote:                java_bin = get_out(["bash", "-c", "type -p java"])
remote:              File "/tmp/pip-install-qllzoihk/python-javabridge/javabridge/locate.py", line 130,
in get_out
remote:                raise Exception("Error finding javahome on linux: %s" % cmd)
remote:            Exception: Error finding javahome on linux: ['bash', '-c', 'type -p java']
remote:            ----------------------------------------
remote:        ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
remote:  !     Push rejected, failed to compile Python app.
remote:
remote:  !     Push failed

What I have done:

Just running pip install javabridge works perfectly, no problems.

I added JAVA_HOME to my system variables and to path, didn't help. Writing javac or java --version on the cmd outputs the version as expected.

I use Windows 10, Python 3.7, Anaconda3, Spyder terminal to run the commands.

Pine

You should explain more detailed what you are doing there and especially how Heroku is involved. – Xeres

Thanks for the feedback, I added some more info, apologies for the lack of clarity :) – Pine

Heroku is running on Linux. Those logs are from Heroku's servers (notice the "remote:"). You push your code to Heroku, and it will try to install your app's dependencies on its servers. – Jagannath

Ah thank, this is too new to me. I will change the title so it is not misleading. I'll look for the problem in java_home then. I'm confused because pip install javabridge works fine, the java_home seems to be correctly set according to everything I read about it online. :( – Pine

Heruko runs on Linux and will build your application in Linux. And for that it can't use your locally installed software like Java. To debug why the removed dependency is still installed check git status and post the requirements. – Xeres

If pip install javabridge works fine locally, on your machine, it does not mean it will also work on Heroku. So it's not just your environment's JAVA_HOME you need to fix, but also how Heroku can install that same package on its own environment (which is totally separate from your Windows machine). – Jagannath

Thank you so much, that's super insightful. If it happens with a critical package, how would I troubleshoot a Linux problem from my Windows machine? – Pine

@KlausD. git status just shows there are no changes, but running git show master:requirements.txt does show the edited dependencies (with the deleted packages, including python-javabridge) .. so no idea what is making it install it anyway.. – Pine

@Isquare1 you are deploying aap with git or you are using docker – Teleology

it is with git, this seems to be the way heroku usually does it :) – Pine

Answer 1

If you want to use Java from a Python application on Heroku you'll need to use multiple buildpacks. Note that

The buildpack for the primary language of your app should always be the last buildpack in the list. This ensures that defaults for that primary language are applied instead of those for another language, and allows Heroku to correctly detect the primary language of your app.

So in this case, do something like

PS C:\> heroku buildpacks:set heroku/python
PS C:\> heroku buildpacks:add --index 1 heroku/jvm

If you run heroku buildpacks you should now see the JVM buildpack listed first and the Python buildpack listed second. Note that we are using heroku/jvm here, not heroku/java, since you aren't building a Java application.

After doing this you'll need to redeploy your application. If you have changes to make, go ahead and make them, then commit, and push.

Note that your application might need to use the Java bridge even if you aren't directly depending on it in your requirements.txt, e.g. if one of your dependencies depends on it.

Accommodative

Thank you loads for taking the time to help me out with this, I have some more reading to do, the links you provided were very useful! I set up the buildpacks and although now java_home is located, now I get a ModuleNotFoundError: 'No module named 'numpy' from the python-javabridge setup.py file...it's the first time I've had this error – Pine

PS Looks like it has to do with heroku/git running the setup.py file before downloading numpy so solving it seems to involve delving into some setuptools that I'm not familiar with. But I select your answer as the correct one, as it solved the java_home problem that I was stuck on for days, thank you for that :) – Pine

Weird. Is javabridge in your requirements.txt? I know you said you took it out, but did you put it back in? If not, how are you installing it? What about numpy? – Accommodative

I took it out for good, but it still tries to install it anyway, together with a few other packages.. You mentioned it could be that they are requirements for some of the dependencies? Numpy is installed and is in the requirements.txt.I have pip installed javabridge and numpy on my local PC and it is all fine. It seems it is a common problem with deploying apps but I am not that proficient to be changing the setuptools of javabridge myself.. forum.predix.io/questions/30529/… – Pine

Are you entirely sure that you have committed your updated requirements.txt file, and included that commit when you push to Heroku? Do you also have a Pipfile and Pipfile.lock? – Accommodative

Yes, I even checked the requirements.txt as committed to git (with git:show) and it is indeed the updated requirments.txt. I suspect the java wrapper is just a requirement for all Dash apps, because entering pip check in the given env does not show any of the packages to have javabridge as a dependency. I don't have a Pipfile, I will look into it, thank you for the tip! – Pine

@Isquare1, to clarify, you probably should not have a Pipfile. That's an alternative to requirements.txt, and it takes precedence on Heroku. I was just trying to see if Heroku might be using a different set of dependencies. Use either (a) requirements.txt or (b) Pipfile and Pipfile.lock, not both. – Accommodative