pycharm always "uploading pycharm helpers" to same remote python interpreter when starts

Question

When I start PyCharm for remote python interpreter, it always performs "Uploading PyCharm helpers", even when the remote machine IP is the same and already containing previously uploaded helpers. Is the behaviour correct?

Shirr

Answer 1

This is a well known problem that can be a major obstacle in productivity especially if you use disposable instances in your workflow. It leads to a forced coffee break of 20 minutes every time you want to connect to a remote system. Unacceptable.

Seems like PyCharm creates a build.txt file in the remote helper folder that just has the current PyCharm build number as its contents, for instance:

PY-171.4694.38

So it's possible to upload the helpers manually by using rsync on /Applications/PyCharm.app/Contents/helpers/ and finally manually creating a build.txt file with your current build number. After that, PyCharm should not attempt to re-upload them.

Example:

 $ rsync -avz /Applications/PyCharm.app/Contents/helpers/ cluster:/home/xapple/.pycharm_helpers/
 $ echo "PY-171.4694.38" > /home/xapple/.pycharm_helpers/build.txt
 $ python /home/xapple/.pycharm_helpers/pydev/setup_cython.py build_ext --inplace

Gwen

Answer 2

This problem came back again 6 years later with PyCharm 2022.3.2.

The directory /Applications/PyCharm.app/Contents/helpers/ doesn't exist anymore, so the previous trick doesn't work.

What solved it this time is simply to:

1. Quit PyCharm.
2. Delete the ~/.pycharm_helpers directory on the remote server.
3. Relaunch PyCharm and let it do it's thing.

Gwen

Answer 3

In my case, several projects are projected to the remote server by Pycharm. All of them get stuck when one of the projects goes wrong on the remote server. Solution: leave only one that you need to work on and restart the PyCharm by "Invalidate caches".

Sedum

The year is 2022 and 'Invalidate caches' has solved my problem – Rollmop

Answer 4

According to the docs,

PyCharm checks remote helpers version on every remote run, so if you update your PyCharm version, the new helpers will be uploaded automatically and you don't need to recreate remote interpreter.

Prudie

Answer 5

Note that -- at least as late as version 2018.3.x -- PyCharm also appears to require re-uploading of the helpers when the local network connection changes as well, for some reason.

What I've observed in my case is that if, while PyCharm remains running, I relocate my laptop and connect to a different LAN, the next remote debugging session I initiate will trigger the lengthy helper upload. It turns out that the contents of the helpers directory actually uploaded in this case are exactly identical to the contents already present in that directory on the remote system (I compared them), so this upload is entirely superfluous, but PyCharm isn't able to detect this.

As there's no way I know of in PyCharm to bypass or cancel automatic helpers upload, the only recourse is to completely exit from PyCharm (close all open project windows) after each change of network connection and restart the IDE. In my experience, this will cause the helper upload to succeed in the "checking remote helpers" phase, before actually uploading all the helpers again. Of course, this is major annoyance if you have multiple projects open, but it's faster than waiting the (tens of) minutes for the agonizingly slow helpers upload to complete.

All of what other responders describe for the course of action to take when changing PyCharm versions is true. It is sufficient to use rsync, ftp, scp, or whatever to transfer the contents of the new local helpers directory (on Linux, a subdirectory of where the app is installed) to the remote system (on Linux, ~/.pycharm_helpers, where ~ is the home directory of the user name used for the remote debugging session), and update the remote build.txt in the helpers directory with the new PyCharm version.

Fablan

Answer 6

For me, this worked, removing the Pycharm helpers directory from /root folder.

rm -rf /root/.pycharm_helpers

It forces Pycharm to reupload everything from scratch, but at least it doesn't stick at the uploading helpers process.

Just remember, since this is the generic dir for pycharm projects, if you delete this, if you have other projects on that server, you might need to wait for helpers to be uploaded for them too. Didn't mess with my projects, But you need to consider it.

Pneumatophore

Answer 7

It's year 2023 and unfortunately none of the answers mentioned before work anymore (at least for me). What I did is:

1. Removed /root/.pycharm_helpers (already mentioned)
2. Updated tar
3. Manually extracted helpers.tar.gz

I don't know whether OS had some troubles with old version of tar, or it has to be done manually, but it works.

Bryon

Answer 8

fast (less than 3 second between me an digitalocean) solution inspired by excellent xApple's answer
on remote server:

export SOURCE=<your ip>
export PORT=9000
export HELPERS=$HOME/.pycharm_helpers
# PyCharm Help -> About
export BUILD=PY-172.4343.24  # 2017/10/11
cd  $HELPERS
rm -fr *
# my OS - ubuntu, change firewall rules to yours if you're not so lucky
sudo ufw allow from $SOURCE proto tcp to any port $PORT
netcat -l -v -p $PORT | tar xz # here you waiting for connection
# after finish
sudo ufw delete allow from $SOURCE proto tcp to any port $PORT
echo -n $BUILD > build.txt
python $HELPERS/pydev/setup_cython.py build_ext --inplace

on your workstation:

export TARGET=<remote server ip>
export PORT=9000
export HELPERS=<path to helpers> # for me it's $HOME/opt/pycharm-2016.3/helpers
cd $HELPERS
tar cfz - . | netcat -v $TARGET $PORT

Olgaolguin

Answer 9

Turning off the firewall addressed the problem in my case (macOS - Mojave). Note that this is not a general solution as it was not tested in any other environments/OS.

Milligan

This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review – Adduct

edited to accommodate the above comment. This is a solution to at least some users (which includes myself) and doesn't require any further clarifications. – Milligan