Using emacs tramp vs. rsync for remote development

Question

I have been doing some remote development using emacs tramp and found that it was quite slow. Every time I save a file, it takes about 10 seconds to complete the save. So, now I am using rsync to transfer the files remotely and it works much faster, it takes about a second plus the local saves from emacs are instant. Are there any configuration options within tramp to get it to run as fast as rsync does on the command line? Are there any advantages to using tramp instead of rsync even though I am seeing such poor performance?

Theoretical

Answer 1

While tramp can be configured to use a large number of transport method, I suppose you're using ssh to connect to the remote server.

I believe most of the time it takes to complete the operation comes from setting up a connection and authenticating. If you're using a new enough version of OpenSSH, this can be helped by using the connection sharing feature, see ControlMaster in ssh_config(5).

I suggest you try

ControlMaster auto

and see if that improves the situation.

Mariellemariellen

Answer 2

If you're using tramp for ssh/scp functionality, you'll find opening a persistent SSH connection will make tramp operations a lot faster: they'll reuse the existing connection rather than creating a fresh one each time, cutting out a great deal of cryptographic overhead without affecting security. See this article on the SSH ControlMaster functionality.

Greathearted

It still takes around 10 seconds, but I am able to connect to other servers instantaneously. I wonder if it could be a server problem. – Theoretical

If it still takes 10 seconds, it could be the .bashrc/.bash_profile on the remote machine causing the delay. – Lamebrain

Answer 3

A couple of answers have mentioned enabling ControlMaster to keep a persistent ssh connection open (which avoids the expensive connection set-up / hand-shaking otherwise needed for each Tramp operation over ssh), but you don't need to configure anything outside of Emacs to use this -- if you look at the tramp-methods variable you will see there are existing methods which enable ControlMaster.

Specifically, the rsyncc and scpc methods.

Both sides of the connection need to support ControlMaster, of course (for instance Cygwin cannot do so, which is a shame for Windows users*), but I would suggest setting one of those as your tramp-default-method.

Not having a persistent connection makes extended Tramp usage pretty painful, but with one it's incredibly usable (to the point where, given a reasonably fast connection, you can almost forget that it's happening).

In answer to the final part of your question, yes, there are very good reasons to use Tramp instead of rsyncing the files manually. The primary one is that shell commands can be executed directly on the remote server, and you don't even have to think about it, as Emacs takes care of the details.

For instance, from a remote-file buffer, M-x shell RET opens a shell on the remote server, and commands like M-x rgrep RET and M-x find-grep-dired RET will run the find + grep on the remote server. I believe that this applies in general to functions which invoke shell commands.

For me, that's more than enough reason to use Tramp (unless installing Emacs on the remote server and using your local display was an option in which case, for long-term usage, I would consider doing that instead).

(*) When using Windows I used to host a Linux VM locally and run Emacs inside that (with Cygwin providing the X display) for the sole reason of using ControlMaster for Tramp ( https://mcmap.net/q/176456/-using-tramp-with-emacsw32-and-cygwin-possible ).

Falgoust

My emacs 25.3.2 doesn't have any scpc or rsyncc in tramp-methods, but I see the variable tramp-ssh-controlmaster-options is inserted for %c and the regular ssh method uses %c – Hebephrenia

Yes indeed; starting from Emacs 24.4 "The connection methods "plink1", "ssh1", "ssh2", "scp1", "scp2", "scpc" and "rsyncc" are discontinued. The ssh option "ControlMaster=auto" is set automatically in all ssh-based methods, when possible. See tramp-use-ssh-controlmaster-options." – Falgoust

Answer 4

Are you aware of the section in the tramp documentation on gnu.org that mentions using rsync? It says, in part:

5.3 External transfer methods

The external transfer methods operate through multiple channels, using the remote shell connection for many actions while delegating file transfers to an external transfer utility.

This saves the overhead of encoding and decoding that multiplexing the transfer through the one connection has with the inline methods.

...

rsync — ssh and rsync Using the ssh command to connect securely to the remote machine and the rsync command to transfer files is almost identical to the scp method.

While rsync performs much better than scp when transferring files that 
exist on both hosts, this advantage is lost if the file exists only on one side 
of the connection.

The rsync based method may be considerably faster than the rcp based
methods when writing to the remote system. Reading files to the local
machine is no faster than with a direct copy.

This method supports the ‘-p’ hack.

Cloddish

Answer 5

Since you said earlier that you have a problem on a particular server, check the sshd config file on the problem machine. It's probably something like /etc/ssh/sshd_config.

Look for a config option named "UseDNS". Set that to "No."

See if that doesn't speed things up for you. If so, then ssh is probably timing out waiting on a reverse dns lookup that you likely don't care about.

Approbation

Answer 6

Recently I've come to embrace sshfs so that the remote files are locally editable. Works well for Linux, Mac and Unix systems and puts the ssh parts in one command rather than every access. And exists entirely in User space.

Sproul

Answer 7

If you are using the ssh method, you can try the sftp method instead, which (I believe) opens a persistent sftp connection to the remote ssh server. Also, if you are using GNOME, you can add sftp to tramp-gvfs-methods to have TRAMP connect to sftp servers using GNOME's GVFS, which may or may not be faster/more convenient.

Standice