using .netrc with sftp [closed]

Question

I've some existing scripts wherein am using ftp + .netrc.

I want to switch to sftp now but it seems it doesn't support macros / .netrc.

Is there any other alternative?

Please help.

Bike

This is quite an old post but for any future readers. FTP(s) and SFTP are completely different mechanisms. FTP is a full control protocol akin to HTTP and SMTP. SFTP is the Secure File Transfer Protocol a streaming protocol which is akin to kermit or gopher over SSL/SSH tunnels. – Averil

Answer 1

Simply put, you cannot use .netrc with sftp, scp or ssh. These products are part of the OpenSSH standard, which has the keyword 'secure' in the name. It is not a secure practice to automate logins the way .netrc does, and the standard prohibits this kind of automation (storing passwords). There is definitely an alternative, three actually.

Authorization

For either of the first two alternatives, you will want to setup keys and exchange them. On the machine you are connecting from run ssh-keygen, for your purposes it will be much simpler if you do not give the key a pass-phrase, though this is risky. You now have two files in .ssh/, an id_rsa and a id_rsa.pub. Of these the id_rsa must be kept secret or secured (hence the pass-phrase). The pub file is actually one line of text. This one line can be added to the ~/.ssh/authorized_keys file on the receiving host's side. You can add the key to the file manually; but there is also ssh-copy-id shortcut command which does just that, also taking care of file permissions. Having authorized a key, you should be able to connect from the machine with the private key to the machine which has the authorized public key, when you connect as the appropriate user. Test it with ssh -v. If you entered a pass-phrase, you will be prompted for it; if you did not you are now automation ready. You can use an ssh-agent to keep a private key active between sessions while only entering the pass-phrase once. If you are making multiple ssh hops, the option to forward agents will allow the private key from the original sourced box's ssh-agent to be communicated though each hop. Personally I find this overwrought, and hence suggest not using a pass-phrase.

Now that you can make ssh, sftp, and scp connections without entering any password or pass-phrase you're ready to automate the rest.

Alternative 1,

is the preferred alternative were you convert your .netrc macro to a shell script or other script calling a few scp commands. This is similar to automating all your ftp connections with curl or wget. E.G.:

scp -qr $USER@$REMOTE_HOST:$PATH_FILE_OR_DIR $LOCAL_PATH_FILE_OR_DIR #download
scp -qr $LOCAL_PATH_FILE_OR_DIR $USER@$REMOTE_HOST:$PATH_FILE_OR_DIR #upload
scp -pqr $USER@$REMOTE_HOST:$PATH_FILE_OR_DIR $USER@$REMOTE_HOST2:$PATH_FILE_OR_DIR #mirror between separate hosts.
ssh $USER@$REMOTE_HOST chmod 644 $PATH_FILE #set permissions

Alternative 2,

using sftp as you mentioned, you can script it with the expects command, with a batch file using the -b option, or by piping commands into sftp. This is a little more similar to an .netrc macro, but has no advantage over alternative 1. I'll show an example of the latter:

#!/bin/sh
echo "OK, starting now..."
sftp -b /dev/fd/0 remotehost <<EOF
cd pub
ascii
get filename.txt
bye
EOF

Alternative 3,

use an sftp program that breaks the SSH standard by allowing you to store connection parameters such as the password. For example using cyberduck and AppleScript, or FileZilla and a queue.

Further notes:

There is an ~/.ssh/config file you can use to give hostnames shorter names, set forwarding parameters, default directories, default usernames, and specific identities for each host. I also like the -l option of scp which limits my transfer rate to something more reasonable.

P.S. You'd think there's a tool out there for converting .netrc macros to (alternative 1 styled) shell scripts. But I found nothing. Is that a tiny niche business opportunity?

Jericajericho

if someone steals his .ssh private key with no passphrase, its just as bad as if they stole his .netrc file, they can still upload whatever they want to the server. the key is to only do automated uploads from a machine you know will never be stolen/compromised/etc. – Wyne

Please learn ssh-copy-id, you've wasted too much text on what can be done with a single command. – Incomprehensible

@Incomprehensible I'm certain people landing here would appreciate an answer with a link and more details. You can also edit this answer and insert the use of ssh-copy-id where necessary. It's certainly not a single command for everything, though sure, once you have a key you copy it over, and then you can write scripts that replace the rest of the netrc script. – Jericajericho

The annoying thing about ssh-copy-id is that whenever your host or client changes (e.g. you buy a new computer) your old ssh keys become useless and you have to set up new keys every few months. – Breadbasket

@Sridhar-Sarnobat also I had a distribution that didn't include ssh-copy-id. You should be able to keep backups of your identity files though for when you setup a new machine. You probably should. – Jericajericho

The "secure" part of the ssh name refers primarily to the protocol itself -- it guards against people intercepting traffic and sniffing passwords from it. The part about potential vulnerability of local files is completely separate -- ssh could read plain text passwords from .netrc without its network protocol becoming any less secure. – Heyman

@MikhailT. Sure. OpenSSH could also allow you to use private keys stored as publicly readable files without its network protocol becoming any less secure. But it does not. There's a few very easy to implement extra precautions in place to generally keep everyone safer. – Jericajericho

Yes, you got the gist of my comment perfectly. I was referring to the somewhat condescending sentence in your answer regarding "security" -- and how support for .netrc would be against the "s" in "ssh". There are plenty of situations, where the local machine and its local users are trusted, while the traffic with the remote is not... Storing a clear-text password in a file would've been perfectly fine in such a case, as long as it can not be sniffed off the network. – Heyman

Answer 2

If you can use passwordless authentication on your machine (which might be forbidden by your sysadmin, but usually isn't), then you can conveniently use scp in a shell script rather than macros in .netrc. But if you have to type a password to log into the remote machine, then I would use the "here script" (the bit with EOF in it) to do the magic. You can use a shell script to cook up the ftp script if it changes from time to time.

Naturalistic

Answer 3

You can use lftp calling a sftp:// URL. It obeys the .netrc file.

Duplet

it could benefit from a bit more specificity or an example to illustrate the process for users who might be less familiar with event – Firebrat

My ~/.netrc line looks like machine 123.123.123.123 login my_login password my_password On the command line I call lftp sftp://[email protected]:1234 With ":1234" as optional port if differing from the default port expected. That logs me in without prompting for a password. – Duplet