using .netrc with sftp

Question

I've some existing scripts wherein am using ftp + .netrc.

I want to switch to sftp now but it seems it doesn't support macros / .netrc.

Is there any other alternative?

Please help.

Answer 1

Try looking into scp, as you can use passwordless login (ssh keys) to do this.

You may be able to use the same techniques with SFTP, I'm not sure.

Answer 2

Simply put, you cannot use .netrc with sftp, scp or ssh. These products are part of the OpenSSH standard, which has the keyword 'secure' in the name. It is not a secure practice to automate logins the way .netrc does, and the standard prohibits this kind of automation (storing passwords). There is definitely an alternative, three actually.

Authorization

For either of the first two alternatives, you will want to setup keys and exchange them. On the machine you are connecting from run ssh-keygen, for your purposes it will be much simpler if you do not give the key a pass-phrase, though this is risky. You now have two files in .ssh/, an id_rsa and a id_rsa.pub. Of these the id_rsa must be kept secret or secured (hence the pass-phrase). The pub file is actually one line of text. This one line can be added to a file on the receiving host's side. For the user you are connecting as, look in, or create, $HOME/.ssh/authorized_keys. Add the pub file's line to this file. This file's permissions must be octal 600, and possibly the .ssh directory must be 700. Now that you have done this, you should be able to connect from the machine with the private key to the machine which has the authorized public key, when you connect as the appropriate user. Test it with ssh. If you entered a pass-phrase, you will be prompted for it; if you did not you are now automation ready. You can use an ssh-agent to keep a private key active between sessions while only entering the pass-phrase once. If you are making multiple ssh hops, the option to forward agents will allow the private key from the original sourced box's ssh-agent to be communicated though each hop. Personally I find this overwrought, and hence suggest not using a pass-phrase.

Now that you can make ssh, sftp, and scp connections without entering any password or pass-phrase you're ready to automate the rest.

Alternative 1,

is the preferred alternative were you convert your .netrc macro to a shell script or other script calling a few scp commands. This is similar to automating all your ftp connections with curl or wget. E.G.:

scp -qr $USER@$REMOTE_HOST:$PATH_FILE_OR_DIR $LOCAL_PATH_FILE_OR_DIR #download
scp -qr $LOCAL_PATH_FILE_OR_DIR $USER@$REMOTE_HOST:$PATH_FILE_OR_DIR #upload
scp -pqr $USER@$REMOTE_HOST:$PATH_FILE_OR_DIR $USER@$REMOTE_HOST2:$PATH_FILE_OR_DIR #mirror between separate hosts.
ssh $USER@$REMOTE_HOST chmod 644 $PATH_FILE #set permissions

Alternative 2,

using sftp as you mentioned, you can script it with the expects command, with a batch file using the -b option, or by piping commands into sftp. This is a little more similar to an .netrc macro, but has no advantage over alternative 1. I'll show an example of the latter:

#!/bin/sh
echo "OK, starting now..."
sftp -b /dev/fd/0 remotehost <<EOF
cd pub
ascii
get filename.txt
bye
EOF

Alternative 3,

use an sftp program that breaks the SSH standard by allowing you to store connection parameters such as the password. For example using cyberduck and AppleScript, or FileZilla and a queue.

Further notes:

There is an ~/.ssh/config file you can use to give hostnames shorter names, set forwarding parameters, default directories, default usernames, and specific identities for each host. I also like the -l option of scp which limits my transfer rate to something more reasonable.

P.S. You'd think there's a tool out there for converting .netrc macros to (alternative 1 styled) shell scripts. But I found nothing. Is that a tiny niche business opportunity?