How do I copy a file to a remote server in python using scp or ssh?

Question

I have a text file on my local machine that is generated by a python script run daily in cron. I would like to add a bit of code to have that file sent securely to my server over ssh. Help.

Answer 1

If you want the simple approach, this should work.

You'll want to ".close()" the file first so you know it's flushed to disk from Python.

import os
os.system("scp FILE USER@SERVER:PATH")
#e.g. os.system("scp foo.bar [email protected]:/path/to/foo.bar")

You need to generate (on the source machine) and install (on the destination machine) an ssh key beforehand so that the scp automatically gets authenticated with your public ssh key (in other words, so your script doesn't ask for a password).

ssh-keygen example

Answer 2

Kind of hacky, but the following should work :)

import os
filePath = "/foo/bar/baz.py"
serverPath = "/blah/boo/boom.py"
os.system("scp "+filePath+" [email protected]:"+serverPath)

Answer 3

You'd probably use the subprocess module. Something like this:

import subprocess
p = subprocess Popen(["scp", myfile, destination])
sts = os.waitpid(p.pid, 0)

Where destination is probably of the form user@remotehost:remotepath. Thanks to @Charles Duffy for pointing out the weakness in my original answer, which used a single string argument to specify the scp operation shell=True - that wouldn't handle whitespace in paths.

The module documentation has examples of error checking that you may want to perform in conjunction with this operation.

Ensure that you've set up proper credentials so that you can perform an unattended, passwordless scp between the machines. There is a stackoverflow question for this already.

Answer 4

There are a couple of different ways to approach the problem:

1. Wrap command-line programs
2. use a Python library that provides SSH capabilities (eg - Paramiko or Twisted Conch)

Each approach has its own quirks. You will need to setup SSH keys to enable password-less logins if you are wrapping system commands like "ssh", "scp" or "rsync." You can embed a password in a script using Paramiko or some other library, but you might find the lack of documentation frustrating, especially if you are not familiar with the basics of the SSH connection (eg - key exchanges, agents, etc). It probably goes without saying that SSH keys are almost always a better idea than passwords for this sort of stuff.

NOTE: its hard to beat rsync if you plan on transferring files via SSH, especially if the alternative is plain old scp.

I've used Paramiko with an eye towards replacing system calls but found myself drawn back to the wrapped commands due to their ease of use and immediate familiarity. You might be different. I gave Conch the once-over some time ago but it didn't appeal to me.

If opting for the system-call path, Python offers an array of options such as os.system or the commands/subprocess modules. I'd go with the subprocess module if using version 2.4+.

Answer 5

To do this in Python (i.e. not wrapping scp through subprocess.Popen or similar) with the Paramiko library, you would do something like this:

import os
import paramiko

ssh = paramiko.SSHClient()
ssh.load_host_keys(os.path.expanduser(os.path.join("~", ".ssh", "known_hosts")))
ssh.connect(server, username=username, password=password)
sftp = ssh.open_sftp()
remote_file = sftp.file(remotepath, "wb")
remote_file.set_pipelined(True)
remote_file.write(data)
sftp.close()
ssh.close()

(You would probably want to deal with unknown hosts, errors, creating any directories necessary, and so on).