Python’s ftplib STOR reliable?

Question

I'm using this code to upload myfile.txt from my windows machine to a ftp server. after the upoad the script deletes the file on my local machine (I'm not deleting it on the ftp).

try:
    ftp = FTP(ftp.host.com)
    ftp.login(your_username, your_password)
    file = open(myfile.txt, "rb")
    ftp.storbinary('STOR myfile.txt', file)
    print 'STORing File now...'
    ftp.quit()
    file.close()
    subprocess.Popen('del myfile.txt', shell=True)
    print 'File deleted'
except all_errors:
    print 'An error occured'

This code runs, however it's not reliable! At every ~10th upload my script hangs while STORing the file.

print 'STORing File now...' # So I just get 'STORING File now...'

The file is not big and should be uploaded within a few seconds, but I often have to wait an hour or two and only then the exception is thrown:

print 'An error occured'

If the exception were thrown 'earlier' it would be nice so I could just restart the upload (e.g. in a while loop). Because I need this file to be uploaded as soon as possible I need to make the file upload faster (I don't want to wait so long for the exception being thrown)

Second issue: Sometimes this happens: After the file has been successfully uploaded, the script fails to delete the file on my local machine because 'some other process is accessing it already' <- I think ftplib did not 'released' the file. What can I do to prevent this?

I'm searching for a better/reliable simple fileupload solution. Anyone have an idea? Thanks!

Answer 1

To get the exception earlier, use socket.setdefaulttimeout: e.g.,

import socket
socket.setdefaulttimeout(20.0)

will give you an exception if a socket it blocked for 20 seconds.

To remove a file from your Pyton script, use os.unlink -- much better than shelling out to a separate process for a del.

Answer 2

Don't use subprocess to shell out to delete a file - the os.unlink call will allow you to do this portably (the shutil library fills in the gaps when os fails)

Right now, you are gobbling the error with your silly print statement - get a traceback from the exception which would give you a large number of clues. However, your problem is likely related to socket timeout issues - either you are not running passive FTP, or the server is misconfigured and given you an invalid passive connect port number (something its firewall blocks).

Answer 3

how about this

def upload(ftp, filename):
    ftp.storbinary("STOR " + filename, open(filename, "rb"), 1024)
try:
    ftp = FTP("ftp.host.com")
    ftp.login(your_username, your_password)
except Exception,e:
    print e
else:
    file = open("myfile.txt", "rb")
    print 'STORing File now...'
    try:
        upload(ftp,file)
    except Exception,e:
        print e
    else:
        ftp.quit()
        file.close()
        try:
            os.remove("myfile.txt")
        except Exception,e:
            print e
        else:
            print 'File deleted'

remember, Python has its own file removing module. don't call system del unnecessarily