Multithreaded FTP upload. Is it possible?

Question

I need to upload multiple files from directory to the server via FTP and SFTP. I've solved this task for SFTP with python, paramiko and threading. But I have problem with doing it for FTP. I tried to use ftplib for python, but it seems that it doesn't support threading and I upload all files one by one, which is very slow.

I'm wondering is it even possible to do multithreading uploads with FTP protocol without creating separate connections/authorizations (it takes too long)?

Solution can be on Python or PHP. Maybe CURL? Would be grateful for any ideas.

Answer 1

Another approach to concurrency besides through is asynchronous io. For Python the standard toolkit for asynchronous network is Twisted.

Take a look at this FTP client example in Twisted.

Answer 2

There is an effor of creating an asynchronous FTP client but basically the FTP protocol allows synchronous commands. To allow parallel downloads/uploads you need to open multiple FTP connections. This can be done by ftplib and threads.

Answer 3

It looks like something might be possible with the pycurl module.

There are two examples that if you put them together would work.

Go here: http://pycurl.cvs.sourceforge.net/pycurl/pycurl/examples/

and look into the retriever-multi.py and file_upload.py for examples on how to possibly put them together.