I want to optimise the upload of a file onto the server - whats the best way to do it?

Question

Hi all,

I have a rails app that does the following (at the moment linear) process:

(1) User uploads a file via HTTP and a standard upload form on a standard HTML page (2) The file is uploaded to an apache server (same server as the one hosting the app) (3) The server uploads the file to remote storage service (call this storage 1) (4) When (3) is done, the user is taken automatically to some other page

While the file is being uploaded the user sees a progress bar etc.

Now question: - (1) -> (4) takes quite a long time for large files (because processes (2) and (3) take a while) and the browser waits for a server response before moving to (4) (i.e. a response from server indicating the completion of (3) have completed).

Is there any way of speeding up (1) -> (4). I don't mind if the file loads into storage 1 and onto the server in the background while the user is allowed to browse other page on the site.

Now the additional thought is - the server to which the file is uploaded to (lets call it server 1) can be different to the hosting server (server 2) if necessary.

Any ideas appreciated.

Answer 1

As soon as the file is uploaded to the server, move them on. Let the server itself handle the moving the file to "storage 1" using something like Delayed Job and then notify the user (using a system such as email) that the file is now available to be downloaded.