Do large file uploads block an applications request/response cycle? I have an app that allows users to upload multiple large files (images in particular). These files are stored on a remote host. I cannot use async background jobs to upload these images as these have to be immediately accessible to the user once the upload finishes. How best should i handle such large uploads? Does it affect concurrency? This is my first time with uploads on a large scale. What should i be wary of other than the huge bills of course? Any input from developers who have created apps which use large file uploads will be greatly appreciated.
+1
A:
Why can't you use an async upload, and just handle the event that signifies that it's done? That's generally how async operations work - you kick them off and then store the pointer somewhere, and then either handle the "Complete" event, or just periodically iterate through he pointers for uploads you've started and check each one to see if it's complete.
rwmnau
2010-08-16 13:56:39
@rwmnau: If you mean async upload on the client side, yes that is what i am doing as of now. When i said async upload, i spoke about asynchronous background processing (non-blocking I/O) on server side(spawn extra processes/threads). I am more worried about uploads blocking my app. Concurrency to be precise. Does that happen with large file uploads? A non responsive app is the last thing that i want. :)
2010-08-16 14:09:10
@user374142: I'm not sure exactly what you mean - are you concerned that a partially uploaded file may block your server app somehow? Or are you concerned about multiple uploads in the background on causing performance problems for the client? I wouldn't expect it to, if you're running them on additional threads - the OS should give processing priority to the UI thread, and the only concern I'd have is about physical disk contention - if you're uploading multiple files at the same time, the client will need to read all those off disk at the same time, potentially causing disk thrashing.
rwmnau
2010-08-16 14:36:32
@rwmnau: Ok, let me make it more simpler. I just want to know if file uploads block a web app until the upload finishes. ex: If one user is uploading a file, does it make the app unresponsive for another user (i.e for the duration of the upload)?
2010-08-16 14:51:06
@user374142: Ah, I see what you're asking. As far as I know, every major web server works with each client in a different thread, so no, a large file upload wouldn't block additional users of a web application at all - it would happen in the background, as far as they're concerned. I know that's the case at least for IIS and Apache, but web servers are built to be multi-threaded, so I imagine it's the same for every mainstream server available.
rwmnau
2010-08-16 15:04:11
@rwmnau: Thanks a ton! Relief. Up vote to you! :)
2010-08-16 15:13:06