Is this a good canditate for a web-service?

Question

Ok so coming in from a completely different field of software development, I have a problem that's a little out of my experience. I'll state it as plainly as possible without giving out confidential details:

1. I want to make a server that "does stuff" when requested by a client on the same network. The client will most likely be a back-end to a content management system.

2. The request consists of some parameters, an input file and several output files.

3. The files are quite large, from 10MB - 100MB of data that must be processed (possibly more). The client can specify destination for output files.

4. The client needs to be able to find out the status of the request - eg position in queue, percent complete. And obviously when and where to pick up output.

So, my questions are - What is a good method for the client and server to communicate? Should the client poll the server, or provide a "callback" somehow for status updates?

At this point the implementation platform is completely open - anything from C to scripting languages like Ruby are available (at either end), my main issue is how the communication should occur.

Answer 1

First thought, set up some webservices between the machines. But webservices aren't going to be too friendly or efficient with the large files.

Simple appoach:

• ServerA hits a web method on ServerB "BeginProcess". The response give you back a FTP location username/password, and ticket number.
• ServerA delivers the files to FTP location.
• ServerA regularly polls a webmethod "GetProcessStatus(ticketNumber)", possible return values: Awaiting files, Percent complete, Finished

Slightly more complicated approach, without the polling.

• ServerA hits a web method on ServerB "BeginProcess(postUrl)", and you send along a URL you want status updates POSTed to. Response: FTP location username/password, and ticket number.
• ServerA delivers the files to FTP location.
• ServerB sends thru updates to the POST location on ServerA every XXX% completed.

For extra resilience you would keep the GetProcessStatus in case something gets lost in the ether...

Answer 2

Files that will be up to 100MB aren't a good choice for a webservice, since you run a risk of the HTTP session timing out before you have completed your processing.

Having a webservice for checking the status of these jobs would be more ideal. Handle the file transfers via FTP or whatever file transfer method you choose and poll a webservice for updates on status. When the process is completed, you might have an output file url returned that can be downloaded.