Multi-Threaded Update Server

Question

We are wanting to replace our current update system for the POS software at our stores. Currently, we have a folder on our webserver that hosts a script that our registers call every 30 minutes or so. They pass in their current version number, and the server returns an XML response containing any files that need to be updated, hash, and any other pertinent information for the register to download the updated files.

We want a service that will listen for UDP packets that we send out that we can use to trigger an update. This will keep the registers from unnecessarily polling for updates. We are writing an update server that will listen for a TCP connection from a machine, authenticate, do some logging, and then send along updates for the files that have changed. We want to move to a custom update server, because we have the need to release versions of the POS to certain machines for a trial run after testing before releasing it company-wide.

I've written the code to handle incoming connections and create a new thread to handle communicating with that client, etc, but I am having a hard time with handling file I/O in a multithreaded manner.

I'm pretty sure I won't be able to just access a file by opening it from one thread if it's already open in another. I don't want to read the files into memory and keep them in a threadsafe container beforehand, since that would use a lot of memory, especially if it turns out I need to keep a copy for each thread in order to avoid threading issues.

What would be the best way to go about being able to access these files from disk/memory in a thread-safe manner without using a lot of memory or causing threads to wait on other threads to finish with the files?

EDIT: Forgot to mention we are using C#.

Answer 1

See C# async IO documentation.

Are the files changing? Why wouldn't you be able to read them from separate threads? You can generally open a file twice in separate threads. But don't expect your writes/reads to be ordered.

What happens if a register misses an update, as it might with a UDP update notification? You should probably continue to poll at least now and then.

it sounds like you're reinventing a webserver... could proper configuration of a webserver acheive the same effect (TCP connection, logging, file delivery)

Answer 2

Multiple threads can open a file if it's for reading only. Just set your file mode correctly.

You will get errors (on Windows at least) if two threads try and open a read/write handle to a file.

Answer 3

Windows has the TransmitFile function that may be of interest. That way you let the OS handle any caching for you. This is a C call, but it should be fairly easy to use from a C++/CLI project. The call can be done asynchronously using overlapped io.

This way the Windows Cache manager reads the file and sends the data on the socket without you having to touch the data at all. The function is intended for high-performance file data transfers over sockets.

Just open the file, keep handles to files cached (so that you don't need to open them again and again) and call TransmitFile to send them on a connected socket. You need to be on a server OS for the function to work well (there are limits on transfers on client OS versions).