http based filesystem with seek and lock support

Question

Hi -

I'm building an application that needs to use a web server like a file system. Specifically, the ideal solution would be:

• A server side component that would allow, via HTTP, opening (locking), reading, writing, seeking and truncating one file. I need to be able to lock multiple files simultaneously. I need this to run on any of the standard web hosts out there, so it must run on top of IIS or Apache, using server side scripting (Web Services, PHP, etc)
• A client interface to this that I hand it a URI, and it opens and acts just like a file. My client app is in .NET, so ideally the client API would implement a FileSteam subclass.

The reason I need this is that I have rather large files that will be sitting on a web host. I need to read parts of the files (using seek), truncate, and append to the end. Downloading and uploading the entire file is too bandwidth intensive. Also, I need to lock multiple files simultaneously.

My questions:

• Does such a thing exist?
• If not, why not? Does this not seem like a very useful service?

It seems like such a component would allow me to use any web hosting company out there as a network file server. That seems useful, no?

Thanks for your help!

Eric

Answer 1

It seems what you are looking for is a WebDAV server.

Answer 2

Long version
Web hosts were originally designed to send content tothe user in individual connection. The whole concept of sessions is a relatively modern phenomenon (key word: relatively) - most of which are simply methods to track individual connections from the same user. More often than not, its just giving the impression that you're still connected to the server, when in reality you've connected, downloaded, disconnected - and do the same connect/download/disconnect every time you take an action.

Things like WebDav that are tack-on additions to the standard HTTP dialog that allow specially designed software to have more complex communication via HTTP to a server. Very common use for WebDav is Subversion, iCal (calendaring support), etc - these allow for single connection uploading of data.

What you're talking about is having a session-based filesystem service. While it seems that it would be a good service, i suspect it hasn't been implemented due to the sheer complexity and danger outweighing the benefits. For example, not only would such a service have to worry about standard filesystem issues (authorization, file permissions, locking, timeouts, etc), but now also has to worry about the complexities of being on the web (authentication, session-tracking, disconnects, replay attacks, DDoS, etc). For example, what happens if a client locks a file then falls offline? Does the file stay locked until the server reboots? If not, when do you automatically unlock the file? What happens if the auto-unlock triggers and the original locker comes back online? Do you force everything to be done in a single connection to compensate? What happens if they get disconnected in the middle of a session?

They are but a few things that you have to worry about. I've rambled on much longer than i meant to, but the bottom line is that...

Short Version
You can certainly use WebDav to implement this, which will require you to build a server-side library. However, while its not impossible, you're in for a world of hurt as designed. I'd suggest simplifying the design it so that you don't need it to act like a filesystem so much.