File upload on load balanced server

Question

I have a web project that I developed where one of the main functions is uploading files.

All the permissions are based on Windows Authentication. When I first created the project, I didn't realize that it would be on a load balanced server. Now it's in the environment with the load balanced server and the file upload has become a major issue.

I originally thought that I could just create a virtual directory to a network share that would be the upload folder, however that solution has turned into a permissions nightmare. I haven't been able to do any successful test runs in the production environment.

So I'm thinking now, my solutions would be to create a service on the servers that checks the upload directory and syncs them up. The other possible solution I see is to store the files in the database which I've had some issues with in the past.

Does anyone know of a straight forward solution for uploading files to a load balanced server that authenticates using Windows Permissions?

Thanks.

UPDATE - So I have changed the application pools to run through a domain account that has full permissions on the share and the folder itself in the Windows permissions. The Virtual directory has full privileges in IIS, but I'm still getting the same issue.

One of the things that needs to be done with this is also the ability to create directories, which I think might be tripping me up now. I'm just using System.IO.Directory.CreateDirectory

Any other thoughts on where I could be missing permissions?

Answer 1

The easiest thing IMHO is to store them in the database as you suggested. Just get the bytes from the file stream and save it in your db in a VARBINARY(MAX) field. I believe this has a maximum size of 2 GB.

This article lists some good advantages and disadvantages to storing in a database:

Advantages

Of course some of the advantages of a database are the exact opposite of the disadvantages of saving them as physical files: since they are stored in the database, they're easy to relate to other records. They can be retrieved in JOIN style queries, and even be deleted automatically with cascading delete turned on. You also don't need additional permissions on the server; if you can write to the database, you can store files in it.

But another advantage of storing your files in a database is the fact that all data is contained in a single location. Make a backup of your database, and you have everything you need. That makes it a lot easier to move your data to another server; other than the database, you don't need to copy files, set up permissions and so on.

Disadvantages

At the top of the list of disadvantages of storing your files in a database is probably performance. While I don't have any hard figures to support this, the "word is" that it's slow. How slow may depend on your situation, the type of files you have, the server, and so on.

Another downside is the lack of easy access to the files. When you store them on disk, it's easy to download them to your desktop machine and batch process them; for example, use an imaging program to scale or rotate all your images. When you use a database, you need to "materialize" them to disk first and upload them again afterwards.

A final problem with the database is backups. Whenever you make a full backup of your database, all the files are included, whether they have been changed or not. If you copy your backups to a different machine or network for safety reasons, this could be problematic as you need to move the entire backup file. With a file based solution, you can use diff programs that can determine which files have been changed since the last backup, and only download those.

Answer 2

If you really want to avoid storing in a database, I think your next best bet would be the virtual directory solution you already spoke about, though I can see how the permissions issues could very easily get out of hand depending on your setup.

In our production environment, all our distinct sites have their own app pools. The app pools themselves each have their own domain account, which makes managing permissions in the filesystem and in SQL server much easier. Each of these accounts is a member of an IIS WP domain group. On each of the servers in the cluster, the domain IIS WP is a member of the local built-in IIS_WPG group. The IIS configuration is identical on each server in the cluster. The important effect of this is to ensure that a given web application is always running as the same identity regardless of which server in the cluster is being hit.

With the setup I've described, it would be very straightforward to implement the virtual directory solution, though you would still have to worry about other obvious issues like naming collisions and such. If you're still using the default app pool identity setup, I think implementing what I've described will help make your life easier in general when dealing with your cluster configuration.

This article lists some good advantages and disadvantages to storing in the file system:

Advantages

One of the main benefits of storing the file on disk is that it's very easy to do. Just call SaveAs on a FileUpload control and you're pretty much done.

Another advantage is that files on disk are easy to backup; you just copy the files to another location. This also makes it easier to do incremental backups; files that have already been backed up don't need to be copied again.

Disadvantages

Storing your files in the file system has a few disadvantages as well. Probably the most problematic issue is the loosely coupled nature of the files on disk. They have no strong relation with a record in the database. So, when you delete, say, a product from the database, you may end up with an orphaned product image. There is no direct way to do an INNER JOIN between the product table and your images folder to determine what orphaned files you have left. This means that a page developer is responsible for writing code that deletes the file from disk whenever the associated database records gets deleted.

Also, to store uploaded files on disk, your web server needs permissions to write to the file system. This is easy to come by when you run your own server, but may prove to be more problematic in an ISP scenario.

Answer 3

Your solution should work fine:

1. Create a network account
2. have the application pools of the application on each server run under this account
3. create a folder on a specific server and give this network account rights to it
4. set the folder as a 'share' and give the account rights to the share as well
5. Set all servers to use this directory to store the uploaded files

Answer 4

FYI,

The solution to this problem was to do temporary impersonation with a user account that had access to the share, permissions on the folder and permissions on the virtual directory.

When I had initially tried this, I had some mistakes in my impersonation code, then when fixed a bad password was entered so I abandoned this approach. I had finally tried everything else and came back to this - looks like it's working.

So the answer is impersonation when dealing with network shares as virtual directories it looks like.

Here's the link to the code that made it all happen:

http://support.microsoft.com/kb/306158#4