Storing uploaded content on a website

Question

For the past 5 years, my typical solution for storing uploaded files (images, videos, documents, etc) was to throw everything into an "upload" folder and give it a unique name.

I'm looking to refine my methods for storing uploaded content and I'm just wondering what other methods are used / preferred.

I've considered storing each item in their own folder (folder name is the Id in the db) so I can preserve the uploaded file name. I've also considered uploading all media to a locked folder, then using a file handler, which you pass the Id of the file you want to download in the querystring, it would then read the file and send the bytes to the user. This is handy for checking access, and restricting bandwidth for users.

Answer 1

I think the file handler method is a good way to handle files, as long as you know to how make good use of resources on your platform of choice. It is possible to do stupid things like read a 1GB file into memory if you don't know what you are doing.

In terms of storing the files on disk it is a question of how many, what are the access patterns, and what OS/platform you are using. For some people it can even be advantageous to store files in a database.

Creating a separate directory per upload seems like overkill unless you are doing some type of versioning. My personal preference is to rename files that are uploaded and store the original name. When a user downloads I attach the original name again.

Answer 2

Consider a virtual file system such as SolFS. Here's how it can solve your task:

If you have returning visitors, you can have a separate container for each visitors (and name it by visitor login, for example). One of the benefits of this approach is that you can encrypt the container using visitor's password.

If you have many probably one-time visitors, you can have one or several containers with files grouped by date of upload.

Virtual file system lets you keep original filenames either as actual filesnames, or as a metadata for the files being stored.

Next, you can compress the data being stored in the container.