Design Solution For Storing-Fetching Images

Question

This is a design doubt am facing, I have a collection of 1500 images which are to be displayed on an asp.net page, the images to be displayed differ from one page to another, the count of these images will increase in the time to come,

a.) is it a good idea to have the images on the database, but the round trip time to fetch the images from the database might be high.

b.) is it good to have all the images on a directory, and have a virtual file system over it, and the application will access the images from the directory

Do we have in particular any design strategy in a traditional database for fetching images with the least round trip time, does any solution other than usage of a traditional database exists?

Edit 1: Each image is replaced by its new entry for every 12 hours, so having them on the database might not be a good idea as far I can think of, but how better will it be to use a data-store and index these images?

Edit 2: Yes, we are planning to run the application on a cluster. And if we try to use a datastore (if it is a good option to go with) then is it compatible with C# & ASP.NET?

ps: I use SQL Server to store these images.

Answer 1

You basically have 2 options

1) Store the binary in the database. VARBINARY(MAX) field will be a good choice of datatype. 2) Store the path to the image stored on disk in the database. NVARCHAR(MAX) will be a good choice for datatype.

There are of course pro's and con's of both solutions. Without knowing more about your requirements Its hard to advise which is the best way.

Answer 2

I prefer not to store images in the database - Instead just store a link(path/filename/id etc) to the correct image.

Then if you implement a HttpHandler to serve up the images, you can store them in whatever location you like. Heres a very basic implementation:

public class myPhototHandler: IHttpHandler
{    

    public bool IsReusable {
        get { return true; }
    }

    public void ProcessRequest(System.Web.HttpContext context)
    {

            if (Context.User.Identity.IsAuthenticated) {
                var filename = context.Request.QueryString("f") ?? String.Empty;
                string completePath = context.Server.MapPath(string.Format("~/App_Data/Photos/{0}", filename));
                context.Response.ContentType = "image/jpeg";
                context.Response.WriteFile(completePath);             
            }

    }

}

For a great resource on setting up a handler check out this blog post and the other related posts.

Answer 3

I wouldn't overcomplicate your solution. The downside to storing images in a database is the database bloat and storage requirements for backups, especially if the images are only good for 12 hrs. If in doubt, keep it simple, so when requirements change, you haven't invested that much time anyways.

This is how I did it on my site.

• Store the images in a folder.
• If you want control over the filename the user sees, or employ conditional logic, use an HttpHandler to serve up the image, otherwise just use its full path and filename in an img tag.
• If you are talking high-volume mega site, perhaps consider using a content delivery network.

Answer 4

Have you considered caching your images to mitigate the round-trip time to SQL server? Caching might be appropriate at the browser (via HTTP Headers) and/or the HTTP handler serving the image (via System.Web.Caching).

Storing images in SQL server can be convenient because you don't have to worry about maintaining pointers to the file system. However, the size of your database will obviously be much larger which can make backups and maintenance more complex. You might consider using a different file group within your database for your image tables, or a separate database altogether, so that you can maintain row data separate from image data.

Using SQL server also means you'll have easy options for concurrency control, partitioning and replication, should they be appropriate to your application.

Answer 5

All of the previous comments are really good... In the absence of very specific requirements we have to make broad generalizations to illustrate your options. Here are a few examples.

• If raw speed is what you need, then flat files are a clear winner. Whether your using Apache or IIS, they're both optimized to serve static file-based content very fast. High performance sites all know this, and will store much of their content with dynamic handling in mind but will then "publish" select pieces of their dynamic content; in static versions; to their web farm on a periodic or event-driven basis. Although it requires a bit of orchestration, it can be done cheaply and can really reduce the load of your database server, backend network, etc. As a simple example, publish to a folder with the root of that structure being dynamically assessed. when you're ready to publish updates, write a new folder, and then change the root path. No down time, cheap, and easy. On a related note, pulling all this information from a backend store will require you to load these things into memory. This will ultimately translate to more time in Garbage Collection and consequently will mean your application is slower, even if you're using multi-processor/core gardening.

• If you need fine-grained control over how images are organized/exposed, then folders may not be the most appropriate. If for example, you need to organize an individual users images and you need to track a lot of meta data around the images then a database may be a good fit. With that said, your database team will probably hate you for it, because this presents a number of challenges from a database management perspective. Additionally, if you're using an ORM you may have some trouble making this work and may find your memory footprint grows to unacceptable levels due to hidden proxy objects, second-level caching, etc. This can all be mitigated so just watch out and make sure you profile your application. With that said, a structured store (like a DB) is more ideal for this use case.

• Considering security... Depending on what these images represent, flat files inevitably lead to concerns about canonicalization attacks, brute-force enumeration of browsable folder structures, replaying cookies, urls, viewstate, etc. If you're using a custom Role-based or Claims-based security model you may find using flat files becomes somewhat painful since you'll have to map filesystem security constraints to logical/contextual security constraints. Issues like these often lead me to favoring a structured store.

The aformentioned cache idea is a good one and can help create some middle-ground with respect to how often you actually hit your database, although it will not help with concerns related to memory consumption, GC, etc... You could employ built-in caching mechanisms although a Cache/Grid that supports backing stores would be much better if you can afford it (Ex. NCache, ScaleOut, etc.). These provide nice scalability/redundency, and can also be used to offload storage of session state, viewstate, and a lot more.

Hope this helps.