How to handle 30k files in a project which requires them?

Question

Visual Studio 2010 RC -> Silverlight Application

We have a library of images that we need to have access to. They are given to us from a vendor (through an installer) and they are not in a database, they are files in a folder (a very large monster of a folder). We do not control when the images change, so the vendor needs to be able to override them individually. We get updates frequently enough from this vendor to state that these images change "randomly" and without our (programmer) knowledge.

The problem: I don't want 30K images in SVN. Heck, I don't even want to imagine them in my Solution. However, our application requires them in order to run properly. So, our build/staging servers need access to these images (we have two build servers).

The Question: How would you handle it when your application will not work as specified without access to each of 30k images and you don't control when those images change?

I'm do not want to have a crazy large SVN repository. Because I don't know when any of these images change, I really don't want them in my solution (definitely do not want a large solution, either). I also don't want a bunch of manual steps to do every time these images change.

Our mantra, up to this point, has always been, any developer could download from SVN, compile and run our app. These images are going to kill that mantra.

I'm tempted to make a WCF service that will return images if they exist and a dummy image if they don't. This way all dev boxes will return a dummy image and our build/staging/production boxes will return real images (ones that actually have the vendor's image installer installed on).

This has to be a solved problem.

What have other people done to handle these types of problems?

I'm open to suggestions.

Answer 1

Typically with large amounts of pictures, especially up to 30k, you are going to need a content delivery network specifically for your images. It shouldn't be more than setting up a simple HttpHandler that can handle requests which can in turn return those photos to your app.

Answer 2

In the past, I've created a virtual directory in IIS whose sole purpose was serving images.

Answer 3

I would strongly discourage storing the images in source control, especially if they are going to change frequently. Additionally, adding each of these images to your project/solution is probably going to kill performance in VS.

I think the idea of pulling in a dummy image for devs is good. As part of your deployment process, you could add something to your build script that will copy the images to the correct location.

Answer 4

If theses pictures are not not available in your source control they are not in your build and are not shipped with your deployment package.

Thus your application should work with and without the image files present. Returning a dummy probably is a safe solution if an image is missing.

Answer 5

I'd imagine merging images between branches would be a nightmare of epic proportions so I agree that you should keep them away from CVS/SVN

The question you have to ask yourself is, are the images integral to the project at developer level? The dummy images idea would be a suitable answer for this.

Answer 6

I would definitely recommend against keeping those images in source control.

I don't know which way your application accesses the files.

If it's through the file system, how about putting a hard link to a central images directory into the repository? That way, your developers can check out the app, and have a working version right from the start (provided it's possible to check Windows hard links in and out of a repository, may require some scripting otherwise.)