Asp.net caching and w3p memory usage

Question

Web application storing images in database.

In our first setup we were storing (and serving) from images from database resized in asp.net cache. This was all good, but as web started getting more and more hits, we started experiencing weird problems, and by weird, I do mean weird - ranging from w3p crashing without any details to random OutOfMemoryExceptions occurring everywhere couple of times a day.

I suspected that asp.net cache my be misused by us here, so I rewrote image caching to go to filesystem. Our image handler now checks for resized image in cache dir, if no such image exists, fetches it from database and saves it in cache dir, and serves it from there.

The thing that's bothering me is memory usage of w3p for this web site. Before, when we were using asp.net cache object, typical memory usage for w3p was around 600mb. Now, when handler serves images from filesystem typical memory usage is around 750mb. Somehow I expected it go down (at least a bit), but definitely not up. Is there any explanation for this?

Second, is there a way to inspect w3p process in order to see how much memory is used for what? One thing that comes to mind is performance counters, but haven't had the time to check it out.

Answer 1

This is just speculation on my part, but if the file I/O is buffered, wouldn't that use a lot of memory? More so perhaps than the ASP cache as each file served up requires a buffer instead of taking a shared resource from the cache.

As for memory profiling, there are a number of profiler programs out there you could use them to analyse what's going on.

• .NET Memory Profiler
• DotTrace
• ANTS Performace Profiler

Answer 2

Are you sure that you've disabled the OutputCache for those items? It seems like you still execute dynamic code to determine whether to create the resized image if necessary, so there still could be an active OutputCache

Answer 3

That many images into cache causes it to grow and grow. Default behaviour of asp.net is restart when process riches 60% of available memory.

OutOfMemoryException can be throwed when one process will be recycled and second will be created but first doesn't be finished for a long time. Then you have 2 processes taking a lot of memory. If application is busy and there is a lot of requests then could be more than 2 processes at time. This lead to OutOfMemoryException.

I rewrote image caching to go to filesystem. Our image handler now checks for resized image in cache dir, if no such image exists, fetches it from database and saves it in cache dir, and serves it from there.

This is the best solution for this. FileSystem is cached at OS level.