Hello,
I am experiencing a really curious problem with a HttpHandler and I am hoping someobody here might be able to shed light on this. Many thanks in advance for reading this.
We have created a HttpHandler that sits in the pipeline of an IIS website that serves images, videos, and other assets. The HttpHandler is very lightweight. Its sole purpose is to check if the media asset requested exists on disk and, if it does not, to re-write the URL for the asset to a location where the asset does exist. The handler has been created in this way to allow us to migrate our media assets into a new folder structure. We also plan to use the handler (which I will refer to as the URLRewriter from here on) for SEO on image and video URLs.
As mentioned, the URLRewriter class is very lightweight. We have run memory profiling over it and determined that it only consumes about 12B of memory while running. However, when we put the handler into the IIS pipeline we see some strange behaviour that ultimately results in a large amount of memory consumption and, invariably, that the w3 worker process recycles. The behaviour we see is this:
When a request comes in for an image on http://www.ourimageserver.com/media/a/b/c/d/image1xxl.jpg (not an actual URL) we notice that W3WP.exe creates, and hangs on to, a handle for every single folder in the path to the image:
• /media • /media/a • /media/a/b • /media/a/b/c • /media/a/b/c/d
This is a big problem because we have hundreds of thousands of media assets that are stored in a very wide and very deep folder structure. The number of handles created by IIS/W3WP grows rapidly when the URLRewriter is deployed to our production environment, and the memory consumption of W3WP goes up correspondingly. Within less than an hour of running (at a relatively quiet period in terms of traffic) the number of handles held by W3WP was in excess of 22000 and the process died. We also noticed that the kernel memory usage had increased on the servers where the URLRewriter was deployed.
Careful inspection of W3WP’s behaviour using Process Explorer and Process Monitor (both with and without a VS debugger attached) have revealed that the handles are created before the URLRewriter is called. In fact, the handles are created before the BeginRequest event is fired. When the URLRewriter is removed from the pipeline none of these handles are created. Now, a really curious thing is that it appears that the handles are created as result of a NotifyChangeDirectory operation carried out by W3WP. Why would W3WP request to be notified for changes to these directories? And how can we prevent it from doing so? Surely this is not default/normal behaviour?
If you have any ideas as to what might be causing this problem I would be most grateful for your input. The behaviour is the same on IIS6 and IIS7.