Using an HTTPContext across threads

Question

User hits page spawn.aspx which then spawns a half-dozen threads, rendering pages all using

 ((System.Web.IHttpHandler)instance).ProcessRequest(reference to spawn's HTTPContext);

Don't worry about the fact that ASP.Net is seemingly sending the user 7 responses for 1 request, that part is handled and only one response gets sent.

The problem is, in a high-traffic enviroment (our Production enviroment) with many threads (quad-quads) we get an error:

System.IndexOutOfRangeException 
at System.collections.ArrayList.Add 
at System.Web.ResponseDependencyList.AddDependencies(String[] items, String argname, Boolean cloneArray, DateTime utcDepTime) 
at System.Web.ResponseDependencyList.AddDependencies(String[] items, String argname, Boolean cloneArray, String requestVritualPath)
at System.Web.UI.Page.AddWrappedFileDependencies(Object virtualFileDependencies) 
at ASP.spawned_page_no_1_aspx.FrameworkInitialize()
at System.Web.UI.Page.ProcessRequest

We can't duplicate it elsewhere. My coworker believes this is because I'm reusing the original HTTPContext and passing it into the other threads, and that it's not Thread-Safe.

Following this logic, I've tried making a new HTTPContext to pass into the threads. But parts of it seemingly won't "combine". Specifically, I need to get the Session object into the new HTTPContext. I imagine I'd want to get other parts in as well, like Cache. For the record HTTPContext.Current.Session.IsSynchronized is false.

My questions are:

1. Do you think the error is from using HTTPContext across threads?
2. How can I fix it?
3. If the fix is duplicating the HTTPContext for each thread, how can I get the Session (and Cache) into the new one? Request and Response come in the ctor, but Session is not settable.

Edit: More Details

So going back to this statement: "Don't worry about the fact that ASP.Net is seemingly sending the user 7 responses for 1 request, that part is handled and only one response gets sent." Huge fan of Raymond Chen, I agree with you: "Now you have two problems" is a reasonable statement in the absence of any more information.

What's actually happening is that I'm building an Excel Document to send back. In the spawn.aspx page it's setting up some state information, including the fact that it's rendering to excel, and the object to do the rendering to. Each spawned page gets that information, and will block until it's their turn to render to the object. If literally looks like this:

 protected override void Render(System.Web.UI.HtmlTextWriter writer)
 {
    if (this.RenderToExcel)
    {
      Deadlocker.SpinUntilCurrent(DeadLockToken);
      RenderReport(this, this.XLSWriter);
      Deadlocker.Remove(DeadLockToken);
    }
    else
      base.Render(writer);
 }

But all the processing up to that point - database access, control heirarchy, all that's done in parallel. And there's a lot of it - enough that parrallizing it while still letting it block on Render will cut the overall time in over half.

And the best part of it is - nothing had to be rewritten for the Excel render. All the controls know how to render themselves to excel, and you can visit each spawned page independently (that's the 'normal case' actually - the excel report is just an aggregation of all the spawned pages.)

So I figured the end result was going to be "you can't do this, you need to rethink the approach" - but I had to at least try, because the fact that everything works so nicely without duplicating any logic or any code or having to abstract anything is just so perfect. And it's only multi-threading that's the problem, if I render the pages serially everything is fine, just slow.

Answer 1

Your co workers are right, if one thread locks a resource and another thread attempts to use it then your thread pool goes boom! Not very good outcome. Most people resolve this by creating new objects and passing them into paramaterized threads. If you absolutely need to use the same object then you must implement some code that first checks to see if the resource is being used by another thread and then waits a little while before checking again. An example would be to create a IsInUse bool that you always check first, then your thread sets it to true if it is using that resource, then false when it is done, preventing other threads from attempting to use the underlying resource (your httpContext). I hope this helps.

Answer 2

Whilst the HttpContext is designed to handle a context that isn't thread specific (because the http context can start on one thread and finish on another), it isn't implicitely thread safe.

Essentially the problem is you are doing something that isn't intended, these requests would be multiple generally and each have their own assigned HttpApplication to forefill the request, and each have their own HttpContext.

I really would try and let the asp.net infrastructure delegate the requests itself.

Answer 3

Since HttpContext is part of the .Net library, I would expect that all static functions are threadsafe but any non-static members are not threadsafe when calling against the same instance of the object. So hopefuly next time you will anticipate that sharing the HttpContext instance across threads would have problems.

Is there a way you can decouple the operations you need to carry out in parallel from the HttpContext? If they are just loading data and writing to CSV format, that code has no necessary dependency on ASP.NET user control or page lifecycle. Once that dependency is removed, you can implement the page with an asynchronous HttpHandler, running the parallel operations during between IHttpHandler.BeginProcessingRequest() and IHttpHandler.EndProcessingRequest().