Performance of WCF with net.tcp

Question

I have a WCF net.tcp service hosted with the builtin ServiceHost, and when doing some stress tests I get a strange behavior. The first time i send a bunch of requests, 5 to 10 requests are answered quickly, and the rest are returning at about 2 second intervals. The second time i send the requests, 10 - 20 are returned quickly, and rest with 2 sencond intervals.

The above repeats until I can get over 100 requests returned quickly, but if I wait a minute or so the memory usage of the service goes down and the requests go back to 5-10 returning quick.

The service I am testing has a small delay, so that I can get many open connections at the same time, if this delay is removed the requests return so quickly that i have perhaps 2-5 connections open at the same time. This delay is to simulate DB connections and other outgoing stuff.

From the behavior it looks like the ServiceHost is allocating something, threads, class instances, but I can not figure out what it is.

I could have a timer in the client that calls the service to keep it working, but that seems like a bad solution.

If I have a high sustained load to the service it will crunch all requests quickly, but if I have a period of low activity and then a surge of connections comes in the service will be slow.

I guess my question is WHAT is it the get allocated during high load of the WCF service, and HOW can I configure the service to preallocate more of the things that get allocated.

EDIT: I did some more testing, and looking at the taskmgr for the process I can see that when the servicehost is 'resting' there are 10 threads open, but when I start sending requests, the threadcount goes up. As long as the threadcount is high the servicehost can process incoming requests quickly, but if I pause sending the requests, the open threadcount decreases, and subsequent requests starts taking longer time to process.

Now, how can I tell the servicehost to keep a bunch of threads open? Or more than the 10-12 that it keeps by default?

Answer 1

The thing that determines how may request are handled simultaneous is the ServiceThrottlingBehavior. There are a number of different threasholds that will limit the amount of request being processed. This also depends on the binding your are using, for example wsHttpBinding defaults to sessions on while basicHttpBinding uses no sessions and the default session limit of 10 is no problem.

See http://msdn.microsoft.com/en-us/library/ms735114.aspx for more details.

Answer 2

Well, after lots of googling, it seems that the problem is the threadpool. The CLR threadpool allocates a few threads, and when they are used, it throttles the creation of new threads, and after a time it also deallocates unused threads.

There is some confusion about a bug that meant that the ThreadPool did not honor the SetMinThreads call.

http://www.michaelckennedy.net/blog/PermaLink,guid,708ee9c0-a1fd-46e5-8fa0-b1894ad6ce0f.aspx

I am not sure if this bug is solved, or what, because when I modify the ThreadPool settings, the problem persists.

Answer 3

The bug you referenced is fixed in .NET 3.5 SP1. That may have had something to do with the problem, I think it's more likely (much more likely) that throttling is your problem rather than thread as Maurice keyed into.

<system.serviceModel>
  <service name="???" >
   <endpoint ... />
  </service>
</system.serviceModel>

What's the throttle limit for this "empty" config? 10 session, 16 concurrent calls! Beware.

Here's more on the threading: http://www.michaelckennedy.net/blog/2008/08/20/ThreadPoolBugInNET20SP1IsFixed.aspx