Getting WCF Service timeout after a while (20 minutes)

Question

I have a windows service that is heavily multithreaded. Each thread calls various methods of WCF Service. After around some time all the calls I do to the WCF Service result in a timeout:

The request channel timed out attempting to send after 00:10:00. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout

As you can see the timeout is 10 minutes, also I tried this:

serviceThrottling maxConcurrentCalls="200" maxConcurrentInstances="200" maxConcurrentSessions="200"/>

But it didn't help.

I've checked the w3wp.exe with Process Explorer but I couldn't find anything that would point me somewhere. All the connections I make to the webservice are closed.

Does any one have any idea how to troubleshoot this? I'd like to know if the calls are being processed properly but there's just too many of them, so they start timing out, or are they hanging there, just waiting?

How can I check this out? I was thinking server just gets overloaded, but only CPU is over 50%, memory isn't.

Answer 1

Have a look at idleTimeout:

<system.web>
<hostingEnvironment idleTimeout="20"
                    shutdownTimeout="30"/>
 </system.web>

This is normally set to 20 minutes and you need to change it.

UPDATE More info here:

http://msdn.microsoft.com/en-us/library/bb332338.aspx

Answer 2

The answer is Performance counters. This is such complex solution that you can simply monitor it in some tracing. You need heavy weight tool for health monitoring = build in and custom performace counters. You can implement custom counter for your windows service and monitor how many calls per second is created and how many is completed. You can also use build in counters for WCF service to monitor how many instances is running and how many calls is processed per second. If these values differs to much you have a problem - you are flooding your service with request which cannot be processed in time.

You can also start with simple test to monitor how long does the processing on WCF service takes. If you have processing which takes 1s you simply cannot call the service more than 200 times per second.