Reliable n-Tier WCF (Threading issue?)

Question

I am working on an n-Tier application using WCF between the layers such that:

Tier 1: Silverlight application Invokes the search request

        IClientBroker clientBroker = UIContext.CreateWcfInterface<IClientBroker>("Data/ClientBroker.svc");
        clientBroker.BeginSearchForClients(SearchTerm, 20, (result) =>
        {
            SearchResult[] results = ((IClientBroker)result.AsyncState).EndSearchForClients(result).ToArray();

            // do stuff, update UI, etc.

        }, clientBroker);

Tier 2: is a WCF web service using basicHttp for Silverlight to call. This is acting as a proxy to the 3rd tier.

[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
class ClientBroker : IClientBroker 
{
            [OperationContract] // as defined in IClientBroker
    public SearchResult[] SearchForClients(string keywords, int? maxResults)
    {
        ClientBrokerClient clientBroker = CreateClientBrokerClient();

        SearchResult[] searchResults=clientBroker.SearchForClients(keywords, maxResults);
        return searchResults;
    }
    }

Tier 3: is the "server", in that this offers a net.tcp endpoint (allowing secure clients to connect without using Silverlight). This is the ultimate target of a request.

    public class ClientBroker : IClientBroker // note this is different to tier 2 interface
    {

        public SearchResult[] SearchForClients(string keywords, int? maxResults)
        {
                    // do stuff
            if (maxResults.HasValue)
            {
                return results.Take(maxResults.Value).ToArray();
            }
            else
            {
                return results.ToArray();
            }
        }
     }

So my calls go:

Silverlight -> httpBasic -> IIS-hosted Proxy WCF service --> net.tcp --> EXE-hosted WCF service

This works well. I can pass headers through the layers, and maintain sessions, etc. And it is pretty trim.

BUT it takes only a few calls through this to cause a Timeout in the communication.

The time the server-EXE takes to do its work is neglible. The problem I am seeing is that the server "freezes" in returning results to Tier 2.

I think this is to do with a thread getting locked.

I've looked around and see that the ideal way of doing this is to make my Tier 2 run asyncronously, similar to the code below:

    public SearchResult[] SearchForClients(string keywords, int? maxResults)
    {
        ClientBrokerClient clientBroker = CreateClientBrokerClient();
        clientBroker.BeginSearchForClients(keywords, maxResults, result =>
            {
                SearchResult[] searchResults=((ClientBrokerClient)result.AsyncState).EndSearchForClients(result);
                // how to return results from here?
            }, clientBroker);

    }

But how do I achieve this when my Tier 1 client is waiting for the result of this method, which will just drop straight out before the callback execute? Am I missing something on my construction of my OperationContract methods?

UPDATE:

I have since put my server (Tier 3) through its paces from a client avoiding Tier 2 by making many requests from it. Seems the net.tcp WCF channel for Tier 3 is rock solid.

UPDATE 2:

This blog post outlined using the IAsyncResult pattern, which I have eluded to here. Am I barking up the wrong tree here? http://blogs.msdn.com/wenlong/archive/2009/02/09/scale-wcf-application-better-with-asynchronous-programming.aspx

UPDATE 3:

Ok, this paragraph from that blog:

"If you are building N-tier WCF services, you would have WCF service operations invoking WCF client proxies for other backend services. In this case, you would need to make sure that the middle-tier (routing layer) has asynchronous service operation invoking asynchronous WCF proxy operations. In this way, your middle-tier won’t run out of threads when processing many slow operations."

seems to confirm my suspicions that the problem lies in the middle layer (Tier 2). How can I achieve this Begin/End asynchrony? Do I have to do this manually or can I retain VS tools to generate the proxy classes for me? (REALLY don't want to have to do this manually, the contracts have a degree of flux in them)

Answer 1

Well, I think I've solved it. This topic helped me:

http://stackoverflow.com/questions/2190238/wcf-service-stops-after-few-requests

Basically, I hadn't been closing my client proxies in my Tier 2, which I realise would cause blocking. The evolution of the code was such that I ended up removing using() {} blocks to facilitate exceptions not getting extinguished with the client proxy. However, I have restructured and retested and my Tier 2 code now looks like:

    public SearchResult[] SearchForClients(string keywords, int? maxResults)
    {
        ClientBrokerClient clientBroker = CreateClientBrokerClient();

        SearchResult[] searchResults=clientBroker.SearchForClients(keywords, maxResults);
        clientBroker.Close();
        return searchResults;
    }

... and exceptions are not extinguished.