Implementing observer pattern using WCF

Question

When I first posted this question I had strong coupling between my web service and application controller where the controller needed to open multiple threads to the service and as it received back data it had to do a lot of processing on the returned data and merge it into one dataset. I did not like the fact that the client had to so much processing and merge the returned data before it was ready to be used and wanted to move that layer to the service and let the service open the asynchronous threads to the suppliers and merge the results before returning them to the client.

One challenge I had was that I could not wait till all threads were complete and results were merged, I had to start receiving data as it was available. That called me to implement an observer pattern on the service so that it would notify my application when new set of results are merged and ready to be used and send them to the application.

I was looking for how to do this using either on ASMX webservices or WCF and so far I have found implementing it using WCF but this thread is always open for suggestions and improvements.

Answer 1

One of the ways to achieve this is by invoking your WS asynchronously (http://www.stardeveloper.com/articles/display.html?article=2001121901&page=1, http://www.ondotnet.com/pub/a/dotnet/2005/08/01/async_webservices.html), and then updating the GUI in the callback.

However, you could have timeout problems if the querying of data takes too long. For example, if one of the supplier's web site is down or very slow, this could mean that the whole query could fail. Maybe it would be better if your business logic on the client side does the merging instead of WS doing it.

Answer 2

Not sure if this solution fits your particular task, but anyway:

1. Add paging parameters to your WS API (int pageNumber, int pageSize, out int totalPages)
2. Add a short-living TTL cache that associates request details (maybe a hash value) with output data

When your application asks for the first page, return it as soon as it's ready and put the whole bunch of collected/merged data to cache so when the next page is required you may use what is already prepared.

But note that you won't get the most up-to-date data, configure cache reloading interval cautiously.

Answer 3

This sounds like a perfect use case for Windows Workflow Foundation. You can easily create a workflow to get information from each supplier, then merge the results when ready. It's much cleaner, and WF will do all the async stuff for you.

Answer 4

The absolute best way to archive in your scenario and technology would be having some kind of token between your web app / library against your web service and your controller needs to have a thread to check if there are new results etc. However please note that you will require to get the complete data back from your WS as it's merge can result in removed items from the initial response.

Or I still think that handling threads would be better from controller with the use of WCF Webservices

Answer 5

OK the solution to my problem came from WCF

In addition to classic request-reply operation of ASMX web services, WCF supports additional operation types like; one-way calls, duplex callbacks and streaming.

Not too hard to guess, duplex callback was what I was looking for.

Duplex callbacks simply allow the service to do call backs to the client. A callback contract is defined on the server and client is required to provide the callback endpoint on every call. Then it is up to the service to decide when and how many times to use the callback reference.

Only bidirectiona-capable bindings support callback operations. WCF offers the WSDualHttpBinding to support callbacks over HTTP (Callback support also exists by NetNamedPipeBinding and NetTcpBinding as TCP and IPC protocols support duplex communication)

One very important thing to note here is that duplex callbacks are nonstandard and pure Microsoft feature. This is not creating a problem on my current task at hand as both my web service and application are running on Microsoft ASP.NET

Programming WCF Services gave me a good jump start on WCF. Being over 700 pages it delves deep into all WCF consepts and has a dedicated chapter on the Callback and other type of operations.

Some other good resources I found on the net are;

Windows Communication Foundation (WCF) Screencasts

MSDN Webcast: Windows Communication Foundation Top to Bottom

Web Service Software Factory

The Service Factory for WCF

Answer 6

I'm not so sure that duplex is needed here... IMO, a standard async call with a callback should be more than sufficient to get notification of data delivery.

What is the biggest problem? If you are talking about async etc, then usually we are talking about the time taken to get the data to the client. Is this due to sheer data volume? or complexity generating the data at the server?

If it is the data volume, then I can think of a number of ways of significantly improving performance - although most of them involve using DTO objects (not DataSet/DataTable, which seemed to be implied in the question). For example, protobuf-net significantly reduces the data volume and processing required to transfer data.