Using 50+ 3rd party web services, should I use BizTalk or just C#?

Question

I'm building a back-end application that needs to fetch data on various schedules from over 50+ 3rd party web services and that number will continue to grow. The data from these services can currently be grouped into 3 types so each response needs to be mapped to 1 of 3 known schemas.

Writing custom c# to hit each web service appears to be a management nightmare never mind having all that data mapping in code.

The current thinking is build this on top of BizTalk 2009, still a lot of maintenance but at least a well defined platform with mapping/transformation capabilities already.

I'm looking for any advise from anyone who might have done this before, does this really buy us anything? I am aware of a lack of polling features in BTS however there are enough work arounds to feel comfortable about the solution.

Thanks!

Answer 1

Its refreshing to hear someone considering BizTalk over C#!

I would recommend the BizTalk approach, leveraging the functionality of the ESB Toolkit 2.0 and UDDI Services v3 (found in BizTalk Server 2009), for several reasons:

• The 50 web-service endpoints can be maintained in the UDDI registry - a common management portal where future endpoints can be added and maintained;
• Each web-service call be polled and the resultant message brought onto the bus, mapped into one of three messages and delivered to a specific endpoint;

With regards to mapping, schemas will need to be defined for each of the three common message types and each web-service response; maps will then need to be created to map the response message to the appropriate common schema.

The beauty of using the ESB capabilities in this scenario is that no aspect of the solution will be tightly-coupled: as the web-service response is received, the correct map is resolved (at runtime) from properties on the response message. Once the message is mapped to its common schema format, it can be routed accordingly.

Answer 2

Having implemented solutions like this before, a large factor in the determination of toolset will be the complexity of the webservices you need to invoke. With BizTalk, you will to develop transformations for each of the 3rd-party webservices. This is a simple task the majority of the time, but you may well encounter situations where the mapping becomes non-trivial - in these instances, the time required to implement a mapping rapidly exceeds the equivalent time needed to code a mapping directly in C#.

Will any of the webservices require multiple calls? E.g. obtaining an authentication token prior to polling them? Such services will require more complicated orchestrations to invoke through BizTalk.

Speaking generally, I think you will find it easier to develop and maintain a targeted C# solution to the problem, than utilizing an enterprise system like BizTalk. In my experience of implementing BizTalk solutions, the amount of maintenance required and difficulty in troubleshooting problems massively outstripped any benefit the platform provided. However, my experience was BizTalk 2006 R2 - 2009 may have addressed the issues we encountered.