Persistent Messaging/Service Bus - Roll my own or risk the learning curve?

Question

We have a client application that needs to send messages to a server for various notifications. In order that the client can run occasionally connected I'm going to go with a message-queue approach. The queue processing will take messages off the queue and call a web service that will put them on another queue to finally be processed. This question is about the client environment; the server environment is already decided on.

I don't want to use MSMQ because we don't have control over all the client PCs in order to install/configure and secure MSMQ properly, and because support is more challenging due to the quality of tooling for investigating the contents of MSMQ queues. SQL Server 2005 Express is on all the machines, and is used to store data for our application.

I currently have it down to two options:

1. Write a fairly basic persistent message-queue that stores the messages in a table after serialising them, then uses ThreadPool.QueueUserWorkItem to have them processed by handlers configured against each message type. All in a System.Transactions.TransactionScope so they are only removed from the persistent queue if they are successfully processed.
2. Use NServiceBus (this is the service bus we've gone with as a team, so MassTransit etc aren't options) on the client, with a Service Broker transport that uses the local database.

I have little experience with service buses (I still don't really get service bus terminology) so I'm concerned about the learning curve compared to writing something much more simple that meets my requirements in the way I need it to (deployment is a big consideration).

Does anyone have any thoughts?

Answer 1

In the end I wrote a basic messagebus configured with my own SqlTransport so that messages are serialised and saved to a SQL Server database table, before events are raised and a separate thread is signalled to process the messages.

Answer 2

http://www.nservicebus.com/

Answer 3

Well, I don't know that I'm going to suggest MSMQ, but I will suggest that there's a lot of edge cases to think about for 'roll your own'.

Even with a thread pool approach, be aware that there may be ordering issues if you care - two items posted sequentially to thread pools may not be executed in order, due to how the thread pools handle work items.

You'll also need to think about persistence of messages, and how long they should exist, how to detect 'fatal' non-delivery status, and what to do in that case.

There's also a number of potential edge cases in scenarios where your app goes down around the same time as it's queueing a message - for instance, it may not get the acknowledgement that the message was queued, even though it was. Yes, you can ack the queue acknowledgement, but you can endlessly get into ack circles...

Why not just have the app detect when it gets connected, and send all the data at that point?