ASP.NET - best queue system for a new application

Question

My organization is getting ready to implement a new system, which is a asp.net application. The application will have a large queue of offline work that is initiated by the website. This queue will hold different types of activity, ideally in XML messages. Think of things like email notifications, scheduled tasks, etc.

In the past, the organization would likely have used MSMQ to accomplish this task. However, they consider MSMQ to be old school (and I partly agree with them), so we are going to do an architectural review to determine the "best" solution.

In my mind, there are a few potential choices:

1. Stick with a new implementation on the latest version of MSMQ - not ideal, but a known product.
2. Use Windows Workflow Foundation, which I've heard from a few other developers that have used this for this type of thing.
3. Develop a custom database solution.

Am I missing any obvious solutions? This ideally will be a Microsoft product, but really just needs to work in a Microsoft centric shop.

I'm concerned about the following:
1. Ease of implementation and maintenance
2. A solution which will be around for awhile
3. Able to handle a good volume of rows, with medium sized XML data in them
4. Absolutely reliable queue system, with quick updating (multiple utility processes will likely be grabbing records out of the queue to process them).

Answer 1

Have you looked into a Service Broker in SQL Server? It is a queing system which uses the database as a backing store.

Answer 2

I would suggest looking into WCF, you can configure it to specify persistent, queued messaging and it uses MSMQ technology under the covers. The WCF abstraction/interface and technology should be around for a good long time.

Answer 3

Reading the post seems like the only reason you think MSMQ is not suitable is because someone thinks it is "old school". I would not think that is a good enough reason not to use it, since it looks like your company has experience with it, so there would be no learning curve, and that means easy implementation and maintenance.

Plus, MSMQ would perfectly address all the concerns you mentioned you have. So, unless there is another "real" reason not to use it, I would think stick with MSMQ.

Answer 4

You have a couple of options:

1. Biztalk: It is built for guaranteed message delivery and routing at the corporate level. It is difficult to set up, expensive, and has a steep learning curve however once you turn it on it's solid.

2. MSMQ: Fast, cheap as in Free, easy to use, and just plain works.

3. SQL Service Broker. It is a step up from MSMQ, but a huge step down from Biztalk.

The main issues really boil down to the feature set you need. Biztalk is pretty much it's own dev environment. Whereas MSMQ requires you to build everything around it.

Answer 5

You could look at using an open source message broker such as Apache ActiveMQ

Answer 6

I agree with moose-in-the-jungle that MSMQ probably is what you should stick with.

I would maybe research some alternative API's that use MSMQ under the cover, like nServiceBus from Udi Dahan.

Answer 7

It seems like for your whole system though, you could use Windows Workflow to manage your business logic, and just use MSMQ as the storage for your task list. Your workflow will start by pulling the next message off the queue, then determine what to do with it.

Queueing is something that you don't want to mess with yourself, put your trust in something off the shelf that works and that has been tested by lots and lots of people already.

Answer 8

As an alternative to ActiveMQ (mentioned above) there is open-source RabbitMQ. From what they say it integrates nicely with ASP.NET and WCF.

http://www.rabbitmq.com/