Please suggest .Net Job/Task Queue

Question

I need to build a network of services that will process data. Each service needs its own task queue. Preliminary I will need operations like QueueTask, CancelTask, StopTask, GetTaskStatus and GetTaskProgress.

I am looking for framework or fully built implementation of Task Queue.

At this time I know some options:

MSMQ - It OK for my needs, but require too much coding for support.

Castle.Sheduling - Scheduling solution. It provides pluggable Trigger, which I could implement to achive my needs. However still too many coding.

ServiceBUS - for example MassTransit. However, I am not sure how they support long queues and still require plumbing code.

An ideal solution will be framework that will provide hosting of task queue. That will be able to restart without loosing tasks. That will be accessible from network for example as SOAP end-point.

The question is actually what do you use, why? What do you suggest to use?

Answer 1

What's your question?

Answer 2

There's also ActiveMQ from the Apache organizations. It's got lots of good cross platform/language support and is free!

In the commercial sector there's also Tibco EMS which has good Java/C# bindings

Answer 3

Hmm, it sounds as though you are going to have to write some plumbing, especially to expose the task queues to the outside world via SOAP.

I'd suggest looking at:

• Udi Dahan's - NServiceBus
• Oren Eini's - Rhino Service Bus

Both are open source service bus implemenations on the .NET framework and MSMQ.

Answer 4

Maybe you're looking for something like CONTROL-M?

http://en.wikipedia.org/wiki/CONTROL-M

Answer 5

Well I am perhaps going through a similar process. Sorry, this isn't an answer as such, however maybe from a contextual perspective.

I have a situation where a web app will allow a user to upload images for a design.. the problem is I need to obviously handle the initial upload, but then queue the actual process of the images. For instance, I need to:

1. Create scaled thumbnails of the main images, along with some watermarking etc
2. Send the actual images over to an Amazon S3 storage unit.
3. Update some logs/notifications etc.

Again, I instantly thought MSMQ from my old days working in finance along with IBM-MQseries, and because I am using SQL 2008 R2, I notice it too has (probably for a while!) queuing mechanisms.

So I am sort of in the same boat trying to find the correct option!

Thanks for the replies so far.. will check out some of the solutions, but again, any other ideas would be grateful