Queue-Based Background Processing in ASP.NET MVC Web Application

Question

Hello,

How can I implement background processing queues in my ASP.NET MVC web app? While most data changes, updates etc. need to be visible immediately, there are other updates that don't need real time processing which I would like to hand off to a lower-priority background process which will take care of it at its own pace.

As an example, take StackOverflow's badge award system. Usually you may take a specific action that would award you a badge, but the actual 'award' happens later (typically between 10 minutes and a couple hours later). I assume this is done via a separate background process as it is not critical for SO's workings to award badges immediately when earned.

So, I'm trying to create some kind of queue system in which I could stuff tasks (say anything implementing ITask interface which will have a Process() method) which will eventually get executed by a separate process.

How would I go about implementing such a system? Ideas/Hint/Sample Code?

Thank you!

Answer 1

Windows Services and MSMQ to communicate with them (if you even need to).

-- Edit

To slightly expand.

You'll create several services, depending on what you want to do, and have them all run an endless thread of various sleeping levels, to do what you want. They will then update the database appropriately, and you will not have had to do anything on the client side.

You may wish to interact with them from an admin point of view, hence you may have an MSMQ that they listen to admin commands on. Depending on your implementation, you may need to restart them for some reason, or possibly just 'force' a running of whatever they wish to do.

So you'll use an MSMQ Private Queue to do it (System.Messaging namespace). One of the main things to note about MSMQ, is that message need to be < 4meg. So if you intend to send large object graphs, serialise to a file first and just send the filename.

MSMQ is quite beautiful. You can send based on a 'Correlation ID' if you need to, but, for some amusing reason, the correlation ID must be in the form of:

{guid}\1

Anything else does not work (at least in version 2.0 of the framework, the code may have changed).

-- Edit

Example, as requested:

using System.Messaging;

...


MessageQueue queue = new MessageQueue(".\\Private$\\yourqueue");
queue.Formatter = new BinaryMessageFormatter();

Message m = new Message();
m.Body = "your serialisable object or just plain string";

queue.Send(m);


// on the other side

MessageQueue queue = new MessageQueue(".\\Private$\\yourqueue");
queue.Formatter = new BinaryMessageFormatter();

Message m = queue.Receive();

string s = m.Body as string;

// s contains that string now