MSMQ - Fast producer/slow consumer

Question

Hello All

I have a problem with messaging (with MSMQ) which is a variation of fast producer/slow consumer. Is there a way to get outstanding unconsumed message count in a private MSMQ queue? I would like to use that to throttle the producer.

I would like to use a semaphore paradigm with MSMQ where the producer application will only send messages if the outstanding message count is less than a specified number.

Essentially, I would like to do something like the following

///Producer pseudo-code 
    public void SendMessage(Message message, int totalMessagesSentCounter)
    {
       if (totalMessagesSentCounter % 1000 == 0)
       {
         while (outgoingQueue.GetMessageCount() > X)  ///Is this possible?
         {
             Sleep(Y milliseconds);
         }
        }
       outgoingQueue.Send(Message);
       totalMessagesSentCounter++;
    }

My configuration : Win XP/2003 with MSMQ 3.0

Answer 1

I have not used MSMQ itself, but I do have a recipe that I've found useful.

Rather than a single queue, you have two queues - one in each direction.

The 'producer' consumes an item from the queue that comes from the consumer, and sends a new message out to the consumer. Each time the consumer consumes, it puts a new message in the queue to the producer.

So in this way, a 'token' provides feedback from the consumer to producer that throttles the producer to the speed of consumer.

A finesse if appropriate is for the messages to simply be returned, rather than fresh allocations each time. If its single-process, and message data is large and fixed size, this can itself be a powerful driver for the adoption of this design.

Answer 2

I had to use the COM library for MSMQ. I found the following link which helped.

http://blog.codebeside.org/archive/2008/08/27/counting-the-number-of-messages-in-a-message-queue-in.aspx