How do you detect unrecoverable data loss when using MSMQ's multicast?

Question

When using MSMQ multicast, if a (multicast-enabled) queue receives some messages and then the machine experiences a temporary network connection loss then it will begin receiving messages again when the connection is re-established. If the network outage is short in duration then the queue will receive all of the messages (even those that were sent while the receiving machine was disconnected). However, if the duration is long enough (a couple minutes) then some messages will be lost... there will be a gap in the received messages.

This is expected behavior as MSMQ multicast provides reliable (but not guaranteed delivery). My question is whether MSMQ exposes information when there is data loss. If I understand correctly, PGM is already capable of detecting "unrecoverable data loss", but does MSMQ expose this information?

One thought is to include an integer with each message that is incremented on each send. This would allow the receiver to detect a gap in the messages (from an individual sender). However, I believe PGM already does this. I don't want to add this at the application level if PGM/MSMQ can already detect this.

Any suggestions?

Answer 1

I've never worked with MSMQ, so I'm not sure if there is a way to be notified of an unrecoverable loss of the underlying PGM socket. However, on a raw PGM socket, the socket would be disconnected if it's unable to recover missing packets (e.g. the sender doesn't have the data anymore in his resend cache). In that case, the Receive method of that socket would return with WSAECONNRESET. Are there connect/disconnect events in MSMQ that could maybe alert you of an underlying socket connection reset?

Personally, I always include some kind of application level header in my messages, with e.g. for your case a source (you want this in case a receiver, receives data from multiple senders) and a sequence number property and then check for gaps, paired with some out of band snapshot service that allows you to request missing data via a reliable protocol like TCP.