Deliver multicast to several different geo-locations

Question

I need to use one logical PGM based multicast address in application while enable such application "seamlessly" running across several different geo-locations (i.e. think US/Europe/Australia).

Application is quite throughput (several million biz. messages a day) and latency demanding whith a lot of small but very frequently send messages. Classical Atom pub will not work here due some external limits of latencies.

I have come up with several options to connect those datacenters but can’t find the best one. Options which I have considered are: 1) Forward multicast messages via VPN’s (can VPN handle such big load). 2) Translate all multicast messages to “wrapper messages” and forward them via AMQP. 3) Write specialized in-house gate which tunnels multicast messages via TCP to other two locations. 4) Any other solution

I would prefer option 1 as it does not need additional code writes from devs. but I’m afraid it will not be reliable connection.

Are there any rules to apply for such connectivity?

What the best network configuration with regard to the geographical configuration is for above constrains.

Answer 1

Hi Libor,

Just wanted to say hello :)

As for the topic, we have not much experience with multicasting over WAN, however, my feeling is that PGM + WAN + high volume of data would lead to retransmission storms. VPN won't make this problem disappear as all the Australian receivers would, when confronted with missing packets, send NACKS to Europe etc.

PGM specification does allow for tree structure of nodes for message delivery, so in theory you could place a single node on the receiving side that would in its turn re-multicast the data locally. However, I am not sure whether this kind of functionality is available with MS implementation of PGM. Optionally, you can place a Cisco router with PGM support on the receiving side that would handle this for you.

In any case, my preference would be to convert the data to TCP stream, pass it over the WAN and then convert it back to PGM on the other side. Some code has to be written, but no nasty surprises are to be expected.

Martin S.

Answer 2

Hi Libor, at CohesiveFT we ran into a very similar problem when we designed our "VPN-Cubed" product for connecting multiple clouds up to servers behind our own firewall, in one VPN. We wanted to be able to run apps that talked to each other using multicast, but for example Amazon EC2 does not support multicast for reasons that should be fairly obvious if you consider the potential for network storms across a whole data center. We also wanted to route traffic across a wide area federation of nodes using the internet.

Without going into too much detail, the solution involved combining tunneling with standard routing protocols like BGP, and open technologies for VPNs. We used RabbitMQ AMQP to deliver messages in a pubsub style without needing physical multicast. This means you can fake multicast over wide area subnets, even across domains and firewalls, provided you are in the VPN-Cubed safe harbour. It works because it is a 'network overlay' as described in technical note here: http://blog.elasticserver.com/2008/12/vpn-cubed-technical-overview.html

I don't intend to actually offer you a specific solution, but I do hope this answer gives you confidence to try some of these approaches.

Cheers, alexis