Server Push vs Client Pull for Agent-Server Topology

Question

I need to create a system comprising of 2 components:

• A single server that process and stores data. It also periodically sends out updates to the agents

• Multiple agents that are installed at remote endpoints. These collect data in (often, but not always) long-running operations, and this data needs to get to the server

I'm using C# .NET, and ideally I want to use a standards compliant communications method (i.e. one that could theoritically work with Java too, as we may well also use Java agents in the future). Are there any alternatives to web services? What are my options?

The way I see it I have 3 options using web services, and have made the following observations:

• Client pull
  • No open port required at the agent, as it acts like a client
  • Would need to poll the server for updates
• Server push
  • Open port at the agent, as it acts like a server
  • Server must poll agents for results
• Hybrid
  • Open port at the agent, as it acts like both a client and a server
  • No polling; server pushes out updates when required, client sends results when they are available

The 'hybrid' (where agents are both client and server seems the obvious choice - but this application will typically be installed in enterprise and government environments, and I'm concerned they may have an issue with opening a port at the agent. Am I dwelling too much on this?

Are there any other pros and cons I've missed out?

Answer 1

If you use some kind of messaging server (JMS for Java, not sure for C#) then your messaging server is the only server that needs to open a port and you can have two way communication from your agent to the messaging server and from the server to the messaging server. This would allow you to accomplish the hybrid model without needing to open a port on the agent server.

Answer 2

Our friends at http://www.infrastructures.org swear by pull-based mechanisms: http://www.infrastructures.org/papers/bootstrap/bootstrap.html

A major reason why they prefer client-pull over server-push is that clients may be down, and clients must (in general) apply all the operations pushed by servers. If this criteria isn't important in your case, perhaps their conclusion won't be your conclusion, but I do think it is worth reading the "Push vs Pull" section of their paper to determine for yourself.

Answer 3

IMHO, I find your best option is the pull option.. that can satisfy your main system requirements as follow:

The first part: Data needs to get to the server, that's obviously can be done through invoking a web method that send that data as a parameter

2nd part:(Server periodically sends out updates to the agents): You can still do that that thru client (regular) pulls by some sort of a web service method that "asks" for the updates since its last pull (some sort of s time stamp to get the updates it missed)

The hybrid method seems a bit weird to me given that I think of an agent as a part of the system that probably might go "offline" quite often, what will the server then do if that failed? it's usually a tough question/decision, specially if you're not sure if this an intended "going offline" or a system/network failure.. etc

Answer 4

I would say that in this day and age you can seriously consider only pull technologies. The problem with push is that clients often are hidden behind Network Address Traversal devices (NAT) like wireless routers, broadband modems or company firewalls and they are, more often than not, unreachable from the server.

Making outbound connections ('phone-home'), specially on well known ports like HTTP/HTTPS can basically be assumed as 'possible' even under most constricted networks.