How to make a superfast webserver for "check for updates"?

Question

Which is the best approach for creating a fast response in case a client application asks webserver for "check for updates".

Skype for example takes about 1 second to answer. How to achieve the same?

Answer 1

I assume you are running one or more web servers and one or more back-end servers (with business logic).

One possible approach that I have seen: keep a change counter in webserver and when the back-end state changes, let the business logic notify all webservers with new change counter value.

Each web browser polls regularly the webserver for counter value and compares the value to the previous value. In case old_value != new_value, the web browser goes and asks the webserver for new content.

This allows the regular polling to be super-fast (1ms) and cheap. And only if something has really changed the browser will ask for more resource-expensive content generation.

The other option would be to use some asynchronous HTTP magic (cometd) but the approach outlined above is simpler, more understandable and easier to troubleshoot.

Answer 2

Use a large, distributed systems, depending on the number of your users. Put your web server(s) closer to clients, avoiding longer latencies. Use cluster and load balancing software to enhance performance. Use reverse proxies to cache data. But is is really important that a "check for updates" is that fast? You can also check in a background thread. I would improve performance for other tasks.

Answer 3

The simple approach is to just have a flat text or XML file on the server, containing the details of the most recent version. The client app fetches it via http GET, compares the version, and reacts accordingly. The http server is simply returning a small file, which is what http servers are designed to do. You should be able to handle hundreds of requests per second this way.