lowest latency, least overhead app server?

Question

I'm designing an application which will have a network interface for feeding out large numbers of very small metadata requests. The application code itself is very fast, basically looking up data cached in memory and sending it to the client.

What's the absolute lowest latency I can get for a network application server running on a linux box? This will be an internal app running on gigE with no authentication. Any language/framework considered, with a preference for C, C++, or Python. Likewise for protocol, although HTTP would be nice.

Answer 1

For a small, fast HTTP server, you could take a look at gatling. Key features:

• free, opensource
• small, fast, scalable, comes with anti-DOS features
• speaks HTTP, HTTPS/TLS, FTP, IPV6, understands SMB
• supports SCGI and FastCGI and can thus be used to run ELF binaries or scripts
• transparent content negotiation (serves gz if the client supports it)
• makes use of OS-specific shortcuts to improve performance
• several benchmarks included

Answer 2

If it suits your needs, consider C or C++ and zmq

Though, for an app like this the language wouldn't be the biggest factor from a performance view. If you need to support a large number of clients, the programming model would be the deciding factor, thread-per-client(slower), or async/non-blocking(faster).

Answer 3

Facebook recently got their customized memcached to handle 200,000 requests per second with 173 microseconds latency.

You could read the source code to see how they did it. One of their biggest changes was dropping TCP and using UDP instead.