Book about low-latency architectures

Question

Hi, do you know any books about low-latency systems architectures (caches, messaging, data storing etc)? I'm working primary with Java and with financial applications, but it is not essential. Thanks.

Answer 1

I'm not aware of any specifically on that subject. We do a lot of ultra-low latency work at my workplace. The general guidelines are:

• Create as little garbage as possible (GC accounts for lots of latency spikes). We do this to the extent of having a "fixed point" math library that uses scaled longs instead of using, for example, BigDecimal.
• Avoid JNI calls if possible. JNI calls are very expensive.
• Avoid thread boundaries. Context switches are very expensive. We get an enormous throughput using a single threaded architecture.
• Try and write code that is CPU/architecture friendly. Understand what's going on in the cache and write code that's cache friendly.
• Try and use lock-free data structures in the places where you do have multiple threads. We use a lot of CAS-style operations. You'll burn some CPU, but you'll almost always spend less time doing that than doing locks/context switches.
• Try and have more cores than threads. Obviously this is simpler if you go for a single-threaded architecture. Again, this helps the kernel (I recommend Linux for this kind of thing, BTW) keep threads pinned to cores and avoids cache flushes, etc. associated with moving a thread.
• Try and avoid virtualisation. I know it's all the rage right now, but all virtualisation adds huge latency overheads.