Synchronous vs asynchronous programming

Question

When designing a server, we are considering two approaches:

1. A asynchronous (select based) approach so the backend rpc can be parallelized in a single thread.

2. A synchronous approach where each backend rpc is processed in another thread from the thread pool.

There are trade offs: 1 has better performance and 2 has less code complexity. Does 1 really matter now with machine going multi-core and 64 bits?

Answer 1

I think it should be opposite, async select() approach makes the code simpler because there are no thread sync involved.

All you need to do is to pass a list of socket handles into the select() function. It will block and only return when something happened in one or more of the socket handle. As oppose to using a thread pool where you will have to have a job dispatch queue where all threads in the thread pool grab jobs from.

The down side of async select() approach is that it won't scale with the number of processor-core because everything was executed in one thread.

Answer 2

I would recommend to read these books on ACE

• C++ Network Programming: Mastering Complexity Using ACE and Patterns
• C++ Network Programming: Systematic Reuse with ACE and Frameworks

to get ideas about patterns allowing you to create an efficient server.

Answer 3

Without knowing more detail about your server requirements, it is hard to tell which approach may give you better performance and/or reduce code complexity.

The funny thing is, performance and simplicity usually go together in real life server design (but coming up with the simplistic design is hard). select/poll by itself does not automatically give you better performance. And thread pool definitely does not reduce your code complexity.

To answer your question: 1 still matter. Async design does not scale well multi-proc machine but that's only true if your server run on one process. What if your server is designed to spawn multiple process on a multi-proc machine? Different process can be run under different user account and have different security setting. How about virtual machines? One thread per process doesn't mean you can't take advantage of multi-core machines.

However, i really can't remember the last time i saw a real server design based on async select(). Maybe in text books back when i was in college but not any real production system.

Answer 4

This is a relevant read (its a sort of minority report):

http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren.pdf

This is also a good resource:

http://www.kegel.com/c10k.html

Keep in mind you can combine the two approaches, in a way, to maximize utility of multi-core architectures. Assuming that your server ("Krumpets here!" ;) is partionable -- meaning you can run multiple instances of it on the same machine -- write the server using select, and then run one instance of the server per cpu core.