Posix AIO Bad/Broken?

Question

I'm working on a TFTP implementation that is transitioning away from a convoluted multi-threaded implementation to a single-thread/single-process implementation which uses a state machine to track the state of the sessions connected. TFTP is simple enough, and the number of concurrent sessions are small enough that the there really is no impact to the software other than massive code-size & complexity savings.

Of course, I can't just block on a single session when others are connected. To remedy this, my first thought was POSIX AIO, though after some research I read that it's

• Poorly Documented, and not complete
• Only works on Disk I/O and does not support sockets, or works on sockets but only for read/writes - Not for listening.

An example is contained in this link (http://davmac.org/davpage/linux/async-io.html), though I found others as well. Some additional perspective was given in a prior stackoverflow post (http://stackoverflow.com/questions/87892/what-is-the-status-of-posix-asynchronous-i-o-aio) from '08.

For a C developer, is AIO still as broken as people claim? Do people really not use AIO, and stick primarily to poll/select or finite size thread pools?

Answer 1

You might consider Boost.Asio for a cross-platform asynchronous socket library. It has excellent examples and is documented extensively.

Answer 2

I can't answer your question about POSIX AIO, but I've used libev for events. Small, fast, simple. Makes a good wrapper for IO in place of poll/select.

Answer 3

Poorly documented is certainly the case.

Most people do stick with poll() / select(), simply because these are well-understood, well-tested, well-documented and well-supported. If I were you I would use select() unless you have a compelling reason not to.

Answer 4

The issues with aio depend on the platform, so a big part of your decision is what platform you are targeting. Quality varies widely and in some cases it is implemented in terms of poll/select type calls.

People do tend to use poll/select or similar interfaces like kevent/kqueue or epoll for this kind of thing on Unix platforms.

There are problems with the aio interface, and additions like aio_waitcomplete() and the integration of aio and kqueues makes a difference.

Lots of threads for dealing with lots of I/O is not a good approach.

Answer 5

for disk, why do you have to have AIO instead of just buffered read/write, unless you want to 1) use your own caching 2) control the impact on dirty pages or 3) use IO priorities?

Because if your goal is only code refactoring, then you probably go through cache in the current version. Changing from buffered IO to direct IO is a huge change.

For example, on ext3/SSD/noop with 1,5G of RAM, just 3 threads doing streamed writes of 300Mb starve small writes and reads. Switching the offenders to direct IO fixes that, but the writes now take forever.