views:

1723

answers:

1

There are pages scattered around the web that describe POSIX AIO facilities in varying amounts of detail. None of them are terribly recent. It's not clear what, exactly, they're describing. For example, the "official" (?) web site for Linux kernel asynchronous I/O support here says that sockets don't work, but the "aio.h" manual pages on my Ubuntu 8.04.1 workstation all seem to imply that it works for arbitrary file descriptors. Then there's another project that seems to work at the library layer with even less documentation.

I'd like to know:

  • What is the purpose of POSIX AIO? Given that the most obvious example of an implementation I can find says it doesn't support sockets, the whole thing seems weird to me. Is it just for async disk I/O? If so, why the hyper-general API? If not, why is disk I/O the first thing that got attacked?
  • Where are there example complete POSIX AIO programs that I can look at?
  • Does anyone actually use it, for real?
  • What platforms support POSIX AIO? What parts of it do they support? Does anyone really support the implied "Any I/O to any FD" that <aio.h> seems to promise?

The other multiplexing mechanisms available to me are perfectly good, but the random fragments of information floating around out there have made me curious.

+9  A: 

Network I/O is not a priority for AIO because everyone writing POSIX network servers uses an event based, non-blocking approach. The old-style Java "billions of blocking threads" approach sucks horribly.

Disk write I/O is already buffered and disk read I/O can be prefetched into buffer using functions like posix_fadvise. That leaves direct, unbuffered disk I/O as the only useful purpose for AIO.

Direct, unbuffered I/O is only really useful for transactional databases, and those tend to write their own threads or processes to manage their disk I/O.

So, at the end that leaves POSIX AIO in the position of not serving any useful purpose. Don't use it.

Zan Lynx
What about reading/writing from network (NFS, Samba) filesystems?
Alex B
well. I have several big dumb writers which, if I let them go to cache, will hit dirty_ratio at peaks, blocking everybody else. If I just use direct IO on them, it is way too slow. If I just had 1 thread I could manage on my own, but it'll be hard to support different IO priorities in 1 tread. AIO + CFQ wouls really seem a good combination, if AIO worked
n-alexander
I disagree. Disk I/O tends to be buffered but it can be blocking. When poll()ing a file FD it always reports that the FD is readable, even when it will block. This makes it impossible to perform non-blocking operations on disk files in an evented manner, unless one uses threads or AIO.
Hongli
@Hongli: The DB engines I have seen use a thread or process of their own. Do you have an example of an SQL engine that uses AIO?
Zan Lynx