poll() in Ruby?

Question

I am currently porting a self-written network application from C++ to Ruby. This network application often needs to manage around 10.000 sockets at the same time, that means it needs quick access to any sockets that have readable data available, incoming connections, etc.

I have already experienced while writing it in C++, that select() does not work for this case, because internally it uses 32 DWORDs (128 byte) to manage maximally 1024 sockets with bitmasks. Since I sometimes have to work with more than 10.000 sockets, this function did not suffice. Thus I had to switch to poll() which also made the code more elegant because I did not always have add and remove all the file-descriptors again.

As I can see from the Ruby documentation, Ruby offers IO.select(), which would basically be a wrapper for the C-API (as far as I know). Unfortunately it seems like there is no IO.poll(), which I would need for this particular application.

Does IO.select() have the same limitations as select() on WinSocks and Berkeley Sockets? If yes, is there a way to work around that?

Answer 1

IO::Reactor may do what you need. It has a poll method that looks similar to what you're describing.

Answer 2

The limitations on IO.select() and in fact the number of open connections you can have per process appear to be determined primarily by the underlying operating system support. Definitely no fixed 1024 socket limit.

For example, under WinXP, I max out at 69 socket opens (even before I get to select). I'm sure that is probably tunable, I just don't know how.

Under Linux, the limitation is the number of open files allowed. By default, the limit is usually 1024 (run ulimit -a to check).

However, you can easily change this e.g. ulimit -n 10000. I just ran a test and happily went well over 1024 active sockets created with TCPSocket.new, and using IO.select to test for ready data.

NB: there is a good example of IO.select usage in this GServer article.