How do I do a non-blocking IPC read on Windows?

Question

I have a Perl script that uses an external tool (cleartool) to gather information about a list of files. I want to use IPC to avoid spawning a new process for each file:

use IPC::Open2;
my ($cin, $cout);
my $child = open2($cout, $cin, 'cleartool');

Commands that return single-lines work well. e.g.

print $cin "describe -short $file\n";
my $description = <$cout>;

Commands that return multiple lines have me at a dead end for how to consume the entire response without getting hung up by a blocking read:

print $cin "lshistory $file\n";
# read and process $cout...

I've tried to set the filehandle for non-blocking reads via fcntl:

use Fcntl;
my $flags = '';
fcntl($cout, F_GETFL, $flags);
$flags |= O_NONBLOCK;
fcntl($cout, F_SETFL, $flags);

but Fcntl dies with the message "Your vendor has not defined Fcntl macro F_GETFL."

I've tried using IO::Handle to set $cout->blocking(0) but that fails (it returns undef and sets $! to "Unknown error").

I've tried to use select to determine if there's data available before attempting to read:

my $rfd = '';
vec($rfd, fileno($cout), 1) = 1;
while (select($rfd, undef, undef, 0) >= 0) {
    my $n = read($cout, $buffer, 1024);
    print "Read $n bytes\n";
    # do something with $buffer...
}

but that hangs without ever reading anything. Does anyone know how to make this work (on Windows)?

Answer 1

select only works on sockets in Windows. It looks like IPC::OpenX uses normal filehandles, so you won't be able to use select with the handles it creates.

If you don't need the timeout/detection of activity that select provides, you can set the handles to be non-blocking and just read or write as per normal.

If you need more nuanced control, IPC::Run may work well for you.

You could also look at creating a socketpair and use those handles with your child processes. Newer perls (5.8 and up) support socketpair emulation on Windows using TCP sockets.

If you try to clone STDOUT and STDERR for a program that runs without a console (ie it is started with wperl, instead of perl), you won't be able to get data through STDIO.

In practice this has been a huge pain for me on several projects. What I found worked best was to write the child process to connect to the parent server via TCP. If you don't control the child processes, look at IPC::Run or socketpair.

Answer 2

Non-blocking IO is hard on Windows. In this case, you could send the output of cleartool to a regular file, and to use seek to reset the eof flag on the file each time out read from the file:

my($cin, $cout);
my $somefile = "some/file";
open($cin, "| cleartool > $somefile");
open($cout, '<', $somefile);

...

print $cin "$command_for_cleartool\n";
# if necessary, wait until cleartool finishes with new output
seek $cout, 0, 1;     # clear eof condition from previous read
my @cleartool_output = <$cout>;  # capture recent output
... process output ...

Though this will probably not work that well if cleartool buffers its output.

Answer 3

Another kludge is to use sysread with a large or unlikely buffer size.

    print $cin "$command_for_cleartool\n";
    my ($description, $maxlen, $buffer) = ("", 65336);
    while (my $n = sysread $cout, $buffer, $maxlen) {
        $description .= $buffer;
        last if $n < $maxlen;
    }
    ... do something with $description ...

sysread will hang if there are exactly 0 bytes of input waiting to be read. So the code above will hang if cleartool produces exactly some multiple of 65336 bytes. If you know a good upper bound on the size of output from the program, you can use that value for $maxlen above. Otherwise, you could pick a large and unlikely number and pray ...