How to check for EOF when using pipe file descriptors in Python?

Question

I have 2 threads, Thread A and Thread B.

Thread A takes data from either sys.stdin or from the read end of a pipe (using os.pipe()). Whatever data gets read (from either sys.stdin or from the read end of the pipe) gets sent out on a TCP socket. Thread A uses select.select() to determine whether it should read from sys.stdin or from the read end of the pipe, and select.select() is using a 1 second timeout to periodically check and make sure that the socket connection is still there.

Thread B holds the write end of the pipe, and will typically just write 1 really long string to the pipe, and then close the write end.

How can Thread A check for an EOF on the pipe? Basically, I want Thread A to close the pipe when it's no longer being used. But, I can't risk blocking on read() from the pipe, because then data that is available on sys.stdin would be unnecessarily blocked.

Any help would be greatly appreciated. Thank you!

Answer 1

The only way to check for EOF on the pipe is to read from it. In fact, when you close the write-end of the pipe, select.select() will say the socket is ready for reading, even if there's no actual data to be read. When you read from the read-end, the empty string ('') will indicate EOF. This won't block your read even if the pipe is in blocking mode.

That said, it's probably a good idea to not use blocking reads altogether. You can set the pipes non-blocking using fcntl.fcntl(). And you can just avoid using two threads by sending directly to the socket from thread B, forgoing the seemingly useless gateway thread. If you have multiple sockets you need to manage, or even multiple producer threads, Twisted is a good way of managing the sockets and other communication.