Python (Twisted) - reading from fifo and sending read data to multiple protocols

Question

Hi,

Im trying to write some kind of multi protocol bot (jabber/irc) that would read messages from fifo file (one liners mostly) and then send them to irc channel and jabber contacts. So far, I managed to create two factories to connect to jabber and irc, and they seem to be working.

However, I've problem with reading the fifo file - I have no idea how to read it in a loop (open file, read line, close file, jump to open file and so on) outside of reactor loop to get the data I need to send, and then get that data to reactor loop for sending in both protocols. I've been looking for information on how to do it in best way, but Im totally lost in the dark. Any suggestion/help would be highly appreciated.

Thanks in advance!

Answer 1

The fifo is the problem. Read from a socket instead. This will fit info the Twisted event-driven model much better. Trying to do things outside the control of the reactor is usually the wrong approach.

---- update based on feedback that the fifo is an external constraint, not avoidable ----

OK, the central issue is that you can not write code in the main (and only) thread of your Twisted app that makes blocking read calls to a fifo. That will cause the whole app to stall if there is nothing to read. So you're either looking at reading the fifo asynchronously, creating a separate thread to read it, or splitting the app in two.

The last option is the simplest - modify the Twisted app so that it listens on a socket and write a separate little "forwarder" app that runs in a simple loop, reading the fifo and writing everything it hears to the socket.

Answer 2

You can read/write on a file descriptor without blocking the reactor as you do with sockets, by the way doesn't sockets use file descriptors?

In your case create a class that implements twisted.internet.interfaces.IReadDescriptor and add to reactor using twisted.internet.interfaces.IReactorFDSet.addReader. For an example of IReadDescriptor implementation look at twisted.internet.tcp.Connection.

I cannot be more specific because i never did by my self, but i hope this could be a start point.