Non-blocking file access with Twisted

Question

I'm trying to figure out if there is a defacto pattern for file access using twisted. Lots of examples I've looked at (twisted.python.log, twisted.persisted.dirdbm, twisted.web.static) actually don't seem to worry about blocking for file access.

It seems like there should be some obvious interface, probably inheriting from abstract.FileDescriptor, that all file access should be going through it as a producer/consumer.

Have I missed something or is it just that the primary use for twisted in asynchronous programming is for networking and it hasn't really been worked out for other file descriptor operations, not worrying about the purity of non-blocking IO ?

Answer 1

I'm not sure what you want to achieve. When you do logging, then Python will make sure (by the global interpreter log) that log messages from several threads go into the file one after the other.

If you're concerned about blocking IO, then the OS adds default buffers for your files (usually 4KB), and you can pass a buffer size in the open() call.

If you're concerned about something else, then please clarify your question.

Answer 2

I think you are looking for the fdesc module. For more information on non-blocking I/O in Python, you can also watch this video.

Answer 3

There is an open ticket for this in Twisted - #3983.