Is Twisted an httplib2/socket replacement?

Question

Many python libraries, even recently written ones, use httplib2 or the socket interface to perform networking tasks.

Those are obviously easier to code on than Twisted due to their blocking nature, but I think this is a drawback when integrating them with other code, especially GUI one. If you want scalability, concurrency or GUI integration while avoiding multithreading, Twisted is then a natural choice.

So I would be interested in opinions in those matters:

1. Should new networking code (with the exception of small command line tools) be written with Twisted?
2. Would you mix Twisted, http2lib or socket code in the same project?
3. Is Twisted pythonic for most libraries (it is more complex than alternatives, introduce a dependency to a non-standard package...)?

Edit: please let me phrase this in another way. Do you feel writing new library code with Twisted may add a barrier to its adoption? Twisted has obvious benefits (especially portability and scalability as stated by gimel), but the fact that it is not a core python library may be considered by some as a drawback.

Answer 1

See asychronous-programming-in-python-twisted, you'll have to decide if depending on a non-standard (external) library fits your needs. Note the answer by @Glyph, he is the founder of the Twisted project, and can authoritatively answer any Twisted related question.

At the core of libraries like Twisted, the function in the main loop is not sleep, but an operating system call like select() or poll(), as exposed by a module like the Python select module. I say "like" select, because this is an API that varies a lot between platforms, and almost every GUI toolkit has its own version. Twisted currently provides an abstract interface to 14 different variations on this theme. The common thing that such an API provides is provide a way to say "Here are a list of events that I'm waiting for. Go to sleep until one of them happens, then wake up and tell me which one of them it was."

Answer 2

1. Should new networking code (with the exception of small command line tools) be written with Twisted?
   • Maybe. It really depends. Sometimes its just easy enough to wrap the blocking calls in their own thread. Twisted is good for large scale network code.
2. Would you mix Twisted, http2lib or socket code in the same project?
   • Sure. But just remember that Twisted is single threaded, and that any blocking call in Twisted will block the entire engine.
3. Is Twisted pythonic for most libraries (it is more complex than alternatives, introduce a dependency to a non-standard package...)?
   • There are many Twisted zealots that will say it belongs in the Python standard library. But many people can implement decent networking code with asyncore/asynchat.