Python twisted Reactor class

Question

What is the significance of the decorators

 @reactor.callWhenRunning,
 @results_deferred.addCallback
 @results_deferred.addErrback.

Also what are deferred strings, for example in the

 twisted.internet.utils.getProcessOutput()

returns a deferred string what exactly is happening here?

I am new to twisted hence this might be a very simple question but reading twisted documentation did not help me much

Answer 1

A deferred is a like a promise to return output in the future. You really should read the documentation on Deferreds here and here. Also, you should read up on Python decorators in general. One introduction is here.

More specifically, what is happening is that when you call getProcessOutput(), the result is not quite ready. It might be ready in an instant or in an hour. But you probably don't care: whenever it is ready, you probably want to take the output and pass it to a function. So instead of returning the output (which is not going to be ready right away), getProcessOutput returns a deferred object. When the output is finally ready, the deferred object will notice and call whatever processing function you supply, passing along the actual process output data. You really should read up on deferreds though.

Answer 2

I am not sure about python , but this looks a like Active object pattern, and Futures. Futures is going to be standard in next c++ version. If you read through Active object and Futures you will get an idea

Answer 3

In the normal programming practice you'd do

db = Database.connect()
result = db.getResult()
processResult(result)

Now depending on your Database and network, these 3 statements can take anywhere from a millisecond to a few seconds.

We've all been programming this way for decades now, and for the most part we're fine with 'waiting'.

But there comes a time when your program cant just wait for results. You'd start to think, gee I could do a lot of other things while I wait for the result. Maybe print an output, or process a function, or just quickly check the socket etc.

Enter Twisted and Deferred.

Instead of waiting for result, in Twisted when invoked the special methods you'll get a Deferred. You'll add a callback function to this deferred which means, call this function when you have the result/answer.

deferredResult = db.nonBlockingGetResult()
deferredResult.addCallback(processOutput)

As soon as the first statement is executed, it returns the 'something' back. And that something is Deferred. There's no blocking there, there no waiting. And to this Deferred you add the callback processOutput which is called when deferred is 'fired' - ie result is ready.

HTH

Answer 4

Hi can you please tell me how use different functions in different thread using thread pool in twisted...say

I have a list of ids x=[1,2,3,4] where 1,2,...etc are ids(I got from data base and each one contains python script in some where disk).

what I want to do is

scanning of x traverse on list and run every script in different thread until they completed