In Parallel Python it has something in the submit function called a callback (documentation) however it doesn't seem to explain it too well. I've posted on their forum a couple days ago and I've not received a response. Would someone explain what a callback is and what it's used for? Thank you. ...
This is only the second question with the parallel-python tag. After looking through the documentation and googling for the subject, I've come here as it's where I've had the best luck with answers and suggestions. The following is the API (I think it's called) that submits all pertinent info to pp. def submit(self, func, args=(), ...
I'm writing a program that uses genetic techniques to evolve equations. I want to be able to submit the function 'mainfunc' to the Parallel Python 'submit' function. The function 'mainfunc' calls two or three methods defined in the Utility class. They instantiate other classes and call various methods. I think what I want is all of it in...
I have written a simple program using parallel python, and all works well. However, mainly for curiosities sake, I would like to know on which machine each task ran, and how long it took. Is there any way to programmatically get this information for the job that is returned? ...
I have had multiple problems trying to use PP. I am running python2.6 and pp 1.6.0 rc3. Using the following test code: import pp nodes=('mosura02','mosura03','mosura04','mosura05','mosura06', 'mosura09','mosura10','mosura11','mosura12') def pptester(): js=pp.Server(ppservers=nodes) tmp=[] for i in range(2...
I am using a simple program to send a function: import pp nodes=('mosura02','mosura03','mosura04','mosura05','mosura06', 'mosura09','mosura10','mosura11','mosura12') nodes=('miner:60001',) def pptester(): js=pp.Server(ppservers=nodes) js.set_ncpus(0) tmp=[] for i in range(200): tmp.append(js.submit(ppworktest,(...
I'm using parallel-python and start a new job server in a function. after the functions ends it still exists even though I didn't return it out of the function (I used weakref to test this). I guess there's still some references to this object somewhere. My two theories: It starts threads and it logs to root logger. My questions: can I...
I've inherited the maintenance of some scientific computing using Parallel Python on a cluster. With Parallel Python, jobs are submitted to a ppserver, which (in this case) talks to already-running ppserver processes on other computers, dishing tasks out to ppworkers processes. I'd like to use the standard library logging module to log...