python subprocess block

Question

Hello.

I'm having a problem with the module subprocess. I'm running a script from python through:

subprocess.Popen('./run_pythia.sh',shell=True).communicate()

and sometimes it just blocks and it doesn't finish to execute the script. Before I was using .wait() instead of .communicate() but then because of this:

http://dcreager.net/2009/08/06/subprocess-communicate-drawbacks/

I changed to .communicate(). Nevertheless the problem continues.

Can anyone help me?

Answer 1

Is the script you execute, is run_pythia.sh guaranteed to finish executing? If not, you might not want to use blocking methods like communicate(). You might want to look into interacting with the .stdout, .stderr, and .stdin file handles of the returned process handle yourself (in a non-blocking manner).

Also, if you still want to use communicate(), you need to have had passed subprocess.PIPE object to Popen's constructor arguments.

Read the documentation on the module for more details.

Answer 2

What does the script do? In this case, it cannot be a black box, we need to know how exactly it works. (BTW, welcome to stackoverflow.com)

Answer 3

Hi. Thanks.

Ok I can show it. First the script compile a few files and then execute into a file:

run_pythia.sh:

#

!/bin/bash

PBS -l walltime=1:00:00

./compile.sh ./exec > resultado.txt

#

where the compile.sh is:

#

O=find ./ -name "*.o" | xargs

LOAD cernlib2005

module load libs/cernlib/2005

Compile and Link

FC=g77 CERNLIBPATH="-L/software/local/cernlib/2005/lib -lpacklib"

$FC call_pyth_mix.f analise_tt.f $O $CERNLIBPATH -o exec

#

Answer 4

maybe you can try to do a trace on it, i am new to python also...

import pdb; pdb.set_trace()