Python subprocess + mencoder not working, same command works in terminal

Question

I am having a problem using mencoder (SVN-r30531-4.2.1) through a python (2.6.1) subprocess. I am trying to join two mp4 files which are exactly the same size, codec, etc. Both have no audio. The code I am using to test is:

import subprocess

mp4merge = [ "mencoder", "in1.mp4", "in2.mp4", "-ovc", "copy", "-oac", "copy", "-of", "lavf", "-o", "out.mp4" ]

try:

    pMerge = subprocess.Popen(mp4merge, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

    while pMerge.poll() == None:

        for l in pMerge.stderr.readlines():
            print l

    if pMerge.poll() is not None:

        print "Complete"

except subprocess.CalledProcessError:
    print "fail"

And it doesn't work, it just hangs indefinitely. However, when I run the exact same command through Terminal (OS X 10.6.4) it works. The command is:

mencoder in1.mp4 in2.mp4 -ovc copy -oac copy -of lavf -o out.mp4

You can download the videos from here. I am quite confident the videos aren't the probelm because of the fact that it works from Terminal.

Answer 1

The problem here is that pMerge.stderr.readlines() blocks forever until the process is over. It reads all lines before continuing.

Since mencoder writes a lot to the stdout, the stdout buffer is filled and mencoder is waiting for it to empty before it can continue. So the process never ends.

Here's a way to do the same, that won't hang:

pMerge = subprocess.Popen(mp4merge, stdout=subprocess.PIPE,
    stderr=subprocess.PIPE)
stdout, stderr = pMerge.communicate()
print stdout
print stderr

Another option that allows you to read the output line-by-line is to redirect stderr to stdout, and then read only stdout, (don't use readlines() since it blocks until all lines are read):

pMerge = subprocess.Popen(mp4merge, stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT)
for line in pMerge.stdout:
    print line,

This redirects stderr to stdout so your buffer won't fill.