Python's subprocessing with pipes and large files.

Question

I'm trying to use python + ffmpeg + oggenc to convert any audiofile to ogg. The program works, almost. But for big files (i think > ~6mb) the ffmpeg process starts to sleep at pipe_wait. I don't know which pipe it waits for.

If I kill the ffmpeg process, the oggenc process continues and I get a resulting ogg-file with about ~2:40 of all the sound.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from subprocess import Popen, PIPE
from sys import argv

ffmpeg = Popen([
    "ffmpeg",
    "-i", argv[1],
    "-vcodec", "null",
    "-acodec", "pcm_s16le",
    "-ac", "2",
    "-ab", "44100",
    "-f", "wav",
    "-"
],stdout = PIPE,stderr = PIPE)

oggenc = Popen([
    "oggenc",
    "-", "--raw",
    "-q", "4",
    "-o", argv[2]
],stdin = ffmpeg.stdout,stderr = PIPE)

oggenc.communicate()
ffmpeg.communicate()

EDIT:

Thought I might add that this works perfectly:

#!/bin/bash

ffmpeg -i "$1" -vcodec null -acodec pcm_s16le -ac 2 -ab 44100 -f wav - | oggenc - --raw -q 4 -o "$2"

Answer 1

Its tricky to see who needs to pass the pipe, eer I mean you should use a debugger like the one in NetBeans to help gather more clues. Pipe's may not be the best approach, perhaps using a temporary file would simplify things.

Answer 2

What exactly do you do with the stderr channels of the two pipes?

Encoders/decoders typically produce lots of stderr output, as status updates; this output is piped to your process, and buffers will become full. Perhaps you should add some dummy ffmpeg.stderr.read() call before the (useless, I think) .communicate calls, or even better, drop the stderr=PIPE arguments completely.

UPDATE

For the ">/dev/null" equivalent, do the following:

nulfp= open(os.devnull, "w")
…
…= subprocess.Popen(…, stderr=nulfp.fileno())

Obviously, you can re-use the same nulfp for all stderrs you want to ignore.