Python - How do I pass a string into subprocess.Popen (using the stdin argument)?

Question

If I do the following:

import subprocess
from cStringIO import StringIO
subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=StringIO('one\ntwo\nthree\nfour\nfive\nsix\n')).communicate()[0]

I get:

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 533, in __init__
    (p2cread, p2cwrite,
  File "/build/toolchain/mac32/python-2.4.3/lib/python2.4/subprocess.py", line 830, in _get_handles
    p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'

Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen. How do I work around this?

Answer 1

I figured out this workaround:

>>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
>>> p.stdin.write('one\ntwo\nthree\nfour\nfive\nsix\n')
>>> p.communicate()[0]
'four\nfive\n'
>>> p.stdin.close()

Is there a better one?

Answer 2

"Apparently a cStringIO.StringIO object doesn't quack close enough to a file duck to suit subprocess.Popen"

:-)

I'm afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.

Answer 3

Popen.communicate() documentation:

Note that if you want to send data to the process’s stdin, you need to create the Popen object with stdin=PIPE. Similarly, to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE too.

Replacing os.popen*

    pipe = os.popen(cmd, 'w', bufsize)
    # ==>
    pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin

Warning Use communicate() rather than stdin.write(), stdout.read() or stderr.read() to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

So your example could be written as follows:

from subprocess import Popen, PIPE, STDOUT

p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)

grep_stdout = p.communicate(input='one\ntwo\nthree\nfour\nfive\nsix\n')[0]
print(grep_stdout)
# -> four
# -> five
# ->

Answer 4

Recommended reading: Doug Hellmann's Python Module of the Week blog post on subprocess.

Answer 5

p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)

p.stdin.write('one\n')

time.sleep(0.5)

p.stdin.write('two\n')

time.sleep(0.5)

p.stdin.write('three\n')

time.sleep(0.5)

testresult = p.communicate()[0]

time.sleep(0.5)

print(testresult)