Python subprocess "object has no attribute 'fileno'" error

Question

This code generates "AttributeError: 'Popen' object has no attribute 'fileno'" when run with Python 2.5.1

Code:

def get_blame(filename): 
    proc = []
    proc.append(Popen(['svn', 'blame', shellquote(filename)], stdout=PIPE))
    proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1]), stdout=PIPE)
    proc.append(Popen(['tr', r"'\040'", r"';'"], stdin=proc[-1]), stdout=PIPE)
    proc.append(Popen(['cut', r"-d", r"\;", '-f', '3'], stdin=proc[-1]), stdout=PIPE)
    return proc[-1].stdout.read()

Stack:

function walk_folder in blame.py at line 55
print_file(os.path.join(os.getcwd(), filename), path)

function print_file in blame.py at line 34
users = get_blame(filename)

function get_blame in blame.py at line 20
proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1]), stdout=PIPE)

function __init__ in subprocess.py at line 533
(p2cread, p2cwrite,

function _get_handles in subprocess.py at line 830
p2cread = stdin.fileno()

This code should be working the python docs describe this usage.

Answer 1

looks like syntax error. except first append the rest are erroneous (review brackets).

Answer 2

You want the stdout of the process, so replace your stdin=proc[-1] with stdin=proc[-1].stdout

Also, you need to move your paren, it should come after the stdout argument.

 proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1]), stdout=PIPE)

should be:

 proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1].stdout, stdout=PIPE))

Fix your other append calls in the same way.

Answer 3

Three things

First, your ()'s are wrong.

Second, the result of subprocess.Popen() is a process object, not a file.

proc = []
proc.append(Popen(['svn', 'blame', shellquote(filename)], stdout=PIPE))
proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1]), stdout=PIPE)

The value of proc[-1] isn't the file, it's the process that contains the file.

proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1].stdout, stdout=PIPE))

Third, don't do all that tr and cut junk in the shell, few things could be slower. Write the tr and cut processing in Python -- it's faster and simpler.

Answer 4

There's a few weird things in the script,

• Why are you storing each process in a list? Wouldn't it be much more readable to simply use variables? Removing all the .append()s reveals an syntax error, several times you have passed stdout=PIPE to the append arguments, instead of Popen:

  proc.append(Popen(...), stdout=PIPE)
  

  So a straight-rewrite (still with errors I'll mention in a second) would become..

  def get_blame(filename): 
      blame = Popen(['svn', 'blame', shellquote(filename)], stdout=PIPE)
      tr1 = Popen(['tr', '-s', r"'\040'"], stdin=blame, stdout=PIPE)
      tr2 = Popen(['tr', r"'\040'", r"';'"], stdin=tr1), stdout=PIPE)
      cut = Popen(['cut', r"-d", r"\;", '-f', '3'], stdin=tr2, stdout=PIPE)
      return cut.stdout.read()
  

• On each subsequent command, you have passed the Popen object, not that processes stdout. From the "Replacing shell pipeline" section of the subprocess docs, you do..

  p1 = Popen(["dmesg"], stdout=PIPE)
  p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
  

  ..whereas you were doing the equivalent of stdin=p1.

  The tr1 = (in the above rewritten code) line would become..

  tr1 = Popen(['tr', '-s', r"'\040'"], stdin=blame.stdout, stdout=PIPE)
  

• You do not need to escape commands/arguments with subprocess, as subprocess does not run the command in any shell (unless you specify shell=True). See the Securitysection of the subprocess docs.

  Instead of..

  proc.append(Popen(['svn', 'blame', shellquote(filename)], stdout=PIPE))
  

  ..you can safely do..

  Popen(['svn', 'blame', filename], stdout=PIPE)
  

• As S.Lott suggested, don't use subprocess to do text-manipulations easier done in Python (the tr/cut commands). For one, tr/cut etc aren't hugely portable (different versions have different arguments), also they are quite hard to read (I've no idea what the tr's and cut are doing)

  If I were to rewrite the command, I would probably do something like..

  def get_blame(filename): 
      blame = Popen(['svn', 'blame', filename], stdout=PIPE)
      output = blame.communicate()[0] # preferred to blame.stdout.read()
      # process commands output:
      ret = []
      for line in output.split("\n"):
          split_line = line.strip().split(" ")
          if len(split_line) > 2:
              rev = split_line[0]
              author = split_line[1]
              line = " ".join(split_line[2:])
  
  
          ret.append({'rev':rev, 'author':author, 'line':line})
  
  
  return ret
  
  
  

Answer 5

Like S.Lott said, processing the text in Python is better.

But if you want to use the cmdline utilities, you can keep it readable by using shell=True:

cmdline = r"svn blame %s | tr -s '\040' | tr '\040' ';' | cut -d \; -f 3" % shellquote(filename)
return Popen(cmdline, shell=True, stdout=PIPE).communicate()[0]