Stdout captured from pipe in Python is truncated

Question

I want to capture the ouput of dpkg --list | grep linux-image in Python 2.6.5 on Ubuntu 10.04.

from subprocess import Popen 
from subprocess import PIPE

p1 = Popen(["dpkg", "--list"], stdout=PIPE)
p2 = Popen(["grep", "linux-image"], stdin=p1.stdout, stdout=PIPE)
stdout = p2.communicate()[0]

The content of stdout is:

>>> print stdout
rc  linux-image-2. 2.6.31-14.48   Linux kernel image for version 2.6.31 on x86
ii  linux-image-2. 2.6.32-22.36   Linux kernel image for version 2.6.32 on x86
ii  linux-image-2. 2.6.32-23.37   Linux kernel image for version 2.6.32 on x86
ii  linux-image-2. 2.6.32-24.43   Linux kernel image for version 2.6.32 on x86
ii  linux-image-2. 2.6.32-25.44   Linux kernel image for version 2.6.32 on x86
ii  linux-image-ge 2.6.32.25.27   Generic Linux kernel image

However, this is not the same as running dpkg --list | grep linux-image in a shell:

cschol@blabla:~$ dpkg --list | grep linux-image
rc  linux-image-2.6.31-14-generic         2.6.31-14.48                                    Linux kernel image for version 2.6.31 on x86
ii  linux-image-2.6.32-22-generic         2.6.32-22.36                                    Linux kernel image for version 2.6.32 on x86
ii  linux-image-2.6.32-23-generic         2.6.32-23.37                                    Linux kernel image for version 2.6.32 on x86
ii  linux-image-2.6.32-24-generic         2.6.32-24.43                                    Linux kernel image for version 2.6.32 on x86
ii  linux-image-2.6.32-25-generic         2.6.32-25.44                                    Linux kernel image for version 2.6.32 on x86
ii  linux-image-generic                   2.6.32.25.27                                    Generic Linux kernel image

Looking at the first line, one can see that the output in Python is truncated:

rc  linux-image-2. 2.6.31-14.48

instead of

rc  linux-image-2.6.31-14-generic         2.6.31-14.48

Why does it do that and is there a way to get exactly the same output in Python?

Answer 1

import subprocess
p1 = subprocess.Popen(["dpkg", "--list"], stdout=subprocess.PIPE, env={'LANG':'C'})
p2 = subprocess.Popen(["grep", "linux-image"], stdin=p1.stdout, stdout=subprocess.PIPE)
out,err=p2.communicate()
print(out)

The dpkg command's output depends on the value of the LANG environment variable. Setting LANG=C in subprocess.Popen will make dpkg's output more like what you see from the terminal.

Answer 2

There is no need to use grep !

import subprocess
p1 = subprocess.Popen(["dpkg", "--list"], stdout=subprocess.PIPE, env={'LANG':'C'})
out,err=p1.communicate()
for o in  out.split("\n"):
    if "linux-image" in o:
        print o