views:

691

answers:

3

Earlier, I asked this question:

How can I perform a ping or traceroute using native python?

However because python is not running as root it doens't have the ability to open the raw ICMP sockets needed to perform the ping/traceroute in native python.

This brings me back to using the system's ping/traceroute shell commands. This question has a couple examples using the subprocess module which seem to work well:

Ping a site in Python?

I still have one more requirement though: I need to be able to access the output as it is produced (eg. for a long running traceroute.)

The examples above all run the shell command and then only give you access to the complete output once the command has completed. Is there a way to access the command output as it is produced?

Edit: Based on Alex Martelli's answer, here's what worked:

import pexpect

child = pexpect.spawn('ping -c 5 www.google.com')

while 1:
        line = child.readline()
        if not line: break
        print line,
+5  A: 

pexpect is what I'd reach for, "by default", for any requirement such as yours -- there are other similar modules, but pexpect is almost invariably the richest, most stable, and most mature one. The one case where I'd bother looking for alternatives would be if I had to run correctly under Windows too (where ping and traceroute may have their own problems anyway) -- let us know if that's the case for you, and we'll see what can be arranged!-)

Alex Martelli
This will only run on linux so using pexpect shouldn't be a problem. I'm checking it out now.
Dave Forgac
On Linux (and all other unixoid systems I've tried, inc. MacOSX;-), pexpect works a charm -- let us know if you're having any problems with it, because I'm sure they can be excellently well solved!-)
Alex Martelli
Wow, pexpect worked flawlessly and easily (and can do a bunch more stuff).
Dave Forgac
+3  A: 

You should read the documentation for the subprocess module, it describes how to run an external process and access its output in real time.

Basically, you do

from subprocess import Popen, PIPE
p = Popen(['tracert', host], stdout=PIPE)
while True:
    line = p.stdout.readline()
    if not line:
        break
    # Do stuff with line

Actually, the answers in the SO question you linked to are very close to what you need. Corey Goldberg's answer uses a pipe and readline, but since it runs ping with -n 1 it doesn't last long enough to make a difference.

itsadok
Pipes are buffered (not line-buffered) by default. Unless your traceroute does setvbuf() or similar you will not get the output as it is produced. See my answer.
Thomas
On the system in question, traceroute is line-buffered and ping is not so this won't work.
Dave Forgac
+1  A: 

You can create a tty pair for the subprocess and run inside of that. According to the C standard (C99 7.19.3) the only time stdout is line buffered (as opposed to fully buffered which is what you say you don't want) is when it's a terminal. (or the child called setvbuf() obviously).

Check out os.openpty().

Untested code:

master, slave = os.openpty()
pid = os.fork()
if pid == 0:
    os.close(master)
    os.dup2(slave, 0)
    os.dup2(slave, 1)
    os.dup2(slave, 2)
    os.execv("/usr/sbin/traceroute", ("traceroute","4.2.2.1"))
    # FIXME: log error somewhere
    os.exit(1)
os.close(slave)
while True:
    d = os.read(master)
    if len(d) == 0:
        break
    print d
os.waitpid(pid, 0)

Note that having the child process (just after fork()) call setvbuf() will not work, since setvbuf() is a libc function and not a syscall. It just changes the state of the current process output, which will be overwritten on the exec call when the new binary in loaded.

Thomas
This is really cool, so +1, but Popen worked fine for me on Ubuntu and on XP. No problem with buffering.
itsadok
Then I'd say that your traceroute program uses setvbuf() or fflush(). But keep this in mind when running other stuff thay may not use them.
Thomas