Python popen wont work with block devices

Question

I am wring a small forensic app which looks like this so far

import time, os,sys

def getter():
    filename = sys.argv[1]
    print "File Entered: " + filename
    os.system('file ' + filename)
    print "\n"
    pipe =  os.popen("xxd " + filename, "r")
    print pipe.read()

I input via the command line a file and it prints out the type of file and then it is supposed to create a pipe from the terminal back to the python app under the pipe name "pipe". I then read the pipe named "pipe"

This works great on smaller text files, but it will not work on block devices even when I run this app as super user. In the end this will recover files based on the output of xxd.

any tips, thank you.

Answer 1

The problem probably lies with how you call xxd, given that your application doesn't do anything at all with the file.

Answer 2

What happens if you just run xxd /dev/diskwhatever from the shell prompt? Does it work, and how much information does it spew out? Assuming that, as superuser, you do have read permission, the attempt to read everything at one gulp in your code's last line would be the point where failure could be expected (since the amount of information can be really huge); the workaround would be to read a little at a time, instead of doing a single .read() call.

Edit: whether you get your pipe in the nice modern way (with subprocess) or the deprecated-but-still-working old way (with popen), makes no difference to this problem. In either case, you can get one line at a time by just looping over the pipe object, for example:

child = subprocess.Popen(whatever, stdout=subprocess.PIPE)
for line in child.stdout:
  print "One more line:", line

Answer 3

I assume you've tried the same command from the shell prompt and it worked well, even when piped to less or something of the sort.

I have a strong feeling that using subprocess will fix this behavior; This module replaces the popen call which is deprecated, and gives a great degree of flexibility in running and getting results from the executed commands.

output = Popen(["xxd", "file"], stdout=PIPE).communicate()[0]

Try to run the process under its own shell, and read its pipe. If this still does not work, make sure to set a reasonable limit to its buffer size, and make sure you're not using communicate which will make you wait till the process terminates (a long time when handling a block device).

Comment my answer if it didn't work, and try to accurately describe what wen't wrong - is it freezing, just no output or corrupt results?