tags:

views:

260

answers:

3

Is it possible to read stdin as binary data in Python 2.6? If so, how?

I see in the Python 3.1 documentation that this is fairly simple, but the facilities for doing this in 2.6 don't seem to be there.

If the methods described in 3.1 aren't available, is there a way to close stdin and reopen in in binary mode?

Update

Just to be clear, I am using 'type' in a MS-DOS shell to pipe the contents of a binary file to my python code. This should be the equivalent of a Unix 'cat' command, as far as I understand. But when I test this out, I always get one byte less than the expected file size.

Update #2

First off, thanks for all the answers. I'm slowly working towards a real, usable solution here. In the end, I'm still trying to build a self contained JAR file that executes my Python code automatically passing through all the command line arguments untainted.

The reason I'm going the Java/JAR/Jython route is because one of my main external libraries is only available as a Java JAR. But unfortunately, I had started my work as Python. It might have been easier to convert my code over to Java a while ago, but since this stuff was all supposed to be compatible, I figured I would try trucking through it and prove it could be done.

In case anyone was wondering, this is also related to the question I asked a few days ago.

http://stackoverflow.com/questions/2827771/packaging-and-deploying-a-jython-program-from-eclipse

Some of that question was answered in this question.

So I'll try to update my original question with some notes on what I have figured out so far.

A: 
import sys

data = sys.stdin.read(10) # Read 10 bytes from stdin

If you need to interpret binary data, use the struct module.

Yann Ramin
If I then call sys.stdin.read() with no parameter, it should read all the binary data that was piped in, correct? How then do I determine the length correctly? len(data) returns the incorrect value if the last byte of the data was a zero. How do you check and correct for this situation?
thebeav
`len` counts the \x00 characters in the string. Python does not have null terminated strings. `len("Hello\x00") == 6`
Yann Ramin
I wonder then if it might be the 'type' command from the MS-DOS shell that is causing the loss of the final byte? I guess I will have to test the equivalent on Linux. Thanks.
thebeav
A: 

In python 2.6 strings are byte arrays, therefore they will defacto read binary data.

Kugel
+1  A: 

Use the -u command line switch to force Python to treat stdin, stdout and stderr as binary unbuffered streams.

C:> type mydoc.txt | python.exe -u myscript.py
Dan Menes
I have tested this with 'type' and it appears to work. That is, if I leave out the -u flag, I get one fewer character per line.
Dan Menes
Cool. Thanks for the test.So, just because I like you so much, any idea how to pass the '-u' option through the JarRunner.java class that is used to fire off Jython through an executable JAR file?I know. I never do anything the easy way.
thebeav
According to docs, setting the PYTHONUNBUFFERED environment variable will have the same effect. Not sure if that helps.
Dan Menes
Even easier, it appears that all you need to do is:sys.stdin = os.fdopen(sys.stdin.fileno(), 'rb', 0)That will reopen the fd in unbuffered 'binary' mode.
thebeav
@thebeav: Oddly enough that doesn't work on my system. I don't know if that's because I'm using CPython instead of Jython, or if it's because I'm running Windows XP Pro, and "type" behaves differently, or its because there is a magnetic anomaly in the Manassas area that makes computers do different things.FWIW, I tried a number of ways to get Python to change the file mode after the interpreter had started, including accessing the C runtime's "setmode" function via ctypes. Nothing works for me.
Dan Menes
Uh-oh. I smell a portability issue. Thanks for the info. I guess I'm going to have to do some fairly rigorous testing on multiple platforms. I hope this doesn't have to do with the JVM in use.
thebeav