ansaurus

Question

Using Python struct.unpack with 1-byte variables

Answer 1

+3 A:

how about ord(my_byte) ?

Or if the variable content is like my_byte == "0xff" or ff you can simply do int(my_byte, 16)

If you have a streamof hex digits, you can do:

int_bytes = (int(my_bytes[i:i+2], 16) for i in xrange(0, len(my_bytes), 2) )

jsbueno 2010-01-12 17:46:33

It requires a one-character string. How can I apply it to, say, 0xff?

David Parunakian 2010-01-12 17:48:44

Ah, David - -better provide this example on the question itself

jsbueno 2010-01-12 17:49:24

`0xff` is already a number.

Ignacio Vazquez-Abrams 2010-01-12 17:49:48

No, it's binary data read from a file. I'll update and improve my question now.

David Parunakian 2010-01-12 17:51:57

`int()` can handle a '0x' prefix on its own; no need to involve slicing.

Ignacio Vazquez-Abrams 2010-01-12 17:52:06

my_byte isn't a string containing "0xff". It is a byte containing this exact value, like a C char. It can as well be an unprintable character.

David Parunakian 2010-01-12 17:57:19

@David: Have you *tried* `ord()` yet?

Ignacio Vazquez-Abrams 2010-01-12 17:58:47

David: so, my first answer is the correct one. The fact that the byte is printede as "\xff" does not mean it's content spams for four bytes - it is just python way of representing the value 0xff within a string. You want the numerical value for the a byte inside a lenght-1 string: the python built-in that gives you this value is "ord" as stated.

jsbueno 2010-01-12 17:59:11

[Cross-posting the comment from the reply abvove] I think that ord() will also work fine; however, when I need to parse a sequence of one-byte variables unpack() is slightly more convenient, since it can be used like a, b, c = unpack("3B", file[0:3]).

David Parunakian 2010-01-15 09:44:06

Answer 2

A:

Are you asking about struct.unpack( 'b', someByte )?

A "byte" is a 1-character string.

The value 0xff is an integer, and already unpacked.

If you're finding the 4-character value 0xff in an input file, you're best served by eval() since your input file contains Python code, 0xff.

S.Lott 2010-01-12 17:55:42

`eval()`? Seriously? Not `int(s, 16)`?

Ignacio Vazquez-Abrams 2010-01-12 17:57:45

Yes, struct.unpack('b', someByte) (or, rather, "B") appears to be what I've been looking for. Must have overlooked it in the documentation. Thanks!

David Parunakian 2010-01-12 18:00:19

@David: are you REALLY sure ord(file[0]) would give different result?

Antony Hatchkins 2010-01-12 18:20:03

Depends on `unpack('b',` vs. `unpack('B',`.

S.Lott 2010-01-12 18:21:05

@Ignacio Vazquez-Abrams: Yes. Seriously. I don't really think any sociopaths are going sneak evil code into the input file and replace `0xff` with `import subprocess; subprocess.Popen('format C:')`. I'm really, really sure nothing bad will happen from using `eval()` like this.

S.Lott 2010-01-12 18:22:38

Interesting that you can be "really, really sure nothing bad will happen" when you're answering a question, from a "random" developer, with limited context into the problem domain. What if this code ends up publicly released and used/misused in other contexts? Or ends up getting migrated to a web service, etc, etc.

Travis Bradshaw 2010-01-12 19:51:55

The sociopath -- the one who is so evil that they would attempt to doctor those four bytes of a file -- has the entire Python source code for the application completely available to them. Why mess around with 4 silly bytes of some file, which the sociopath can create real havoc by simply fixing the code to raise havoc.

S.Lott 2010-01-12 20:26:42

Another assumption that's not necessarily true. Python as a web service, for example, does not provide for the "there are so many easier ways to do malicious things, being careless here won't matter" construct. Also, there's no requirement for a sociopath, bad things can also happen on accident. Why use a questionable practice that may lead to a problem when there are other practices that cannot?

Travis Bradshaw 2010-01-12 21:47:31

@Travis Bradshow: Of all the possible permutations of 4 characters, none include anything remotely dangerous; no python statement (except) `def` is short enough. The odds of an "accident" are zero; unless you allow for a sociopath to maliciously edit the file. No, there are *real* security concerns like compromised passwords, bad handling of user authentication credentials and Windows that are far more important than `eval`.

S.Lott 2010-01-12 21:59:41

@Antony: I think that ord() will also work fine; however, when I need to parse a sequence of one-byte variables unpack() is slightly more convenient, since it can be used like a, b, c = unpack("3B", file[0:3]).

David Parunakian 2010-01-15 09:34:53

@S.Lott: I think I'll avoid using eval(). Although these data files we need to parse are automatically generated, they're still supplied to us by a third party.

David Parunakian 2010-01-15 09:42:30

ansaurus

tags:

views:

answers:

Using Python struct.unpack with 1-byte variables

related questions