ansaurus

Question

Django: Reading FileField Contents Immediately After Save

Answer 1

A:

An ASCII codec obviously cannot decode this because it isn't ASCII. I think you will have to find out the encoding of the data and pass an unicode string to Suds. For example, if the encoding is UTF-16 pass unicode(binarydata, 'utf-16') to Suds.

What you regard as a mess of badly-encoded garbage in you first example is simply what your screen displays when you let it show binary data. The characters that are displayed here depend on the character set configuration of your system.

Your second example is Python's string representation of some binary data. A string representation contains only printable ASCII characters. The non-printable or non-ascii characters are displayed using a hexadecimal notation. This string representation just shows the bytes of your data and doesn't tell you whether the data is nicely encoded or not in some character set.

I was not able to properly identify the encoding of your second example. The closest I found was 'utf-16-le'. But this still causes decoding errors at surrogate pairs.

Using s.decode('utf-16-le', 'replace') I got a bunch of Chinese characters:

䘀䬬姘Ḽ뗼㖈ÿ鞽ÿݳ뺔豁鼞籆㐰曹䂾埓ÿ菑忊ᣠ鿀돹诡訨糍缛뇚鰧௄ÿ嚛ܙ燼㭟㱢붡辟仪〰ꋧᎾ喝嫗⧄꨿뿬쳢㑃ᚄ汉䨏좧퉊쎥钨榒촲謋以⚗퇞즵䧪巎䃀᫕ᒺ꥜ꕕ蟎᳦罨�鮉ማ᷈⻝撛妷ⲭꬎ鉱왾䲄쉘宄㋂鷲㶡崟ﱈℒꐀ喥䷚쯒걭꬧➗⋬鰧Ⴠ衯᥵�邓椾̰ᕫ녷챣컍팶㉎⇟怩ਐ詂玮괓懔ꜙ㽰�

The interesting thing is that Google translates the third character, 姘, to http.

UPDATE: The following interactive Python session may clarify what I mean in my comment below:

>>> s = '\x00F,K\xd8Y'
>>> print(s)
F,K�Y
>>> u = s.decode('utf-16-le')
>>> u
u'\u4600\u4b2c\u59d8'
>>> print(u)
䘀䬬姘
>>>

Bernd Petersohn 2010-10-22 13:43:26

I understand the first is the binary encoded version of the file. What I don't understand is why I get it when I access the file at save time, but get the Unicode representation when I open it at the command line. I'm guessing it has something to do with being on Windows and the encoding set on my code files.

Tom 2010-10-23 14:22:18

@Tom: I assume that in the first version you are directly printing the binary content of the file to the screen, whereas in the second version you let Python print a string representation of the data (possibly in an interactive Python shell?). The second version is still not unicode - the string representation just shows the bytes. Python's representation of a unicode string would look like `u'\u4600\u4b2c\u59d8'` (these are the first three Chinese characters from above).

Bernd Petersohn 2010-10-23 22:49:15

ansaurus

tags:

views:

answers:

Django: Reading FileField Contents Immediately After Save

related questions