Zlib in database - Django

Question

When I try to put a zlibbed string in models.TextField

>>> f = VCFile(head = 'blahblah'.encode('zlib'))
>>> f.save()

it fails:

    ...
raise DjangoUnicodeDecodeError(s, *e.args)
DjangoUnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 1: unexpected code byte. You passed in 'x\x9cK\xcaI\xccH\x02b\x00\x0eP\x03/' (<type 'str'>)

Is there any way to fix this (apart from escaping the string - it has to be space-efficent)?

Answer 1

If you don't want to encode it, you have to store it as a binary object (BLOB), not a string. Django doesn't seem to support BlobFields out of the box, so go find it on the net or hack something together.

Answer 2

Like Marcus says, you'll have to use BLOB if you want to keep it in binary format. If you're OK with encoding it, you can use base64 encoding:

from base64 import binascii

f = VCFile(head = binascii.b2a_base64('blahblah'.encode('zlib')))

In my very basic tests with 33k characters, the zlib string was 28% the size of the original string, the base64 encoded zlib string was 37% the size of the original string. Not quite as good on compression, but still a big improvement.