Writing a binary buffer to a file in python

Question

Hi there,

I have some python code that:

1. Takes a BLOB from a database which is compressed.
2. Calls an uncompression routine in C that uncompresses the data.
3. Writes the uncompressed data to a file.

It uses ctypes to call the C routine, which is in a shared library.

This mostly works, except for the actual writing to the file. To uncompress, I get the data uncompressed into a python buffer, created using the ctypes create_string_buffer method:

c_uncompData_p = create_string_buffer(64000)

so the uncompression call is like this:

c_uncompSize = mylib.explodeCharBuffer (c_data_p, c_data_len, c_uncompData_p)

The size of the resulting uncompressed data is returned as the return value.

But... I have no idea how to force python on only write c_uncompSize bytes out - if I do:

myfile.write (c_uncompData_p.raw)

it writes the whole 64k buffer out (the data is binary - so it is not null terminated).

So, my question is - using Python 2.5 how do I get c_uncompSize bytes printed out, rather than the whole 64k?

Thanks Jamie

Answer 1

Slicing works for c_char_Arrays too:

myfile.write(c_uncompData_p[:c_uncompSize])

Answer 2

buffer() might help to avoid unnecessary copying (caused by slicing as in @elo80ka's answer):

myfile.write(buffer(c_uncompData_p.raw, 0, c_uncompSize))

In your example it doesn't matter (due to c_uncompData_p is written only once and it is small) but in general it could be useful.

---

Just for the sake of exercise here's the answer that uses C stdio's fwrite():

from ctypes import *

# load C library
try: libc = cdll.msvcrt # Windows
except AttributeError:
     libc = CDLL("libc.so.6") # Linux

# fopen()
libc.fopen.restype = c_void_p
def errcheck(res, func, args):
    if not res: raise IOError
    return res
libc.fopen.errcheck = errcheck
# errcheck() could be similarly defined for `fwrite`, `fclose` 

# write data
file_p  = libc.fopen("output.bin", "wb")
sizeof_item = 1 # bytes
nitems  = libc.fwrite(c_uncompData_p, sizeof_item, c_uncompSize, file_p)
retcode = libc.fclose(file_p)
if nitems != c_uncompSize: # not all data were written
   pass
if retcode != 0: # the file was NOT successfully closed
   pass