Help me understand why my trivial use of Python's ctypes module is failing

Question

I am trying to understand the Python "ctypes" module. I have put together a trivial example that -- ideally -- wraps the statvfs() function call. The code looks like this:

from ctypes import *

class struct_statvfs (Structure):
    _fields_ = [
            ('f_bsize', c_ulong),
            ('f_frsize', c_ulong),
            ('f_blocks', c_ulong),
            ('f_bfree', c_ulong),
            ('f_bavail', c_ulong),
            ('f_files', c_ulong),
            ('f_ffree', c_ulong),
            ('f_favail', c_ulong),
            ('f_fsid', c_ulong),
            ('f_flag', c_ulong),
            ('f_namemax', c_ulong),
            ]


libc = CDLL('libc.so.6')
libc.statvfs.argtypes = [c_char_p, POINTER(struct_statvfs)]
s = struct_statvfs()

res = libc.statvfs('/etc', byref(s))
print 'return = %d, f_bsize = %d, f_blocks = %d, f_bfree = %d' % (
    res, s.f_bsize, s.f_blocks, s.f_bfree)

Running this invariably returns:

return = 0, f_bsize = 4096, f_blocks = 10079070, f_bfree = 5048834
*** glibc detected *** python: free(): invalid next size (fast): 0x0000000001e51780 ***
*** glibc detected *** python: malloc(): memory corruption (fast): 0x0000000001e517e0 ***

I haven't been able to find any examples of calling functions with complex types as parameters (there are lots of examples of functions that return complex types), but after staring at the ctypes documentation for a day or so I think my calling syntax is correct...and it is actually callling the statvfs() call and getting back correct results.

Am I misunderstanding the ctypes docs? Or is something else going on here?

Thanks!

Answer 1

The manpage for statvfs states that the struct it uses is "defined approximately as follows", so you can't necessarily take the manpage's field listings as complete.

My guess is that there are additional struct fields after the end of the struct as you've defined it. This causes the statvfs function to overwrite memory outside your struct. I made the problem go away for me by adding a huge padding field to the _fields_ in my struct definition:

("padding", c_int * 1000),

Keep in mind that my script manifested the problem differently than yours; I got a segfault, whereas you merely got some error messages. Still, I'm guessing that it's the same problem, so you should try adding some padding and see if the problem persists.

Answer 2

As Eli indicates, looking in /usr/include/bits/statvfs.h, your struct is not defined properly.

On my 64 bit Gentoo system it would be:

 class struct_statvfs (Structure):
    _fields_ = [
            ('f_bsize', c_ulong),
            ('f_frsize', c_ulong),
            ('f_blocks', c_ulong),
            ('f_bfree', c_ulong),
            ('f_bavail', c_ulong),
            ('f_files', c_ulong),
            ('f_ffree', c_ulong),
            ('f_favail', c_ulong),
            ('f_fsid', c_ulong),
            ('f_flag', c_ulong),
            ('f_namemax', c_ulong),
            ('__f_space', c_int * 6) # you are missing this
            ]

Answer 3

According to the successful pyfuse ctypes wrappers for Fuse, the following is used for struct statvfs on Linux:

class c_statvfs(Structure):
    _fields_ = [
        ('f_bsize', c_ulong),
        ('f_frsize', c_ulong),
        ('f_blocks', c_fsblkcnt_t),
        ('f_bfree', c_fsblkcnt_t),
        ('f_bavail', c_fsblkcnt_t),
        ('f_files', c_fsfilcnt_t),
        ('f_ffree', c_fsfilcnt_t),
        ('f_favail', c_fsfilcnt_t)]

More info: http://code.google.com/p/fusepy/

Answer 4

Execute this command to get the exact definition of struct statvfs on your system:

echo '#include <sys/statvfs.h>' | gcc -E - | less

Then press /struct statvfs<enter> to skip to the definition and browse from there.

Also take a look at my patch to fusepy, and their definition.