Strange altered behaviour when linking from .so file with ctypes in python

Question

I am writing a program to handle data from a high speed camera for my Ph.D. project. This camera comes with a SDK in the form a .so file on Linux, for communicating with the camera and getting images out. As said it is a high speed camera delivering lots of data, (several GB a minute). To handle this amount of data the SDK has a very handy spool function that spools data directly to the hard drive via DMA, in the form of a FITS file, a raw binary format with a header that is used in astronomy. This function works fine when I write a small C program, link the .so file in and call the spool function this way. But when I wrap the .so file with ctypes and call the functions from python, all the functions are working except the spool function. When I call the spool function it returns no errors, but the spooled data file are garbled up, the file has the right format but half of all the frames are 0's. In my world it does not make sense that a function in a .so file should behave different depending on which program its called from, my own little C program or python which after all is only a bigger C program. Does any body have any clue as to what is different when calling the .so from different programs?

I would be very thankful for any suggestions

Even though the camera is commercial, some the driver is GPLed and available, though a bit complicated. (unfortunately not the spool function it seems) I have an object in python for Handel the camera.

The begining of the class reads:

class Andor:
  def __init__(self,handle=100):
    #cdll.LoadLibrary("/usr/local/lib/libandor.so")
    self.dll = CDLL("/usr/local/lib/libandor.so")
error = self.dll.SetCurrentCamera(c_long(handle))
    error = self.dll.Initialize("/usr/local/etc/andor/")

    cw = c_int()
    ch = c_int()
    self.dll.GetDetector(byref(cw), byref(ch))

The relevant function reads:

def SetSpool(self, active, method, path, framebuffersize):
    error = self.dll.SetSpool(active, method, c_char_p(path), framebuffersize)
    self.verbose(ERROR_CODE[error], sys._getframe().f_code.co_name)
    return ERROR_CODE[error]

And in the corresponding header it reads:

unsigned int SetSingleTrackHBin(int bin);

unsigned int SetSpool(int active, int method, char * path, int framebuffersize);

unsigned int SetStorageMode(at_32 mode);

unsigned int SetTemperature(int temperature);

The code to get the camera running would read something like:

cam = andor.Andor()
cam.SetReadMode(4)
cam.SetFrameTransferMode(1)
cam.SetShutter(0,1,0,0)
cam.SetSpool(1,5,'/tmp/test.fits',10);
cam.GetStatus()
if cam.status == 'DRV_IDLE':
acquireEvent.clear()
cam.SetAcquisitionMode(3)
cam.SetExposureTime(0.0)
cam.SetNumberKinetics(exposureNumber)
cam.StartAcquisition()

Answer 1

My guess is that it isn't the call to the spooling function itself, but a call series which results in corrupted values being fed to/from the library.

Are you on a 64-bit platform? Not specifying restype for anything which returns a 64-bit integer (long with gcc) or pointer will result in those values being silently truncated to 32 bits. Additionally, ctypes.c_voidp handling is a little surprising — restype values of ctypes.c_voidp aren't truncated, but are returned in the Python interpreter as type int, with predictably hilarious results if high pointers are fed back as parameters to other functions without a cast.

I haven't tested it, but both these conditions might also affect 32-bit platforms for values larger than sys.maxint.

The only way to be 100% certain you're passing and receiving the values you expect is to specify the argtypes and restype for all the functions you call. And that includes creating Structure classes and associated POINTERs for all structs those functions operate on, even opaque structs.