What do you make of this Python error?

Question

Here's the error.

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 295, in 'calling callback function'
  File "USB2.py", line 454, in ff
    self.drv_locked = False
SystemError: Objects/cellobject.c:24: bad argument to internal function

Here's the Python code involved.

def drv_send(self, data, size):
    if not self.Connected():
        return

    def f():
        self.drv_locked = True
        buffer = ''.join(chr(c) for c in data[:size])
        out_buffer = cast(buffer, POINTER(c_uint8))
        request_handle = (OPENUSB_REQUEST_HANDLE * 1)()
        request = (OPENUSB_INTR_REQUEST * 1)()

        request_handle[0].dev = self.usbhandle
        request_handle[0].interface = INTERFACE_ID
        request_handle[0].endpoint = LIBUSB_ENDPOINT_OUT + 1
        request_handle[0].type = USB_TYPE_INTERRUPT
        request_handle[0].req.intr = request
        def f(req):
            print req[0].req.intr[0].result.status, req[0].req.intr[0].result.transferred_bytes
            self.drv_locked = False # Line 454
        request_handle[0].cb = REQUEST_CALLBACK(f)
        request_handle[0].arg = None

        request[0].payload = out_buffer
        request[0].length = size
        request[0].timeout = 5000
        request[0].flags = 0
        request[0].next = None

        r = lib.openusb_xfer_aio(request_handle)
        print "result", r

    self.command_queue.put(f)

And here's the Python source involved.

PyObject *
PyCell_Get(PyObject *op)
{
        if (!PyCell_Check(op)) {
                PyErr_BadInternalCall(); // Line 24
                return NULL;
        }
        Py_XINCREF(((PyCellObject*)op)->ob_ref);
        return PyCell_GET(op);
}

Answer 1

The PyCell_Check function checks that its argument actually is a cell object (an internal type used to implement variables referenced by multiple scopes). If op is not a cell object, you would get this error.

The code you posted does not give enough context/information to determine exactly how the bad parameter came to be passed.

Answer 2

An internal error is clearly a bug in Python itself, and if you're interested in further exploring this and offering a fix for the Python core, then simplifying your code down to where it still triggers the bug would be the right strategy.

If you're more interested in having your code work, rather than in fixing the Python core, then I suggest you avoid some of the several anomalies in your code that might be contributing to confusing Python. For example, I don't know that anybody ever thought to test property for a nested function named f containing yet another further-nested function also named f -- it SHOULD work, but it's exactly the kind of thing that might not have been well tested just because nobody thought of it yet, and while deliberately provoking such anomalies is a very good strategy for strenghtening a suite of tests, it might be best avoided if you're not deliberately out to trigger bugs in Python's internals.

So, first, I would make sure there's no homonimy around. If that still leaves the bug, I would next remove the use of cell objects by turning what currently are accesses to nonlocal variables into "prebound arguments", for example your "semi-outer" f could be changes to start with:

def f(self=self):

and your "fully-inner one" could become:

def g(req, self=self):

This would make accesses to self in either of those functions (currently nonlocal variable accesses) into local variable accesses. Yes, you should not have to do this (there should be no bugs in any software, that requires you to work around them), but alas perfection is not a characteristic of this sublunar world, so that learning bug-workaround strategies is an inevitable part of life;-).