views:

736

answers:

1

Updates:

I wrote an asynchronous C version and it works as it should.

Turns out the speed issue was due to Python's GIL. There's a method to fine tune its behavior. sys.setcheckinterval(interval)

Setting interval to zero (default is 100) fixes the slow speed issue. Now all that's left is to figure out is what's causing the other issue (not all pixels are filled). This one doesn't make any sense. usbmon shows all the communications are going through. libusb's debug messaging shows nothing out of the ordinary. I guess I need to take usbmon's output and compare sync vs async. The data that usbmon shows seems to look correct at a glance (The first byte should be 0x96 or 0x95).

As said below in the original question, S. Lott, it's for a USB LCD controller. There are three different versions of drv_send, which is the outgoing endpoint method. I've explained the differences below. Maybe it'll help if I outline the asynchronous USB operations. Note that syncrhonous USB operations work the same way, it's just that it's done synchronously.

We can view asynchronous I/O as a 5 step process:

  1. Allocation: allocate a libusb_transfer (This is self.transfer)
  2. Filling: populate the libusb_transfer instance with information about the transfer you wish to perform (libusb_fill_bulk_transfer)
  3. Submission: ask libusb to submit the transfer (libusb_submit_transfer)
  4. Completion handling: examine transfer results in the libusb_transfer structure (libusb_handle_events and libusb_handle_events_timeout)
  5. Deallocation: clean up resources (Not shown below)

Original question:

I have three different versions. One's entirely synchronous, one's semi-asynchronous, and the last is fully asynchronous. The differences is that synchronous fully populates the LCD display I'm controlling with the expected pixels, and it's really fast. The semi-asynchronous version only populates a portion of the display, but it's still very fast. The asynchronous version is really slow and only fills a portion of the display. I'm baffled why the pixels aren't fully populated, and why the asynchronous version is really slow. Any clues?

Here's the fully synchronous version:

def drv_send(self, data):
    if not self.Connected():
        return

    self.drv_locked = True
    buffer = ''
    for c in data:
        buffer = buffer + chr(c)
    length = len(buffer)
    out_buffer = cast(buffer, POINTER(c_ubyte))
    libusb_fill_bulk_transfer(self.transfer, self.handle, LIBUSB_ENDPOINT_OUT + 1, out_buffer, length, self.cb_send_transfer, None, 0)
    lib.libusb_submit_transfer(self.transfer)
    while self.drv_locked:
        r = lib.libusb_handle_events(None)
        if r < 0:
            if r == LIBUSB_ERROR_INTERRUPTED:
                continue
            lib.libusb_cancel_transfer(transfer)
            while self.drv_locked:
                if lib.libusb_handle_events(None) < 0:
                    break

    self.count += 1

Here's the semi-asynchronous version:

def drv_send(self, data):
    if not self.Connected():
        return

    def f(d):
        self.drv_locked = True
        buffer = ''
        for c in data:
            buffer = buffer + chr(c)
        length = len(buffer)
        out_buffer = cast(buffer, POINTER(c_ubyte))
        libusb_fill_bulk_transfer(self.transfer, self.handle, LIBUSB_ENDPOINT_OUT + 1, out_buffer, length, self.cb_send_transfer, None, 0)
        lib.libusb_submit_transfer(self.transfer)
        while self.drv_locked:
            r = lib.libusb_handle_events(None)
            if r < 0:
                if r == LIBUSB_ERROR_INTERRUPTED:
                    continue
                lib.libusb_cancel_transfer(transfer)
                while self.drv_locked:
                    if lib.libusb_handle_events(None) < 0:
                        break

        self.count += 1

    self.command_queue.put(Command(f, data))

Here's the fully asynchronous version. device_poll is in a thread by itself.

def device_poll(self):
    while self.Connected():
        tv = TIMEVAL(1, 0)
        r = lib.libusb_handle_events_timeout(None, byref(tv))
        if r < 0:
            break

def drv_send(self, data):
    if not self.Connected():
        return

    def f(d):
        self.drv_locked = True
        buffer = ''
        for c in data:
            buffer = buffer + chr(c)
        length = len(buffer)
        out_buffer = cast(buffer, POINTER(c_ubyte))
        libusb_fill_bulk_transfer(self.transfer, self.handle, LIBUSB_ENDPOINT_OUT + 1, out_buffer, length, self.cb_send_transfer, None, 0)
        lib.libusb_submit_transfer(self.transfer)
        self.count += 1

    self.command_queue.put(Command(f, data))

And here's where the queue is emptied. It's the callback for a gobject timeout.

def command_worker(self):
    if self.drv_locked: # or time.time() - self.command_time < self.command_rate:
        return True
    try:
        tmp = self.command_queue.get_nowait()
    except Queue.Empty:
        return True
    tmp.func(*tmp.args)
    self.command_time = time.time()
    return True

Here's the transfer's callback. It just changes the locked state back to false, indicating the operation's finished.

def cb_send_transfer(self, transfer):
    if transfer[0].status.value != LIBUSB_TRANSFER_COMPLETED:
        error("%s: transfer status %d" % (self.name, transfer.status))
    print "cb_send_transfer", self.count
    self.drv_locked = False
+1  A: 

Ok I don't know if I get you right. You have some device with LCD, you have some firmware on it to handle USB requests. On PC side you are using PyUSB wich wraps libUsb.

Couple of suggestions if you are experiancing speed problems, try to limit data you are transfering. Do not transfer whole raw data, mayby only pixels that changed.

Second, have you measured speed of transfers by using some USB analuzer sofware, if you don't have money for hardvare usb analyzer maybe try software version. I never used that kind of analyzers but I think data provided by them is not very reiable.

Thirdly, see what device is realy doing, maybe that is bottleneck of your data transfers.

I have not much time today to exactly anwser your question so I will get back on this later.

I am watching this thread for some time, and there is dead silence around this, so I tried to spare some time and look deeper. Still not much time today maybe later today. Unfortunetly I am no python expert but I know some stuff about C,C++, windows and most of all USB. But I think this may be LCD device problem, what are you using, Because if the transfers works fine, and data was recived by the device it points that is device problem.

I looked at your code a little, could you do some testing, sending only 1 byte, 8 bytes, and Endpoint size byte length transfer. And see how it looks on USB mon ?

Endpoint size is size of Hardvare buffer used by PICO LCD USB controler. I am not sure what it is for your's but I am guessing that when you send ENdpoint size message next masage should be 0 bytes length. Maybe there is the problem. Regarding the test I assume you have seen data wich you programed to send. Second thing could be that the data gets overwriten, or not recived fast enough. Saying overwriten I mean LCD could not see data end, and mix one transfer with another.

I am not sure what USB mon is capable of showing, but according to USB standart after Endpoint size packet len, there should be 0 len packet data send, showing that is end of transfer.

CrazyChris
Well, I'm not using pyusb. I'm wrapping libusb-1.0 through Python ctypes as pyusb is based on libusb-0.1 which only supports synchronous transfers. Second, I fixed the speed issue, although in saying that I begin wondering if this other issue isn't speed related as well. Thirdly, I'm using usbmon to analyze the transfers. I'm not sure how to analyze the speed though. Anyhow, great to know someone finally tried answering this. :)
Scott
Well, I wrote an asynchronous C version as suggested by a libusb dev. After some hair pulling I got it to work fine. It's a really strange issue. usbmon shows the transfers are making it through. The callback's being called. I'm not sure what else I can do at this point.
Scott
I'm not sure what you mean by "Endpoint size". I did 1 and 8 byte, and they're both correct. Both started with 0x96, the 8 bytes transfer followed by 7 zeros.
Scott
Well, I've written the manufacturer asking for the LCD's documentation. That was two days ago, nothing so far. I've often thought that it may be a speed issue, and I've adjusted transfer rate some with the same results. I still haven't counted that out though.
Scott
Ok, lsusb shows max packet size, which is 64 bytes. I've been sending 64 bytes for a while, when attempting to clear the LCD. What do you mean by "next message should be 0 bytes length?"
Scott
Well, I gave up on Python. I tried OpenUSB, but it had memory leaks. It worked however. Something about libusb-1.0 and Python's threading just didn't get along. I started over writing the program in C++.
Scott
Hey, I figured out what was wrong here. It had to do with a variable in another part of code being garbage collected. I rewrote the program in C++, though. :) Thanks for all the help you gave.
Scott