Re-order list in Python to ensure it starts with check values.

Question

Dear all,

I'm reading in serial data using Pyserial, to populate a list of 17 values (1byte each) at a sampling rate of 256Hz.

The bytes I ultimately want to use are the 5th to 8th in the list. Providing no bytes are dropped, the first two values of the stream are always the same ('165','90'). I'm getting quite a few dropped values though, and my list values are shifting, so when I read the 5th-8th bytes, they aren't the correct values.

I've partially combatted this by ensuring that before the wanted segement is captured, the first few values are checked against what they should be (i.e. if mylist[0]==165 &....).
This is crude but ok since the chances of these two values appearing adjacent to each other in the list elsewhere is small. The problem is that this means as soon as the bytes shift, I'm losing a load of values, until it eventually realigns.

My question is: what code can I use to either:

a) Force the list to realign once it has been detected that it no longer starts with 165,90. (elif....).

b) Detect where '165' & '90' are (next to each other) in the list and extract the values I want in relation to their position (next but one, onwards).

Thanks in advance

S_S

Just noticed from the related Qs that I could use

mylist.append(mylist.pop(0)) 

multiple times until they are in the right place. Is there a better way that anyone can suggest?

Answer 1

In case I understood you well, suppose you have a list like this:

l = [67, 126, 165, 90, 11, 1, 3, 5, 151, 99, 23]

you'd want to obtain: useful = [3,5,151,99]

Then, you could do:

# obtain places where 165 is followed by 90
match = [x for x in xrange(len(l)-1) if l[x]==165 and l[x+1]==90]
# obtain range of useful values
useful = l[match[0]+4:match[0]+8]

You may have to tweak the numbers in case I misunderstood your problem. Hope it helps,

Manuel

Answer 2

If I understand your problem correct, you have data continuously arriving, in 17-byte chunks. Assuming that's right, how about something like this:

while True:    
    mylist.extend(get_more_data())

    # If we're misaligned, skip bytes until we're aligned again
    while mylist and mylist[0] != 165 or mylist[1] != 90:
        mylist.pop(0)

    # If we've skip bytes, we might not have enough data, so get more
    if len(mylist) < 17:
        continue

    # Process one chunk of data and remove it from the list
    process_data(mylist[:17])
    del mylist[:17]

Answer 3

I would use the 165 and 90 as header values, always checking the incoming byte for a match. This solution automatically re-syncs itself and it is as simple as:

def get_message():
    while True: #Some timeout functionality is useful here ;-)
       message = []
       byte = xxx.get_byte()
       if byte == 165:
          byte = xxx.get_byte()
          if byte == 90:
             xxx.get_byte() #drop it
             xxx.get_byte() #drop it
             message[0] = xxx.get_byte()
             message[1] = xxx.get_byte()
             message[2] = xxx.get_byte()
             message[3] = xxx.get_byte()
             #Break out of loop. We have a message! 
             return message

If your designing both sides, you really should insert some kind of checksum. If your just hacking something, look in the bytes for a checksum. It's probably there already.