Removing things from Python list during for loop

Question

Here is my code:

toBe =[]
#Check if there is any get request
if request.GET.items() != []:
    for x in DB:

        try:
            #This is removing the PMT that is spcific to the name
            if request.GET['pmtName'] != "None":
                if not request.GET['pmtName'] in x['tags']:
                    print x['title'], x['tags']
                    toBe.append(x)
                    continue

            #This is removing the peak stuff
            if int(request.GET['peakMin']):
                if int(request.GET['peakMin']) < int(x['charge_peak']):
                    toBe.append(x)
                    continue
            if int(request.GET['peakMax']):
                if int(request.GET['peakMax']) > int(x['charge_peak']):
                    toBe.append(x)
                    continue
            if int(request.GET['widthMin']):
                if int(request.GET['widthMin']) < int(x['charge_width']):
                    toBe.append(x)
                    continue
            if int(request.GET['widthMax']):
                if int(request.GET['widthMax']) > int(x['charge_width']):
                    toBe.append(x)
                    continue
        except:
            pass
#TODO: Stupid hack, this needs to be fixed
for x in toBe:
    DB.remove(x)
del toBe

Essentially I want to remove the item and then skip to the next one. The problem with this is that when that happens, it messes up the order of the list and skips some. Anyone know a work around for this? Or maybe just a different way of doing this?

thanks

Answer 1

for x in DB[:]: makes a copy of the list DB, so you can iterate over it while modifying the original. Care -- memory-intensive and slow.

A nicer way would be to make another layer over the list which yields only some of the values, and then iterate over that when you need it later. You can do that with a generator:

def db_view( DB ):
    for x in DB:

        #This is removing the PMT that is spcific to the name
        if request.GET.get( 'pmtName', None ) not in x['tags']:
                print x['title'], x['tags']
                continue

        #This is removing the peak stuff
        if int(request.GET['peakMin']):
            if int(request.GET['peakMin']) < int(x['charge_peak']):
                continue

        if int(request.GET['peakMax']):
            if int(request.GET['peakMax']) > int(x['charge_peak']):
                continue

        if int(request.GET['widthMin']):
            if int(request.GET['widthMin']) < int(x['charge_width']):
                continue

        if int(request.GET['widthMax']):
            if int(request.GET['widthMax']) > int(x['charge_width']):
                continue

        yield x

which you would use like

for x in db_view( DB ):
    # Do stuff

Answer 2

The answer I usually see for questions of this sort if to loop over the list backwards. Here's how I do it in one of my programs:

for i in range(len(my_list)-1,-1,-1):
    # do something

This works even if I add items to the list. On http://desk.stinkpot.org:8080/tricks/index.php/2006/08/read-a-list-backwards-in-python/ they say you can use the syntax "for i in list[::-1]:" instead. I have not tried doing it that way.

Answer 3

You're running over the same interpolation of request.GET for each value of x. Instead you could build a reusable list of filtering functions once.

For example something like:

if request.GET:
    filters = []
    if 'pmtName' in request.GET:
        n = request.GET['pmtName']
        filters.append(lambda x: n not in x['tags'])
    if 'peakMin' in request.GET and request.GET['peakMin'].isdigit():
        n = int(request.GET['peakMin'])
        filters.append(lambda x: n < int(x['charge_peak']))
    if 'peakMax' in request.GET and request.GET['peakMax'].isdigit():
        n = int(request.GET['peakMax'])
        filters.append(lambda x: n > int(x['charge_peak']))
    if 'widthMin' in request.GET and request.GET['widthMin'].isdigit():
        n = int(request.GET['widthMin'])
        filters.append(lambda x: n < int(x['charge_width']))
    if 'widthMax' in request.GET and request.GET['widthMax'].isdigit():
        n = int(request.GET['widthMax'])
        filters.append(lambda x: n > int(x['charge_width']))

Then you can apply this list of functions to select the members of DB to remove:

remove_these = [ x for x in DB if any(f(x) for f in filters)]
for item in remove_these:
    DB.remove(item)

Or create a generator that will return the values of DB that all of the filters fail on:

filtered_DB = ( x for x in DB if all(not f(x) for f in filters))