Remove items from a list while iterating in Python

Question

Hi,

I'm iterating over a list of tuples in Python, and am attempting to remove them if they meet certain criteria.

for tup in somelist:
    if determine(tup):
         code_to_remove_tup

What should I use in place of code_to_remove_tup? I can't figure out how to remove the item in this fashion.

Answer 1

You need to take a copy of the list and iterate over it first, or the iteration will fail with what may be unexpected results.

For example (depends on what type of list):

for tup in somelist[:]:
    etc....

An example:

>>> list = range(10)
>>> for x in list:
...     list.remove(x)
>>> list
[1, 3, 5, 7, 9]

>>> list = range(10)
>>> for x in list[:]:
...     list.remove(x)
>>> list
[]

Answer 2

Your best approach for such an example would be a list comprehension

somelist = [tup for tup in somelist if determine(tup)]

In cases where you're doing something more complex than calling a determine function, I prefer constructing a new list and simply appending to it as I go. For example

newlist = []
for tup in somelist:
    # lots of code here, possibly setting things up for calling determine
    if determine(tup):
        newlist.append(tup)
somelist = newlist

Copying the list using remove might make your code look a little cleaner, as described in one of the answers below. You should definitely not do this for extremely large lists, since this involves first copying the entire list, and also performing an O(n) remove operation for each element being removed, making this an O(n^2) algorithm.

for tup in somelist[:]:
    # lots of code here, possibly setting things up for calling determine
    if determine(tup):
        newlist.append(tup)

Answer 3

A list comprehension is best for this kind of loop.

somelist = [x for x in somelist if determine(x)]

EDIT: Jobs' comment says that he wants the 'determine' to say what should be deleted. That would then be just.

somelist = [x for x in somelist if not determine(x)]

EDIT:

somelist[:] = [x for x in somelist if not determine(x)]

Brandon Corfman is correct, you will loose reference to the original list unless you do it this way and please look at Alex Martelli comment for the detail.

Also, I liked Cides' suggestion below using itertools. However there is no non iterator filterfalse, so it will have to be.

from itertools import ifilterfalse
somelist[:] = list(ifilterfalse(determine, somelist))

Answer 4

for i in xrange(len(somelist) - 1, -1, -1):
    if some_condition(somelist, i):
        del somelist[i]

You need to go backwards otherwise it's a bit like sawing off the tree-branch that you are sitting on :-)

Answer 5

For those that like functional programming:

>>> somelist[:] = filter(lambda tup: not determine(tup), somelist)
or:
>>> from itertools import ifilterfalse
>>> somelist[:] = list(ifilterfalse(determine, somelist))

Updated to correct my answer. Thanks, David Raznick.

Answer 6

The answers suggesting list comprehensions are ALMOST correct -- except that they build a completely new list and then give it the same name the old list as, they do NOT modify the old list in place. That's different from what you'd be doing by selective removal, as in @Lennart's suggestion -- it's faster, but if your list is accessed via multiple references the fact that you're just reseating one of the references and NOT altering the list object itself can lead to subtle, disastrous bugs.

Fortunately, it's extremely easy to get both the speed of list comprehensions AND the required semantics of in-place alteration -- just code:

somelist[:] = [tup for tup in somelist if determine(tup)]

Note the subtle differene with other answers: this one is NOT assigning to a barename - it's asssigning to a list slice that just happens to be the entire list, thereby replacing the list contents within the same Python list object, rather than just reseating one reference (from previous list object to new list object) like the other answers.

Answer 7

The question was about modifying while iterating, not computing a new list and storing it in the extent of the old list. The only solution which actually answers the question is the one which iterates backwards by index.