(Unintentionally) skipping items when iterating over a list

Question

I have a list and I want to remove from it the items that don't appear in another list. I've tried the following:

for w in common:
        for i in range(1,n):
            if not w in words[i]:
                common.remove(w)

However, this fails to remove some of the items. Adding print statements

for w in common:
        for i in range(1,n):
                        print w
            if not w in words[i]:
                print w
                common.remove(w)

results in some w never being printed. Any ideas as to what's happening? I assume the answer's simple and I just don't have adequate Python knowledge, but I'm completely out of ideas.

Answer 1

You can't delete items from the list you're iterating over. Try iterating over a copy of the list instead.

for w in common[:]:
    for i in range(1,n):
        if not w in words[i]:
            common.remove(w)

Answer 2

You are modifying the list while trying to iterate through it. You could modify the first line of the code to iterate through a copy of the list (using common[:]).

Answer 3

I think you can simplify your statement with something like this:

filtered = filter(lambda x: x in words, common)

That's checking each element in common for it's presence in words and removing based on it. You may need to try x not in words depending on what you're desired result is, but I think that should come close.

I wanted to add one other approach, that might also come close, though I would need to see examples of your initial lists to test it fully.

filtered = [x for x in common if x in words]

-- EDITED -- I had the syntax in the list comprehension backwards, but caught it after I saw the comment. Thanks!

Answer 4

From the Python docs:

It is not safe to modify the sequence being iterated over in the loop (this can only happen for mutable sequence types, such as lists). If you need to modify the list you are iterating over (for example, to duplicate selected items) you must iterate over a copy.

Answer 5

If you delete (say) item 5, then the old item 6 will now be item 5. So if you think to move to item 6 you will skip it.

Is it possible to iterate backwards over that list? Then index-changes happen in parts you already processed.

Answer 6

Unfortunately, I didn't have cookies enabled when I posted, so I'm stuck with adding an answer as the only apparent way of replying. Thank you all for your assistance. Iterating over a copy did work, of course.