views:

52

answers:

3

As per the title, I have a nested lists like so (the nested list is a fixed length):

        # ID,  Name, Value
list1 = [[ 1, "foo",    10],
         [ 2, "bar",  None],
         [ 3, "fizz",   57],
         [ 4, "buzz", None]]

I'd like to return a list (the number of items equal to the length of a sub-list from list1), where the sub-lists are the indices of rows without None as their Xth item, i.e.:

[[non-None ID indices], [non-None Name indices], [non-None Value indices]]

Using list1 as an example, the result should be:

[[0, 1, 2, 3], [0, 1, 2, 3], [0, 2]]

My current implementation is:

indices = [[] for _ in range(len(list1[0]))]
for i, row in enumerate(list1):
    for j in range(len(row)):
        if not isinstance(row[j], types.NoneType):
            indices[j].append(i)

...which works, but can be slow (the lengths of the lists are in the hundreds of thousands).

Is there a better/more efficient way to do this?

EDIT:

I've refactored the above for loops into nested list comprehensions (similar to SilentGhost's answer). The following line gives the same result as the my original implementation, but runs approximately 10x faster.

[[i for i in range(len(list1)) if list1[i][j] is not None] for j in range(len(log[0]))]
+4  A: 
>>> [[i for i, j in enumerate(c) if j is not None] for c in zip(*list1)]
[[0, 1, 2, 3], [0, 1, 2, 3], [0, 2]]

in python-2.x you could use itertools.izip instead of zip to avoid generating intermediate list.

SilentGhost
A: 
[[i for i in range(len(list1)) if list1[i] is not None] for _ in range(len(log[0]))]

The above seems to be about 10x faster than my original post.

Noah
A: 
import numpy as np

map(lambda a: np.not_equal(a, None).nonzero()[0], np.transpose(list1))
# -> [array([0, 1, 2, 3]), array([0, 1, 2, 3]), array([0, 2])]
J.F. Sebastian
this variant is 3 times slower than @SilentGhost's one (on some input).
J.F. Sebastian