Efficient way to count unique elements in array in numpy/scipy in Python

Question

I have a scipy array, e.g.

a = array([[0, 0, 1], [1, 1, 1], [1, 1, 1], [1, 0, 1]])

I want to count the number of occurrences of each unique element in the array. For example, for the above array a, I want to get out that there is 1 occurrence of [0, 0, 1], 2 occurrences of [1, 1, 1] and 1 occurrence of [1, 0, 1].

One way I thought of doing it is:

from collections import defaultdict
d = defaultdict(int)

for elt in a:
  d[elt] += 1

is there a better/more efficient way?

thanks.

Answer 1

If sticking with Python 2.7 (or 3.1) is not an issue and any of these two Python versions is available to you, perhaps the new collections.Counter might be something for you if you stick to hashable elements like tuples:

>>> from collections import Counter
>>> c = Counter([(0,0,1), (1,1,1), (1,1,1), (1,0,1)])
>>> c
Counter({(1, 1, 1): 2, (0, 0, 1): 1, (1, 0, 1): 1})

I haven't done any performance testing on these two approaches, though.

Answer 2

for python 2.6 <

import itertools

data_array = [[0, 0, 1], [1, 1, 1], [1, 1, 1], [1, 0, 1]]

dict_ = {}

for list_, count in itertools.groupby(data_array):
    dict_.update({tuple(list_), len(list(count))})

Answer 3

You can sort the array lexicographically by rows and the look for points where the rows change:

In [1]: a = array([[0, 0, 1], [1, 1, 1], [1, 1, 1], [1, 0, 1]])

In [2]: b = a[lexsort(a.T)]

In [3]: b
Out[3]: 
array([[0, 0, 1],
       [1, 0, 1],
       [1, 1, 1],
       [1, 1, 1]])

...


In [5]: (b[1:] - b[:-1]).any(-1)
Out[5]: array([ True,  True, False], dtype=bool)

The last array says that the first three rows differ and the third row is repeated twice.

For arrays of ones and zeros you can encode the values:

In [6]: bincount(dot(a, array([4,2,1])))
Out[6]: array([0, 1, 0, 0, 0, 1, 0, 2])

Dictionaries can also be used. Which of the various methods will be fastest will depend on the sort of arrays you are actually working with.