Memory error (MemoryError) when creating a boolean NumPy array (Python).

Question

I'm using NumPy with Python 2.6.2. I'm trying to create a small (length 3), simple boolean array. The following gives me a MemoryError, which I think it ought not to.

import numpy as np
cond = np.fromiter((x in [2] for x in [0, 1, 2]), dtype = np.bool)

The error it gives me is:

MemoryError: cannot allocate array memory

However, the following method of obtaining a list (as opposed to an ndarray) works fine (without using numpy):

cond = list((x in [2] for x in [0, 1, 2]))

Have I done anything wrong in the Numpy code? My feeling is that it ought to work.

Answer 1

You should not get any error.

With Python 2.6.5 or Python 2.7, and Numpy 1.5.0, I don't get any error. I therefore think that updating your software could very well solve the problem that you observe.

Answer 2

I can reproduce the problem with numpy 1.1 (but not with anything newer). Obviously, upgrading to a more recent version of numpy is your best bet.

Nonetheless, it seems to be related to using np.bool as the dtype when count=-1 (the default: Read all items in the iterator, instead of a set number).

A quick workaround is just to create it as an int array and then convert it to a boolean array:

cond = np.fromiter((x in [2] for x in [0, 1, 2]), dtype=np.int).astype(np.bool)

An alternative is to convert it to a list, and then set count to the length of the list (or just use np.asarray on the list):

items = list((x in [2] for x in [0, 1, 2]))
cond = np.fromiter(items, dtype=np.bool, count=len(items))

Obviously, both of these are suboptimal, but if you can't upgrade to a more recent version of numpy, they will work.