Error Converting PIL B&W images to Numpy Arrays

Question

I am getting weird errors when I try to convert a black and white PIL image to a numpy array. An example of the code I am working with is below.

    if image.mode != '1':
        image = image.convert('1') #convert to B&W
    data = np.array(image) #Have also tried np.asarray(image)
    n_lines = data.shape[0] #number of raster passes
    line_range = range(data.shape[1])
    for l in range(n_lines):
        # process one horizontal line of the image
        line = data[l]
        for n in line_range:
            if line[n] == 1:
                write_line_to(xl, z+scale*n, speed) #conversion to other program code
            elif line[n] == 0:
                run_to(xl, z+scale*n) #conversion to other program code

I have tried this using both array and asarray for the conversion, and gotten different errors. If I use array, then the data I get out is nothing like what I put in. It looks like several very shrunken partial images side by side, with the remainder of the image space filled in in black. If I use asarray, then the entirety of python crashes during the raster step (on a random line). If I work with a greyscale image ('L'), then neither of these errors occurs for either array or asarray.

Does anyone know what I am doing wrong? Is there something odd about the way PIL encodes B&W images, or something special I need to pass numpy to make it convert properly?

Answer 1

Not sure about this line:

data = numpy.array(image)

In fact, that gives me a segfault. But I just tried the following, and it works fine:

import numpy
import Image

im = Image.open("some_photo.jpg")
im = im.convert("1")

pixels = im.getdata() # returns 1D list of pixels
n = len(pixels)
data = numpy.reshape(pixels, im.size) # turn into 2D numpy array

for row in data:
    # do your processing
    pass

# Check that the numpy array's data is good
im2 = Image.new("1", im.size)
im2.putdata(numpy.reshape(data, [n, 1]))
im2.show()

Answer 2

I believe you've found a bug in PIL! (or possibly in numpy, but I'd wager it's on the PIL side of things...)

@c's answer above gives one workaround (use im.getdata()), though I'm not sure why numpy.asarry(image) is segfaulting for him... (Old version of PIL and/or numpy, maybe?) It works for me, but produces gibberish on 1-bit PIL images (and works for everything else, I use it frequently!).

Another workaround is to convert the BW image back to grayscale (mode 'L') before converting to a numpy array.

Converting the BW image back to grayscale before converting to a numpy array seems to be faster, if speed is important.

In [35]: %timeit np.array(im_bw.convert('L')).astype(np.uint8)
10000 loops, best of 3: 28 us per loop

In [36]: %timeit np.reshape(im_bw.getdata(), im_bw.size)
10000 loops, best of 3: 57.3 us per loop

On a seperate note, if you're modifying the array contents in-place, be sure to use numpy.array instead of numpy.asarray, as the latter will create an array from the PIL image instance without copying memory, thus returning a read-only array. Just mentioning this because I'm using asarray() below...

Here's a standalone example which confirms the bug...

import numpy as np
import Image

x = np.arange(256, dtype=np.uint8).reshape((16,16))
print 'Created array'
print x

im = Image.fromarray(x)
print 'Vales in grayscale PIL image using numpy.asarray <-- Works as expected'
print np.asarray(im)

print 'Converted to BW PIL image...'
im_bw = im.convert('1')

print 'Values in BW PIL image, using Image.getdata() <-- Works as expected'
print '  (Not a simple threshold due to dithering...)'
# Dividing by 255 to make the comparison easier
print np.reshape(im_bw.getdata(), (16, 16)) / 255 

print 'Values in BW PIL image using numpy.asarray() <-- Unexpected!'
print '   (Same occurs when using numpy.array() to copy and convert.)'
print np.asarray(im_bw).astype(np.uint8) 

print 'Workaround, convert back to type "L" before array conversion'
print np.array(im_bw.convert('L')).astype(np.uint8) / 255

Which outputs:

Created array
[[  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15]
 [ 16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31]
 [ 32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47]
 [ 48  49  50  51  52  53  54  55  56  57  58  59  60  61  62  63]
 [ 64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79]
 [ 80  81  82  83  84  85  86  87  88  89  90  91  92  93  94  95]
 [ 96  97  98  99 100 101 102 103 104 105 106 107 108 109 110 111]
 [112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127]
 [128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143]
 [144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159]
 [160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175]
 [176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191]
 [192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207]
 [208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223]
 [224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239]
 [240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255]]

Vales in grayscale PIL image using numpy.asarray <-- Works as expected
[[  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15]
 [ 16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31]
 [ 32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47]
 [ 48  49  50  51  52  53  54  55  56  57  58  59  60  61  62  63]
 [ 64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79]
 [ 80  81  82  83  84  85  86  87  88  89  90  91  92  93  94  95]
 [ 96  97  98  99 100 101 102 103 104 105 106 107 108 109 110 111]
 [112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127]
 [128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143]
 [144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159]
 [160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175]
 [176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191]
 [192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207]
 [208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223]
 [224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239]
 [240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255]]

Converted to BW PIL image...

Values in BW PIL image, using Image.getdata() <-- Works as expected
  (Not a simple threshold due to dithering...)
[[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0]
 [0 0 1 0 1 0 1 0 1 0 0 0 1 0 0 0]
 [0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 1]
 [0 0 0 1 0 1 0 1 0 1 0 1 0 0 0 0]
 [1 0 1 0 1 0 1 0 1 0 0 0 1 1 0 1]
 [0 1 0 1 0 0 1 0 0 1 1 0 1 0 1 0]
 [1 0 1 0 1 0 1 1 0 1 0 1 0 1 0 1]
 [0 1 0 1 0 1 0 1 0 1 1 0 1 1 0 1]
 [1 1 0 1 1 1 1 0 1 1 0 1 1 0 1 1]
 [1 0 1 1 0 1 0 1 1 0 1 1 0 1 1 0]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 0 1 0 1 1 0 1 1 0 1 1 1 0 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]]

Values in BW PIL image using numpy.asarray() <-- Unexpected!
   (Same occurs when using numpy.array() to copy and convert.)
[[0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [0 1 1 1 1 1 1 0 0 0 0 0 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]]

Workaround, convert back to type "L" before array conversion
[[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0]
 [0 0 1 0 1 0 1 0 1 0 0 0 1 0 0 0]
 [0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 1]
 [0 0 0 1 0 1 0 1 0 1 0 1 0 0 0 0]
 [1 0 1 0 1 0 1 0 1 0 0 0 1 1 0 1]
 [0 1 0 1 0 0 1 0 0 1 1 0 1 0 1 0]
 [1 0 1 0 1 0 1 1 0 1 0 1 0 1 0 1]
 [0 1 0 1 0 1 0 1 0 1 1 0 1 1 0 1]
 [1 1 0 1 1 1 1 0 1 1 0 1 1 0 1 1]
 [1 0 1 1 0 1 0 1 1 0 1 1 0 1 1 0]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 0 1 0 1 1 0 1 1 0 1 1 1 0 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]]