"painting" one array onto another using python / numpy

Question

I'm writing a library to process gaze tracking in Python, and I'm rather new to the whole numpy / scipy world. Essentially, I'm looking to take an array of (x,y) values in time and "paint" some shape onto a canvas at those coordinates. For example, the shape might be a blurred circle.

The operation I have in mind is more or less identical to using the paintbrush tool in Photoshop.

I've got an interative algorithm that trims my "paintbrush" to be within the bounds of my image and adds each point to an accumulator image, but it's slow(!), and it seems like there's probably a fundamentally easier way to do this.

Any pointers as to where to start looking?

Answer 1

Have you looked into Tkinter?

Python Image Library may be some help too.

Answer 2

In your question you describe a Gaussian filter, for which scipy has support via a package. For example:

from scipy import * # rand
from pylab import * # figure, imshow
from scipy.ndimage import gaussian_filter

# random "image"
I = rand(100, 100)
figure(1)
imshow(I)

# gaussian filter
J = gaussian_filter(I, sigma=10)
figure(2)
imshow(J)

Of course, you can apply this on the whole image, or just on a patch, using slicing:

J = array(I) # copy image
J[30:70, 30:70] = gaussian_filter(I[30:70, 30:70], sigma=1) # apply filter to subregion
figure(2)
imshow(2)

For basic image manipulation, the Python Image library (PIL) is probably what you want.

NOTE: for "painting" with a "brush", I think you could just create a boolean mask array with your brush. For instance:

# 7x7 boolean mask with the "brush" (example: a _crude_ circle)
mask = array([[0, 0, 1, 1, 1, 0, 0],
              [0, 1, 1, 1, 1, 1, 0],
              [1, 1, 1, 1, 1, 1, 1],
              [1, 1, 1, 1, 1, 1, 1],
              [1, 1, 1, 1, 1, 1, 1],
              [0, 1, 1, 1, 1, 1, 0],
              [0, 0, 1, 1, 1, 0, 0]], dtype=bool)

# random image
I = rand(100, 100)
# apply filter only on mask
# compute the gauss. filter only on the 7x7 subregion, not the whole image
I[40:47, 40:47][mask] = gaussian_filter(I[40:47, 40:47][mask], sigma=1)

Answer 3

You should really look into Andrew Straw's motmot and libcamiface. He uses it for fly behaviour experiments but it's a flexible library for doing just the kind of image acquisition and processing you're doing I think. There's a video of his presentation at SciPy2009.

As for the paintbrush scenario you mention, I'd make a copy of the image with the .copy() method, keep the paintbrush image in an array, and simply add it with

arr[first_br_row:last_br_row, first_br_col:last_br_col] += brush[first_row:last_row, first_col:last_col]

where you set first_br_row, last_br_row first_br_col, last_br_col to address the subimage where you want to add the brush and first_row, last_row, first_col, last_col to clip the brush (normally set them to 0 and # rows/cols - 1, but adjust when you're near enough to the image boundary to only want to paint part of the brush).

Hope all that helps.

Answer 4

Doing a little of math in Fourier space may help: a translation (convolution by a dirac) is equal to a simple multiplication by a phase in Fourier... this makes your paintbrush move to the exact place (a similar solution than catchmeifyoutry & dwf, but this allows a translation finer than the pixel, like 2.5, alas with some ringing). Then, a sum of such strokes is the sum of these operations.

In code:

import numpy
import pylab
from scipy import mgrid

def FTfilter(image, FTfilter):
    from scipy.fftpack import fftn, fftshift, ifftn, ifftshift
    from scipy import real
    FTimage = fftshift(fftn(image)) * FTfilter
    return real(ifftn(ifftshift(FTimage)))

def translate(image, vec):
    """
    Translate image by vec (in pixels)

    """
    u = ((vec[0]+image.shape[0]/2) % image.shape[0]) - image.shape[0]/2
    v = ((vec[1]+image.shape[1]/2) % image.shape[1]) - image.shape[1]/2
    f_x, f_y = mgrid[-1:1:1j*image.shape[0], -1:1:1j*image.shape[1]]
    trans = numpy.exp(-1j*numpy.pi*(u*f_x + v*f_y))
    return FTfilter(image, trans)

def occlude(image, mask):
    # combine in oclusive mode
    return  numpy.max(numpy.dstack((image, mask)), axis=2)

if __name__ == '__main__':
    Image = numpy.random.rand(100, 100)
    X, Y = mgrid[-1:1:1j*Image.shape[0], -1:1:1j*Image.shape[1]]
    brush = X**2 + Y**2 < .05 # relative size of the brush
    # shows the brush
    pylab.imshow(brush)

    # move it to some other position  / use a threshold to avoid ringing
    brushed = translate(brush, [20, -10.51]) > .6
    pylab.imshow(brushed)

    pylab.imshow(occlude(Image, brushed))

    more_strokes = [[40, -15.1], [-40, -15.1], [-25, 15.1], [20, 10], [0, -10], [25, -10.51]]
    for stroke in more_strokes:
        brushed = brushed + translate(brush, stroke) > .6

    pylab.imshow(occlude(Image, brushed))