Soft Paint Bucket Fill: Colour Equality

Question

I'm making a small app where children can fill preset illustrations with colours. I've succesfully implemented an MS-paint style paint bucket using the flood fill argorithm. However, near the edges of image elements pixels are left unfilled, because the lines are anti-aliased. This is because the current condition on whether to fill is colourAtCurrentPixel == colourToReplace, which doesn't work on the blended pixels at the lines. (the colours are RGB uints)

I'd like to add a smoothing/treshold option like in Photoshop and other sophisticated tools, but what's the algorithm to determine the equality/distance between two colours?

if (match(pixel(x,y), colourToReplace) setpixel(x,y,colourToReplaceWith)

How to fill in match()?

Here, an image (left is situation, right is wanted)

Here's my current full code:

            var b:BitmapData = settings.background;
            b.lock();

            var from:uint = b.getPixel(x,y);


            var q:Array = [];


            var xx:int;
            var yy:int;
            var w:int = b.width;
            var h:int = b.height;
            q.push(y*w + x);
            while (q.length != 0) {
                var xy:int = q.shift();
                xx = xy % w;
                yy = (xy - xx) / w;
                if (b.getPixel(xx,yy) == from) { //<- want to replace this line
                    b.setPixel(xx,yy,to);
                    if (xx != 0) q.push(xy-1);
                    if (xx != w-1) q.push(xy+1);
                    if (yy != 0) q.push(xy-w);
                    if (yy != h-1) q.push(xy+w);
                }
            }
            b.unlock(null);

Answer 1

well, i guess the most natural approach is to calculate the difference between to colors. to achieve a sensible value, one should calculate the difference per channel. haven't tested it, but the following should work:

const perChanThreshold:uint = 5;
const overallThreshold:uint = perChanThreshold * perChanThreshold * 3;
function match(source:uint, target:uint):Boolean {
    var diff:uint = 0, chanDiff:uint;
    for (var i:int = 0; i < 3; i++) {
        chanDiff = (source >> (i * 8)) & 0xFF;
        diff += chanDiff * chanDiff;
    }
    return diff <= overallThreshold;
}

Answer 2

Made something that works:

                c = b.getPixel(xx,yy);
                if (c == to) continue;
                if (c != from) d = 
                    Math.pow(f1 - (c & 0xFF), 2) +
                    Math.pow(f2 - (c >> 8 & 0xFF), 2) +
                    Math.pow(f3 - (c >> 16 & 0xFF), 2)
                if (c == from || d < tres) {