I have two bitmaps, and I want to copy pixels from A to B only when the pixels are inside an area defined by four corners (a quadrangle). Bitmap A and B are the same size and the quadrangle is defined as four {x,y} coordinates in the pixel space of the image.
Worst case scenario I can test the center of each pixel against the quad to see if the pixel's center is inside the quad, but this is very slow. What is a better algorithm?
If the quadrilateral is convex, you can use this algorithm:
Short version: For each scanline (horizontal bitmap line), find the edges that intersect that scanline, and copy the pixels between them.
Long version: Go scanline by scanline. Start at the top vertex (minimum y), and keep track of the edges at the left and right sides. For each y value, calculate the x value of the two edges (either using the line equation directly, or using Bresenham's algorithm). Then copy the pixels at (xLeft,y) to (xRight,y) to the second bitmap.
When you reach a vertex at the end of a edge, switch to the other edge connected to that vertex. Continue this until reaching the bottom vertex.
For concave quads, this is more complicated. You can use a similar algorithm, but for some scanlines there will be four edges intersecting the scanline. In that case, you need to copy the pixels between edge #1 to #2, and #3 to #4 (with the edges ordered by x value). Another option is to separate the quad into two triangles, and use the above algorithm on them.