I'm looking for a packing algorithm which will reduce a regular polygon into rectangles and right triangles. The algorithm should attempt to use as few such shapes as possible and should be relatively easy to implement (given the difficulty of the challenge).
If possible, the answer to this question should explain the general heuristics used in the suggested algorithm.
It's not specifically rectangles + right triangles, but a good research point for looking into tesselating polygons is Voronoi Diagrams and Delaunay Triangulations and here and here.
In fact, if "just right triangles" is good enough, these are guaranteed to triangulate for you, and you can always split any triangle into two right triangles, if you really need those. Or you can chop off "tips" of triangles to make more right triangles and some rectangles out of the right-triangles.
You can also try ear-clipping, either by sweeping radially, if you know you have fairly regular polygons, or by "clipping the biggest convex chunk" off. Then, split each remaining triangle into two to create right triangles.
You could try to make less breaks by sweeping one way and then the other to make a trapezoid and split it differently, but you then have to do a check to make sure that your sweep-line hasn't crossed another line someplace. You can always ear-clip, even with something practically fractal.
However, this sometimes creates very slim triangles. You can perform heuristics, like "take the biggest", instead of clipping continuously along the edge, but that takes more time, approaching O(n^2). Delaunay/Vornoi will do it more quickly in most cases, with less slim triangles.
You can try "cuting out" the largest rectangle that can fit in the polygon, leaving behind some leftovers. Keep repeating the cutting out of rectangles on the leftovers, until you end up with triangular pieces. Then, you can split them into two right triangles if necessary. I do not know if this will always yield solutions that will give you the least amount rectangles and right triangles.
I think the answer is fairly simple for regular polygons.
Find an axis of symmetry, and draw a line between each vertex and its mirror. This divides the polygon into trapezoids. Each trapezoid can be turned into a rectangle and two right triangles.
